Finagle Your Own Codec - Scala By The Bay 2016
Download as PPTX, PDF2 likes483 views
The document provides an overview of Finagle, an extensible RPC system for the JVM, focusing on its protocol concepts and particularly how to implement Protocol Buffers within Finagle. It discusses various topics such as connection handling, error handling, client-server messaging, and implementation lessons, offering recommendations on building custom protocols. Ultimately, the document emphasizes the intricacies of handling errors and the importance of providing adequate support and documentation for developers using Finagle.
![Copyright 2016 Tendril, Inc. All rights reserved.
Interfaces
• What sort of interface to expose?
• Finagle interfaces – e.g. Service[Req,Resp]
• Your protocol
• Thrift - special compiler – Scrooge
• Protobuf - code generated by protoc
CONCEPTS
9Scalae By The Bay | November 11, 2016](https://image.slidesharecdn.com/sbtb-finagle-protobuf-161113195229/85/Finagle-Your-Own-Codec-Scala-By-The-Bay-2016-9-320.jpg)
![Copyright 2016 Tendril, Inc. All rights reserved.
Client Creation and Invocation
def buildEchoClient(port: Int, filters:SimpleFilter[ProtobufRequest,ProtobufResponse]*)
(implicit factory: RpcFactory, executorService: ExecutorService, tracer: Tracer) = {
val clientBuilder = ClientBuilder() …
val serviceStub = EchoService.newStub(null)
.asInstanceOf[ {def newStub(channel: RpcChannel): EchoService}]
factory.createStub( … )
}
val distributedEchoClient = buildEchoClient(distributedServerPort)
val controller = factory.createController()
val request = EchoRequest.newBuilder().setPhrase("Hello world”).build()
val callback = new RpcCallback[EchoResponse] { … }
distributedEchoClient.echo(controller, request, callback)
FINAGLE-PROTOBUF
18Scalae By The Bay | November 11, 2016](https://image.slidesharecdn.com/sbtb-finagle-protobuf-161113195229/85/Finagle-Your-Own-Codec-Scala-By-The-Bay-2016-18-320.jpg)
![Copyright 2016 Tendril, Inc. All rights reserved.
Client Creation and Dispatch
• RpcFactory – build stub
• Stub
• Generated by protoc, implements RPC service interface
• Delegates method calls to a channel
• RpcChannelImpl
• Uses ClientBuilder to build the Finagle Service[Request,Response]
• callMethod() delegates to the service
• When service future completes, calls or fails the callback
• Encoder
• Method code from hashed method name (SimpleMethodService)
FINAGLE-PROTOBUF
19Scalae By The Bay | November 11, 2016](https://image.slidesharecdn.com/sbtb-finagle-protobuf-161113195229/85/Finagle-Your-Own-Codec-Scala-By-The-Bay-2016-19-320.jpg)
![Copyright 2016 Tendril, Inc. All rights reserved.
Service Creation
def buildEchoServer(port: Int, impl: EchoService.Interface,
filters:SimpleFilter[ProtobufRequest,ProtobufResponse]*)
(implicit factory: RpcFactory, executorService: ExecutorService, tracer: Tracer) = {
val serverBuilder = ServerBuilder() ...
factory.createServer( … )
}
class ReverseEchoService extends EchoService.Interface {
def echo(controller: RpcController, request: EchoRequest, callback: RpcCallback[EchoResponse]) {
val response = EchoResponse.newBuilder.setPhrase(request.getPhrase.reverse).build
callback.run(response)
}
}
FINAGLE-PROTOBUF
20Scalae By The Bay | November 11, 2016](https://image.slidesharecdn.com/sbtb-finagle-protobuf-161113195229/85/Finagle-Your-Own-Codec-Scala-By-The-Bay-2016-20-320.jpg)
![Copyright 2016 Tendril, Inc. All rights reserved.
Service Creation and Dispatch
• RpcFactory – build service
• Service implementation – implements protoc generated interface
• ServiceDispatcher
• Determine which method to call
• Call method and fulfill Promise (Future returned to Finagle)
• RpcControl – failure and cancellation
• RpcCallback[T] – run method
• One instance for [Response], one for [Throwable]
• In practice we wrap this behind Guava ListenableFuture interface
FINAGLE-PROTOBUF
21Scalae By The Bay | November 11, 2016](https://image.slidesharecdn.com/sbtb-finagle-protobuf-161113195229/85/Finagle-Your-Own-Codec-Scala-By-The-Bay-2016-21-320.jpg)
Finagle Your Own Codec - Scala By The Bay 2016
- 1. Copyright 2016 Tendril, Inc. All rights reserved. FINAGLE YOUR OWN CODEC Extending Finagle with Protocol Buffers
- 2. Copyright 2016 Tendril, Inc. All rights reserved. Agenda INTRODUCTION 2 Finagle Protocol Concepts Finagle Protobuf Lessons and Recommendations Scalae By The Bay | November 11, 2016
- 3. Copyright 2016 Tendril, Inc. All rights reserved. FINAGLE PROTOCOL CONCEPTS 3
- 4. Copyright 2016 Tendril, Inc. All rights reserved. WHAT IS FINAGLE ANYWAY? Finagle is an extensible RPC system for the JVM, used to construct high-concurrency servers. … Most of Finagle’s code is protocol agnostic, simplifying the implementation of new protocols. http://twitter.github.io/finagle/ (emphasis mine) 4Scalae By The Bay | November 11, 2016
- 5. Copyright 2016 Tendril, Inc. All rights reserved. Why RPC and Binary Formats? • RPC vs REST • Operation-oriented • No mapping to fixed verbs • Binary formats vs JSON • Smaller message size • Serialization performance • Versioning and other semantics • Interface Definition Language (IDL) • Available operations • Message schema CONCEPTS 5Scalae By The Bay | November 11, 2016
- 6. Copyright 2016 Tendril, Inc. All rights reserved. What is a Protocol? • Merriam-Webster a set of conventions governing the treatment and especially the formatting of data in an electronic communications system • A protocol includes • Codec • Dispatchers • Client and server configuration and initialization • Error handling • Integration with code generators or compilers CONCEPTS 6Scalae By The Bay | November 11, 2016
- 7. Copyright 2016 Tendril, Inc. All rights reserved. What is a Codec • Codec trait • Defines encoding/decoding • Interface to Netty channel pipeline • Modify the service filter stack • Server and Client sides • Symmetric – same objects both sides? • Request and Response – same encoding both ways? • CodecFactory relates client and server CONCEPTS 7Scalae By The Bay | November 11, 2016
- 8. Copyright 2016 Tendril, Inc. All rights reserved. End to End Flow CONCEPTS 8Scalae By The Bay | November 11, 2016 Client Client Dispatch Encoder Finagle/Net ty Decoder Service Dispatch Service Impl
- 9. Copyright 2016 Tendril, Inc. All rights reserved. Interfaces • What sort of interface to expose? • Finagle interfaces – e.g. Service[Req,Resp] • Your protocol • Thrift - special compiler – Scrooge • Protobuf - code generated by protoc CONCEPTS 9Scalae By The Bay | November 11, 2016
- 10. Copyright 2016 Tendril, Inc. All rights reserved. Connection Handling • Connection pooling • Connections created for requests • One connection == one active request • Reuse only after a request is done • Configuration of pool size limits, Etc • Mux (connection multiplexing) • One connection used for many simultaneous requests • Match responses up with open requests • Simpler configuration model CONCEPTS 10Scalae By The Bay | November 11, 2016
- 11. Copyright 2016 Tendril, Inc. All rights reserved. Service Stack • Finagle services are created from a stack of components • Timeouts • Stats • Stack API • Flexibility but complexity • ClientBuilder / ServerBuilder • Simpler but limited • Provides basic stack • ClientBuilder and ServerBuilder will be deprecated eventually CONCEPTS 11Scalae By The Bay | November 11, 2016
- 12. Copyright 2016 Tendril, Inc. All rights reserved. Error Handling – Two Types of Errors • Application • Something gone wrong in the service • Exception • Non-exceptional fail response • Framework • Timeouts • Rejections • Uncaught exceptions CONCEPTS 12Scalae By The Bay | November 11, 2016
- 13. Copyright 2016 Tendril, Inc. All rights reserved. Error Handling – Ways to Handle • Protocol • E.g. Thrift has an exception type • Transport • Error message is a certain type on the wire • Application • User messages must include some room for errors • How should exceptions be caught and mapped on the server? • How should they map back on the client? • Same mechanism or different mechanisms for client and server? CONCEPTS 13Scalae By The Bay | November 11, 2016
- 14. Copyright 2016 Tendril, Inc. All rights reserved. FINAGLE-PROTOBUF 14
- 15. Copyright 2016 Tendril, Inc. All rights reserved. Protobuf Message IDL message Echo { optional string phrase = 1; optional uint32 offset = 2; optional com.tendril.platform.common.Context context = 3; } val echoMessage = Echo.newBuilder .setPhrase("myPhrase") .setOffset(0) .build echoMessage.hasContext should be(false) FINAGLE-PROTOBUF 15Scalae By The Bay | November 11, 2016
- 16. Copyright 2016 Tendril, Inc. All rights reserved. Protobuf Service IDL service EchoService { rpc Echo (EchoRequest) returns (EchoResponse); } public static abstract class EchoService implements com.google.protobuf.Service { … public interface Interface { public abstract void echo( com.google.protobuf.RpcController controller, Echo.EchoRequest request, com.google.protobuf.RpcCallback<Echo.EchoResponse> done); } … } FINAGLE-PROTOBUF 16Scalae By The Bay | November 11, 2016
- 17. Copyright 2016 Tendril, Inc. All rights reserved. Request and Response message EchoRequest { optional string phrase = 1; } message EchoResponse { optional com.tendril.platform.common.Error error = 1; optional Echo echo = 2; } FINAGLE-PROTOBUF 17Scalae By The Bay | November 11, 2016
- 18. Copyright 2016 Tendril, Inc. All rights reserved. Client Creation and Invocation def buildEchoClient(port: Int, filters:SimpleFilter[ProtobufRequest,ProtobufResponse]*) (implicit factory: RpcFactory, executorService: ExecutorService, tracer: Tracer) = { val clientBuilder = ClientBuilder() … val serviceStub = EchoService.newStub(null) .asInstanceOf[ {def newStub(channel: RpcChannel): EchoService}] factory.createStub( … ) } val distributedEchoClient = buildEchoClient(distributedServerPort) val controller = factory.createController() val request = EchoRequest.newBuilder().setPhrase("Hello world”).build() val callback = new RpcCallback[EchoResponse] { … } distributedEchoClient.echo(controller, request, callback) FINAGLE-PROTOBUF 18Scalae By The Bay | November 11, 2016
- 19. Copyright 2016 Tendril, Inc. All rights reserved. Client Creation and Dispatch • RpcFactory – build stub • Stub • Generated by protoc, implements RPC service interface • Delegates method calls to a channel • RpcChannelImpl • Uses ClientBuilder to build the Finagle Service[Request,Response] • callMethod() delegates to the service • When service future completes, calls or fails the callback • Encoder • Method code from hashed method name (SimpleMethodService) FINAGLE-PROTOBUF 19Scalae By The Bay | November 11, 2016
- 20. Copyright 2016 Tendril, Inc. All rights reserved. Service Creation def buildEchoServer(port: Int, impl: EchoService.Interface, filters:SimpleFilter[ProtobufRequest,ProtobufResponse]*) (implicit factory: RpcFactory, executorService: ExecutorService, tracer: Tracer) = { val serverBuilder = ServerBuilder() ... factory.createServer( … ) } class ReverseEchoService extends EchoService.Interface { def echo(controller: RpcController, request: EchoRequest, callback: RpcCallback[EchoResponse]) { val response = EchoResponse.newBuilder.setPhrase(request.getPhrase.reverse).build callback.run(response) } } FINAGLE-PROTOBUF 20Scalae By The Bay | November 11, 2016
- 21. Copyright 2016 Tendril, Inc. All rights reserved. Service Creation and Dispatch • RpcFactory – build service • Service implementation – implements protoc generated interface • ServiceDispatcher • Determine which method to call • Call method and fulfill Promise (Future returned to Finagle) • RpcControl – failure and cancellation • RpcCallback[T] – run method • One instance for [Response], one for [Throwable] • In practice we wrap this behind Guava ListenableFuture interface FINAGLE-PROTOBUF 21Scalae By The Bay | November 11, 2016
- 22. Copyright 2016 Tendril, Inc. All rights reserved. Wire Formats • Wire Format Version 0 • Wire Format Version 1 FINAGLE-PROTOBUF 22Scalae By The Bay | November 11, 2016 Method code Message length Protobuf message Trace span Protobuf message TracingMarker Span Parent span Method Code Message length Version
- 23. Copyright 2016 Tendril, Inc. All rights reserved. Version Detection • Service is configured V0 or V1 on startup • V1 service can handle V0 or V1 frames • Detects version based on version and marker fields • Client configured as V0 or V1 • All responses V0 • No update to clients for response version detection • Backwards compatibility maintained FINAGLE-PROTOBUF 23Scalae By The Bay | November 11, 2016
- 24. Copyright 2016 Tendril, Inc. All rights reserved. Error Handling and Mapping message Error { optional uint32 code = 1; optional string message = 2; } • ServiceExceptionHandler • Service-side (ServiceDispatcher) • Maps exception to message • ExceptionResponseHandler • Client-side (RpcChannelImpl) • Maps message to exception FINAGLE-PROTOBUF 24Scalae By The Bay | November 11, 2016
- 25. Copyright 2016 Tendril, Inc. All rights reserved. LESSONS AND RECOMMENDATIONS 25
- 26. Copyright 2016 Tendril, Inc. All rights reserved. Why build your own protocol • One-side support (e.g. HTTP) • Legacy or other interop • Wrapping another protocol (e.g. MySQL, Redis) • Use existing protocols (e.g. Thrift) otherwise LESSONS 26Scalae By The Bay | November 11, 2016
- 27. Copyright 2016 Tendril, Inc. All rights reserved. Error Handling • Tricky to get right, tricky to test • Prefer transport or in-protocol handling • Avoid forcing a message format LESSONS 27Scalae By The Bay | November 11, 2016
- 28. Copyright 2016 Tendril, Inc. All rights reserved. Support • Community is solidly Thrift • Twitter support is mostly Mux • If you go another way you’re doing a lot of work on your own • If you build on Mux you get many Twitter enhancements “for free” LESSONS 28Scalae By The Bay | November 11, 2016
- 29. Copyright 2016 Tendril, Inc. All rights reserved. Adapt or Adopt • Generated interfaces • Generated interface or Finagle interface • Existing generator or roll your own • Hide or expose Twitter Futures, Finagle? • Façade approaches are very tricky • Mapping or leaky abstractions • Hand-rolled or generated code ?? • More layers and more magic == fewer experts on your team LESSONS 29Scalae By The Bay | November 11, 2016
- 30. Copyright 2016 Tendril, Inc. All rights reserved. Interfaces and Implementations • Your protocol defines interfaces • Implementations supplied in your libraries? • Default implementations? • Example implementations? • Good docs? LESSONS 30Scalae By The Bay | November 11, 2016
- 31. Copyright 2016 Tendril, Inc. All rights reserved. Make things easy for your developers • Examples • Testbeds • Seed projects LESSONS 31Scalae By The Bay | November 11, 2016
- 32. Copyright 2016 Tendril, Inc. All rights reserved. Learning about Finagle • Configuration settings • Diagnosing failures • What behavior is from Finagle? • What behavior is from your protocol? LESSONS 32Scalae By The Bay | November 11, 2016
- 33. Copyright 2016 Tendril, Inc. All rights reserved. Things to Check Out • ScalaPB • http://trueaccord.github.io/ScalaPB/ • “Spark and Protocol Buffers – An Awesome Combination” • Nadav Samet, Saturday 9:50 • Finagle Serial • https://github.com/finagle/finagle-serial • Finagle Protobuf • https://github.com/finagle/finagle-protobuf LESSONS 33Scalae By The Bay | November 11, 2016
Editor's Notes
- #6: Finagle doesn’t preclude HTTP/REST, or JSON message bodies. It’s protocol agnostic. But it does enable non-rest protocols as well Binary formats great for very chatty services Other semantics – field types, enums, error type in thrift, structural things (one of, embedded message w/out pre-knowledge)
- #7: Twitter uses the term without definition in finagle docs For our purposes we can group the following into a “protocol”
- #8: We recommend that you use the same objects on both client and server (
- #9: Client dispatch converts from the client-side interface to the information needed for the call. Encoder converts for wire. Decoder converts back to objects. Service dispatch determines what to call and calls it. On return, service dispatch would need to catch and handle errors, client dispatch handles errors and executes callbacks (resolves Promises)
- #10: Your protocol interface – e.g. a fixed interface for the capabilities you support A generated interface from IDL etc Some legacy interface
- #12: Stack docs say “Stacks are advanced and sometimes subtle. For expert use only!”
- #14: Transport Message on the wire may be e.g. known message type
- #16: Optional, required, repeated
- #17: - What gets generated for rpc? - Interface - ReflectiveService (self describing/executable api - BlockingStub - Stub (channel based) - Abstract class
- #18: As we implement these Note the error which we’ll talk about later We have done this this way in part to separate the domain object Echo from the request and response envelopes
- #20: Dispatching services and fitting protobuf shape RpcFactory to create a stub Stub extends the protobuf service interface Stub constructed with RpcChannelImpl instance RpcChannelImpl on construction uses clientBuilder to build the client Service Calling a method invokes callMethod() which invokes the service On response calls callback or marks controller resolved Stub -> RpcChannelImpl and ServiceDispatch Server-side:
- #22: Successful result – run callback Failure – attempt to handle
- #24: We wanted to allow new services to support new or existing clients. Did not want to have to update the clients. Could have done different response format based on detected request version.
- #30: Recommend exposing finagle and/or your protocol. Minimize façade layers We constrained ourselves by deciding our interface before creating finagle-protobuf, rather than letting finagle influence the interface shape
- #31: Recommend default or at least example implementations