Driving containerd operations with gRPC
12 likes10,755 views
containerd uses gRPC and protocol buffers to define its API. gRPC provides benefits like code generation, performance, and common standards. The containerd gRPC services include Execution, Shim, and Content, which define methods like Create, Start, and List. Protocol buffer definitions generate client and server code in various languages. Developers can build clients and extend containerd with new services. The containerd API is under active development and aims to stabilize before the 1.0 release while allowing backwards compatibility through gRPC versioning.
Driving containerd operations with gRPC
- 1. An API Intro: Driving containerd operations with gRPC
- 2. WHO AM I? Phil Estes Senior Technical Staff, Office of the CTO, IBM Cloud Maintainer, Docker engine Maintainer, containerd Contributor, OCI/runc Docker Captains program member Blog: https://integratedcode.us Twitter: @estesp
- 4. YOU PROBABLY ALREADY HAVE AN IDEA OF WHAT A CLIENT IS ▸ Command-line client (e.g. runc) ▸ REST/HTTP client (e.g. docker -> dockerd) ▸ Web/HTTP client (e.g. curl, your web browser) gRPC is simply a client-server method execution model built on the long-standing “remote procedure call” (RPC) style interface. gRPC is an open source project created by Google and a full toolkit is available for many platforms: https://github.com/grpc/grpc | https://grpc.io
- 5. “In gRPC a client application can directly call methods on a server application on a different machine as if it was a local object, making it easier for you to create distributed applications and services. As in many RPC systems, gRPC is based around the idea of defining a service, specifying the methods that can be called remotely with their parameters and return types. FROM: http://www.grpc.io/docs/guides/ :
- 6. IMPORTANT CONCEPTS: SERVICES & PROTOCOL BUFFERS ▸ A gRPC API has service definitions. ▸ These service definitions have rpc methods. ▸ These rpc methods have “protobuf” defined message content (request & response)
- 7. “Protocol buffers are a flexible, efficient, automated mechanism for serializing structured data – think XML, but smaller, faster, and simpler. You define how you want your data to be structured once, then you can use special generated source code to easily write and read your structured data to and from a variety of data streams and using a variety of languages. You can even update your data structure without breaking deployed programs that are compiled against the "old" format. FROM: https://developers.google.com/protocol-buffers/docs/overview :
- 8. OK, FINE, BUT WHY gRPC+PROTOBUFS OVER JSON+HTTP/etc? Code Generation No more writing clients by hand! Generators convert .proto files into language-specific bindings for client and server. Speed & Size Protobuf binary packed format marshalled from clients<-->servers is smaller than JSON or XML and better performing for serialization & deserialization. Commonality gRPC and protobufs are growing in popularity. gRPC is used in Docker Swarm, Kubernetes, etcd, and adopted by companies like CoreOS, Netflix, Square and others.
- 9. MORE gRPC BENEFITS ▸ gRPC Connection support & options are rich and completely built-in to the implementation ▹ Less time writing or finding server code to handle special socket options, HTTP2, authentication, streaming, etc. gRPC Connect Example:
- 11. MAJOR API COMPONENTS ▸ Current gRPC services: ▹ Execution, Shim, and Content ▸ Go 1.8 provides plugin support ▹ Expect gRPC services to be pluggable Execution - Create - Start - Delete - List - Events Shim - Create - Start - Delete - Exec - Pty - Events - State - Pause - Resume - Exit Content - Info - Read - Status - Write
- 12. SERVICE API/PROTOBUF DEFINITIONS ▸ Each service has a .proto file with all definitions ▸ Common definitions (e.g. types) can be included ▸ Protobuf compiler generates appropriate language binding (e.g. Go code) api/services/shim/shim.proto
- 13. CODE GENERATION: FROM PROTO -> GO SOURCES *WARNING: NOT FOR THE FAINT OF HEART! commit 83e7610194170c794752088d54b0c3673b311811 Author: Stephen J Day <stephen.day@docker.com> Date: Tue Feb 14 16:48:36 2017 -0800 cmd/ctrd-protobuild: create proper command for building protos After trying to explain the complexities of developing with protobuf, I have now created a command that correctly calculates the import paths for each package and runs the protobuf command. ▸ make protos uses go generate tooling* ▸ <service>.pb.go files are created with the Go bindings ▸ Only required when changing (add/remove/modify) API or message/type definitions
- 14. 3. Create your own gRPC service and/or client
- 15. `CTR LIST` AS AN EXAMPLE: SERVER SIDE (GENERATED) ▸ Generated execution.pb.go has interface for server as well as handler which calls List method func _ContainerService_List_Handler(srv interface{}, ctx context.Context, dec func(interface{}) error, interceptor grpc.UnaryServerInterceptor) (interface{}, error) { … return srv.(ContainerServiceServer).List(ctx, in) … } api/services/execution/execution.pb.go
- 16. `CTR LIST` AS AN EXAMPLE: SERVER SIDE (IMPLEMENTED) ▸ We implement the ContainerServiceServer.List method to perform the operation and return the appropriate gRPC message response:
- 17. `CTR LIST` AS AN EXAMPLE: CLIENT SIDE ▸ The client code to call the gRPC method is already created for us during protobuf generation ▸ We only need to connect our client command to this generated client stub: var listCommand = cli.Command{ Name: "list", Action: func(context *cli.Context) error { containers, err := getExecutionService(context) response, err := containers.List(gocontext.Background(), &execution.ListRequest{}) for _, c := range response.Containers { // print container information } return nil }, } cmd/ctr/list.go
- 18. TAKING THE NEXT STEP CONSUME CONTAINERD VIA gRPC CLIENT HELP IMPROVE/DEFINE NEW AREAS OF CORE CONTAINERD API ADD SERVICE VIA GO 1.8 PLUGINS + gRPC PROTO SERVICE DEFINITION
- 19. 4. Summary Expected that consumers of containerd will use the gRPC API. The ctr tool is a development/admin resource. gRPC benefits outweigh any added complexity with protobufs for defining and implementing the API layer. The API is under heavy development and will stabilize leading up to the 1.0 release of containerd. Post 1.0, gRPC versioning will allow us to retain backwards compatibility.
- 20. CREDITS Special thanks to all the people who made and released these awesome resources for free: ▸ Simple line icons by Mirko Monti ▸ E-commerce icons by Virgil Pana ▸ Streamline iconset by Webalys ▸ Presentation template by SlidesCarnival