Building Services With gRPC, Docker and Go

Building
Services With
,
and !
Martin Kess ♥ Software Engineer
github.com/mdkess

Get the code:
github.com/namely/codecamp-2018-go

Agenda
● Background About Namely
● Why Services?
● Protobufs and gRPC - Defining Interfaces
● JSON
● Docker and Docker Compose
● Questions

About Namely
● Mission: Build A Better Workplace
● HR, Benefits and Payroll
● 1200 customers
● ~$1 billion in payroll/month
● ~100 engineers
● ~40 services, more shipping every week
● Polyglot environment: React, C# (.NET Core), Go, Ruby and Python
● Modern infrastructure: Kubernetes, Istio, AWS, Docker, Spinnaker.
● Big believers in open-source. We've contributed to the official gRPC C# repo. We
open source a lot of the tools we build.

What Are
(Micro)Services?
And Why Build Them, Anyway?

A service is software that...
● is the source of truth for its data.
● is independently deployable.
● prevents coupling through use of API
contracts.
● adds business value and open up new
opportunities.
● has a clear definition of availability (an SLO).

Domain Ownership
Services don't mean containers or AWS or Kubernetes. It
means that pieces of software that own their domain.
Services own the reads and writes for their data. Access to
this data should be done through APIs (not a shared DB).
Don't build a distributed monolith or you'll get all of the
weaknesses of services and none of the benefits.

Why Namely Uses Services
● In a monolith, teams ended up stepping on each others
feet.
○ Accidentally releasing each other team's features.
○ Big changes touching lots of code accidentally break things.
○ Unclear ownership of large parts of the codebase or data.
● Services make teams think in terms of API contracts.
● Teams can use language and tools of their choice.
● Give teams ownership and mastery of their domain.

Companies And Employees
A Company is a collection of Employee
objects and has an Office Location.
Every Employee has a name, works for a
Company and has a badge number.
Every Company has a CEO, who is also an
Employee.
Company
+ company_uuid: uuid
+ ceo_employee_uuid: uuid
+ office_location: Address
Employee
+ employee_uuid: uuid
+ company_uuid: uuid
+ name: string
+ badge_number: int32

A Problem
These models are almost certainly wrong.
Do all companies have a CEO? Do all companies have one CEO?
Do all companies have an office location? Do all companies have only one
office location? Are all companies based in America?
Do all employees have badge numbers? Is a single name field the best choice?
Of course not.

Good Software
Anticipates
Change

Anticipating Change
There is no perfect domain model, but our model might be good enough for
our current customers. Don't design for a future that might not exist. We want
to start with this model and iterate. But in doing so, some things to consider:
● What if you can’t force your old API clients to update?
● How do you release API clients and API servers separately?
○ Very important when doing slow rollouts of software.
● How do you avoid breaking updated API clients after a rollback?
● What if your data is stored on disk?
○ In a message queue, a file or a database.

Protocol Buffers
Use protocol buffers aka "protobufs"!
Message format invented by Google. Supports forward and backward
compatibility: newer servers can read messages from old clients and vice
versa.
A .proto file gets compiled into many languages (C#, Java, Go, Ruby, etc.)
Think fancy JSON with a schema.

A Simple Proto File
example.proto
Think of a message as a
C struct/POJO/POCO -
just data.
On each field in the
message is the field
number (i.e. = 4), this is
used when serializing
protos. It's not a (default)
value.
syntax = "proto3";
package examples;
message Employee {
string employee_uuid = 1;
string company_uuid = 2;
string name = 3;
int32 badge_number = 4;
}
message Address {
string address1 = 1;
string address2 = 2;
string zip = 3;
string state = 4;
}
message Company {
Address office_location = 2;
string ceo_employee_uuid = 3;
}

Compiling Protos
The protobuf compiler
turns protos into code for
your language.
On the right, we turn our
employee.proto from the
previous slide into Go
code.
Can also do C#, JS, Ruby,
Python, Java and many
other languages.
$ docker run
-v `pwd`:/defs namely/protoc-all
-f example.proto
-l go
The above command runs the docker container
namely/protoc-all to compile the example.proto
file into Go code and output the results to `pwd` (the
current directory).
$ ls
example.proto gen/
$ ls gen/pb-go/
example.pb.go

The Generated
Code
example.pb.go looks
something like the right.
This code is generated
automatically by the
namely/protoc-all
container.
Try running
namely/protoc-all with
-l python instead.
// Code generated by protoc-gen-go. DO NOT EDIT.
// source: example.proto
package examples
... snip ...
type Employee struct {
EmployeeUuid string
`protobuf:"bytes,1,opt,name=employee_uuid,json=employeeUuid"
json:"employee_uuid,omitempty"`
CompanyUuid string
`protobuf:"bytes,2,opt,name=company_uuid,json=companyUuid"
json:"company_uuid,omitempty"`
Name string
`protobuf:"bytes,3,opt,name=name" json:"name,omitempty"`
BadgeNumber int32
`protobuf:"varint,4,opt,name=badge_number,json=badgeNumber"
json:"badge_number,omitempty"`
}
func (m *Employee) Reset() { *m = Employee{} }
func (m *Employee) String() string { return
proto.CompactTextString(m) }
func (*Employee) ProtoMessage() {}
func (*Employee) Descriptor() ([]byte, []int) { return fileDescriptor0,
[]int{0} }
... snip ...

Great! But how do
protobufs help us
write services?

We need a way for our services to talk
to each other.
Remote Procedure Calls (RPCs) are
function calls that can be made over
the network.

is an open-source RPC framework for
building language agnostic servers and
clients that can talk to each other.
This means your Go/Ruby/C# client can talk
to your Python/Java/C++ server (and more).
It uses protocol buffers as its message
format.

Adding Services to
example.proto
You can also define
services in your proto file.
These get compiled to
gRPC servers and clients
that can speak protocol
buffers to each other.
You can write your server
and client in any
supported language.
service EmployeeService {
rpc CreateEmployee(CreateEmployeeRequest)
returns (Employee) {}
rpc ListEmployees(ListEmployeesRequest)
returns (ListEmployeesResponse) {}
}
message CreateEmployeeRequest {
Employee employee = 1;
}
message ListEmployeesRequest {
}
message ListEmployeesResponse {
repeated Employee employees = 1;
}

Application
Structure
The Company Service in
company/
The Employee Service in
employee/
The protobufs in
protos/
gen_protos.sh to
compile the protos
Check out the code!
$ git clone
$ ls
CODEOWNERS LICENSE README.md
docker-compose.yml
example.proto
gen_protos.sh
protos/
company/
employee/

Diving Into
Employee Service
Diving into
employee/main.go.
The main() function
listens on a TCP port,
creates a new gRPC
server and registers our
server interface to handle
gRPC calls
func main() {
flag.Parse()
lis, err := net.Listen("tcp",
fmt.Sprintf("0.0.0.0:%d", *port))
if err != nil {
log.Fatalf("error listening: %v", err)
}
server := grpc.NewServer()
pb.RegisterEmployeeServiceServer(
server, newServer())
server.Serve(lis)
}

The Employee
Server
The EmployeeServer
stores all of the
employees in memory.
For a real server you
would use a database.
It also creates a client
that talks to company
service to check that
companies exist.
type EmployeeServer struct {
companies map[string]*EmployeeCollection
conn *grpc.ClientConn
companyClient company_pb.CompanyServiceClient
}
func newServer() *EmployeeServer {
s := &EmployeeServer{}
s.companies =
make(map[string]*EmployeeCollection)
s.conn, _ = grpc.Dial(
*companyAddr, grpc.WithInsecure())
s.companyClient =
company_pb.NewCompanyServiceClient(s.conn)
return s
}

Looking at a
Handler
Let's look at the
CreateEmployee handler
It does three things:
1. Validate the input.
2. Call company service
to make sure the
company exists
3. Saves the employee.
This is the signature of the CreateEmployee function on
the EmployeeServer.
Input is:
- the call's context
- a CreateEmployeeRequest proto - the same one we
defined in our proto file earlier!
func (s *EmployeeServer)
CreateEmployee(
ctx context.Context,
req *employee_pb.CreateEmployeeRequest)
(*employee_pb.Employee, error) {
....
}
Input
parameters
Return
Type
(A tuple)

Looking at a
Handler
Let's look at the
to make sure the
company exists
Here we check that the employee's name is set. If not,
we return an Invalid Argument error to the client.
func (s *EmployeeServer) CreateEmployee(
// The employee must have a name.
if req.Employee.Name == "" {
return nil, status.Error(
codes.InvalidArgument, "employee must have name")
}
....
}

Looking at a
Handler
Let's look at the
to make sure the
company exists
Next we call CompanyService.GetCompany with a
GetCompanyRequest to check that the employee's
company exists.
....
_, err := s.companyClient.GetCompany(
ctx, &company_pb.GetCompanyRequest{
CompanyUuid: req.Employee.CompanyUuid,
})
if err != nil {
return nil, status.Error(
codes.InvalidArgument, "company does not exist")
}
....
}

Looking at a
Handler
Let's look at the
to make sure the
company exists
Finally, we save the employee and return the saved
employee to the caller. In our example, we just save it
in memory, but in real life you'd want to use some data
storage for this (i.e. a database).
....
// If we're here, we can save the employee.
return s.SaveEmployee(req.Employee), nil
}

Docker lets you build your applications into
container (which is sort of like a
lightweight virtual machine).
This makes it easy to distribute your
software and run it anywhere.
You make containers by writing a
Dockerfile.

Dockerfiles
Package your application
in a container that can be
run in various cloud
infrastructure.
Makes it easy to
distribute applications.
Here's the Dockerfile for
employees.
Try building this with
$ docker build -t company .
The above command builds the Dockerfile in the current
directory, and gives the resulting container the name
"company".
FROM golang:alpine AS build
RUN apk add --no-cache git
WORKDIR /go/src/github.com/namely/codecamp-2018-go/employee
COPY . .
RUN go get -d -v ./...
RUN go install -v ./...
FROM alpine
COPY --from=build /go/bin/employee /usr/local/bin/
CMD ["employee"]

Stitching
Applications With
Docker-Compose

Docker-Compose lets you run and configure
multiple docker containers.
It makes starting and stopping containers
easy.
It creates DNS names for your containers so
they can talk to each other.

docker-compose.yml
Defines two services
company and employee.
The build field tells
docker-compose how to
find your Dockerfile to
build your services.
version: "3.6"
services:
company:
build: ./company
command: company -port 50051
ports:
- 50051:50051
employee:
build: ./employee
command: >
employee -port=50051
-company_addr=company:50051
ports:
- 50052:50051
depends_on:
- company

Bringing Everything Up
Build your services with:
$ docker-compose build
And start them up (in the background) with
$ docker-compose up -d

Calling Your
Services With
gRPC CLI

Using the gRPC CLI
Namely provides a Docker container that contains the official gRPC CLI for
querying gRPC services. Get it with
$ docker pull namely/grpc-cli
Create some aliases to make calling it easier. docker.for.mac.localhost is how
the namely/grpc-cli reaches your local machine where the service is running.
$ alias company_call='docker run -v
`pwd`/protos/company:/defs --rm -it namely/grpc-cli
call docker.for.mac.localhost:50051'
$ alias employee_call='docker run -v
`pwd`/protos/employee:/defs --rm -it namely/grpc-cli
call docker.for.mac.localhost:50052'
docker.for.win.localhost
on Windows!

Creating a Company
Let's use the grpc_cli to call CompanyService.CreateCompany. We say
docker.for.mac.localhost to let the grpc-cli docker container find localhost on
your local machine (where we exposed the port in docker-compose)
$ company_call CompanyService.CreateCompany
"" --protofiles=company.proto
company_uuid: "3ac4f180-9410-467f-92b7-06763db0a8f1"

Creating an Employee
We'll take the company_uuid from the
$ employee_call EmployeeService.CreateEmployee
"employee: {name:'Martin',
company_uuid: '3ac4f180-9410-467f-92b7-06763db0a8f1'}"
--protofiles=employee.proto
employee_uuid: "10b286b2-247a-4864-afe5-f56163681af6"
name: "Martin"

Listing the Employees
$ employee_call EmployeeService.ListEmployees
"company_uuid: '3ac4f180-9410-467f-92b7-06763db0a8f1'"
--protofiles=employee.proto
employees {
employee_uuid: "3701a099-7c67-40b2-bfac-c0e627efd0f7"
name: "Martin"
}

What About
JSON?
The Web Doesn't Speak gRPC

Adding HTTP
Annotations
gRPC-Gateway creates an
HTTP-gRPC reverse proxy
for your gRPC server.
Annotations tell it how to
map HTTP requests to
your gRPC requests.
import "google/api/annotations.proto";
service CompanyService {
rpc CreateCompany(CreateCompanyRequest)
returns (Company) {
option (google.api.http) = {
put: "/companies/{company_uuid}",
body: "company"
};
}
...
}

HTTP gRPC
gRPCHTTP
gRPC
Server
gRPC
Gateway
Client
HTTP
Request

Just Kidding No New Code!
Just run Namely's Docker container to generate a new server
$ docker run -v `pwd`:/defs namely/gen-grpc-gateway
-f protos/company/company.proto -s CompaniesService
This generates a complete server in gen/grpc-gateway. Now build it. The
example repo has this in the docker-compose file as well.
$ docker build -t companies-gw
-f gen/grpc-gateway/Dockerfile gen/grpc-gateway/

Using cURL to try
our HTTP API
Let's wire these together
and use cURL to try out
our new API.
gRPC-Gateway makes it
easy to share your
services with a front-end
application.
Bring up our gateway
$ docker-compose up -d companies-gw companies
Create a company
$ curl -X POST
-d '{"office_location":{"address1":"foo"}}'
localhost:8082/companies
{"company_uuid":"d13ecefd-6b63-4919-9b33-e0006ee676ec"
,"office_location":{"address1":"foo"}}
Get that company
$ curl
localhost:8082/companies/d13ecefd-6b63-4919-9b33-e00
06ee676ec
{"company_uuid":"d13ecefd-6b63-4919-9b33-e0006ee676ec
","office_location":{"address1":"foo"}}
EASY!

Organizing Protos
Namely uses a monorepo
of all of our protobufs.
Services add this as a git
submodule.
This lets everyone stay up
to date. It also serves as
a central point for design
and API discussions.

What Did We Learn?
● How to build services with gRPC, Docker and Go.
● Thinking about services and how to get value from them.
● The importance of backward compatibility and how protobufs help.
● How to compile protobufs using namely/docker-protoc.
● How to use namely/grpc-cli to call your services.
● Using namely/gen-grpc-gateway to create HTTP services for your APIs.
● Use Docker to build your services into containers.
● Using Docker-Compose to bring up multiple containers.

Reducing
Boilerplate With
Interceptors

GRPC Interceptors
Interceptors let you catch calls before they get to the handlers, and before
they're returned to the client.
1 2
34
RPC
Handler
Interceptor
Client
RPC

GRPC Interceptors
An interceptor is a function with the signature:
func(ctx context.Context, // Info about the call (i.e. deadline)
req interface{}, // Request (i.e. CreateCompanyRequest)
info *grpc.UnaryServerInfo, // Server info (i.e. RPC method name)
handler grpc.UnaryHandler // Your handler for the RPC.
)
(resp interface{}, err error) // The response to send, or an error

A Typical
Interceptor
Interceptors let you do some
cool stuff:
1. Transform the request before
your handler gets it.
2. Transform the response
before the client sees it.
3. Add logging and other tooling
around the request without
having to copy-paste code in all
of your handlers.
func MetricsInterceptor(
ctx context.Context, req interface{},
info *grpc.UnaryServerInfo,
handler grpc.UnaryHandler)
(resp interface{}, err error) {
// Get the RPC name (i.e. "CreateCompany")
name := info.FullMethod
// Start a timer to see how long things take.
start := time.Now()
// Actually call the handler - your function.
out, err := handler(ctx, req)
// Check for errors
stat, ok := status.FromError(err)
// Log to our metrics system (maybe statsd)
LogMethodTime(name, start, time.Now(), stat)
// Return to the client. We could also change
// the response, perhaps by stripping out PII or
// doing error normalization/sanitization.
return out, err
}
} github.com/namely/codecamp-2018-go

Wiring It Together
grpc.ServerOption(
grpc.WithUnaryInterceptor(
MetricsInterceptor))
Then add the interceptor to the server start code.

Testing
Techniques
Services Are Hard

Mock Services
As you grow, you won't
want to bring up all of
your services.
With Go and Mockgen,
you can make your tests
act like a real service.

Combining Unit and Integration Tests
Your tests can be a hybrid of using unit testing techniques (Mocks) and
integration techniques.
Mock out some of the dependent services. This is very powerful when testing
gRPC servers since we can have tighter control over some dependencies.
Instead of bringing up everything, just bring up the dependencies in your
service's domain. For Employment, we bring up the database, but not
companies service.

Hybrid Integration
Tests
Bring up actual
implementations of main
services.
Use mocks for anything
out of the main flow that
are used for checks.
docker-compose run
--use-aliases
--service-ports
employment-tests
Employee
Tests
Employee
Service
Employees.CreateEmployee
Employee
DB
CompanyService.GetCompany
Calls your test (instead of the
real Company service) so that
you can control behavior.
Docker Compose
services:
# ... snip ...
employee-tests:
ports:
- 50052
environment:
- COMPANIES_PORT=50052
employee:
environment:
- COMPANIES_HOST=employee-tests
- COMPANIES_PORT=50052

Building Services With gRPC, Docker and Go

More Related Content

What's hot (9)

Similar to Building Services With gRPC, Docker and Go (20)

Recently uploaded (20)

Building Services With gRPC, Docker and Go