"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition

Alex Borysov
Software Engineer
Enabling Googley microservices with gRPC.
A high performance, open source, HTTP/2-based RPC framework.
Voxxed Days Minsk, May 26, 2018

Google Cloud Platform
Alex Borysov
• Software engineer / tech lead experienced in large scale
software development.
• 12+ years of software engineering experience.
• Active gRPC user.
• devoxx.org.ua program committee member.
@aiborisov

3
devoxx.org.ua
November 23-24, 2018
Kyiv, Ukraine
Save the Date!
#VoxxedDaysMinsk
@aiborisov

What is gRPC?
#VoxxedDaysMinsk
@aiborisov

What is gRPC?
A high performance, general purpose, feature-rich
RPC framework.
Part of Cloud Native Computing Foundation cncf.io.
HTTP/2 and mobile first.
Open source version of Stubby RPC used in Google.
#VoxxedDaysMinsk
@aiborisov

g in gRPC stands for
#VoxxedDaysMinsk
@aiborisov
1.0 'gRPC'
1.1 'good'
1.2 'green'
1.3 'gentle'
. . .
1.9 'glossy'
1.10 'glamorous'
1.11 'gorgeous'
1.12 'glorious'
https://github.com/grpc/grpc/blob/master/doc/g_stands_for.md

RPC?! But...
#VoxxedDaysMinsk
@aiborisov
AMF?
RMI?
CORBA?

gRPC is not RMI
#VoxxedDaysMinsk
@aiborisov
Maintainability
Ease of use
Performance
Scalability
L
e
s
s
o
n
s
Years of using Stubby

What is gRPC?
Abstractions and best practices on how to design
distributed systems.
Default implementation(s) from Google.
Extension points to plug custom implementations and
modifications.
Supports 10+ programming languages
#VoxxedDaysMinsk
@aiborisov

What is gRPC?
Abstractions and best practices on how to design
distributed systems.
Default implementation(s) from Google.
Extension points to plug custom implementations and
modifications.
Supports 10+ programming languages, including JS !
#VoxxedDaysMinsk
@aiborisov

gRPC vs JSON/HTTP for Google Cloud Pub/Sub
3x increase in throughput
https://cloud.google.com/blog/big-data/2016/03/announcing-grpc-alpha-for-google-cloud-pubsub
11x difference per CPU
#VoxxedDaysMinsk
@aiborisov

Google ~O(1010
) QPS.
#VoxxedDaysMinsk
@aiborisov

Google ~O(1010
) QPS.
Your project QPS?
#VoxxedDaysMinsk
@aiborisov

Continuous Performance Benchmarking
http://www.grpc.io/docs/guides/benchmarking.html
#VoxxedDaysMinsk
@aiborisov
8 core VMs, unary throughput

#VoxxedDaysMinsk
@aiborisov
32 core VMs, unary throughput

• Single TCP connection.
• No Head-of-line blocking.
• Binary framing layer.
• Header Compression.
HTTP/2 in One Slide
Transport(TCP)
Application (HTTP/2)
Network (IP)
Session (TLS) [optional]
Binary Framing
HEADERS Frame
DATA Frame
HTTP/2
POST: /upload
HTTP/1.1
Host: www.voxxeddays.com
Content-Type: application/json
Content-Length: 27
HTTP/1.x
{“msg”: “Welcome to 2018!”}
#VoxxedDaysMinsk
@aiborisov

HTTP/1.x vs HTTP/2
http://www.http2demo.io/
https://http2.golang.org/gophertiles
#VoxxedDaysMinsk
@aiborisov
HTTP/1.1 HTTP/2

Okay, but how?
#VoxxedDaysMinsk
@aiborisov

Service Definition (weather.proto)
#VoxxedDaysMinsk
@aiborisov

syntax = "proto3";
option java_multiple_files = true;
option java_package = "com.voxxeddays.grpcexample" ;
package com.voxxeddays.grpcexample;
service WeatherService {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
#VoxxedDaysMinsk
@aiborisov

syntax = "proto3";
}
message WeatherRequest {
Coordinates coordinates = 1;
message Coordinates {
fixed64 latitude = 1;
fixed64 longitude = 2;
}
}
message WeatherResponse {
Temperature temperature = 1;
Humidity humidity = 2;
}
message Temperature {
float degrees = 1;
Units units = 2;
enum Units {
FAHRENHEIT = 0;
CELSIUS = 1;
}
}
message Humidity {
float value = 1;
}
#VoxxedDaysMinsk
@aiborisov

#VoxxedDaysMinsk
@aiborisov
Service Definition
weather.proto
gRPC Runtime

#VoxxedDaysMinsk
@aiborisov
Service Definition
weather.proto
WeatherServiceImplBase.java WeatherRequest.java
WeatherResponse.java
WeatherServiceStub.java
WeatherServiceFutureStub.java
WeatherServiceBlockingStub.java
Service Implementation Request and response Client libraries
gRPC Java runtime

Implement gRPC Service
public class WeatherService extends WeatherServiceImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
}
}
#VoxxedDaysMinsk
@aiborisov

#VoxxedDaysMinsk
@aiborisov
Thread #X
Thread ...
Thread #7
Thread #6
Thread #5
Thread #4
Thread #3
Thread #2
Thread #1
Thread pool
of size X
Blocking API (aka Thread-per-Request)?

#VoxxedDaysMinsk
@aiborisov
Thread #X
Thread ...
Thread #7
Thread #6
Thread #5
Thread #4
Thread #3
Thread #2
Thread #1
Thread pool
of size X
1

#VoxxedDaysMinsk
@aiborisov
Thread #X
Thread ...
Thread #7
Thread #6
Thread #5
Thread #4
Thread #3
Thread #2
Thread #1
Thread pool
of size X
1
2

#VoxxedDaysMinsk
@aiborisov
Thread #X
Thread ...
Thread #7
Thread #6
Thread #5
Thread #4
Thread #3
Thread #2
Thread #1
Thread pool
of size X
1
2
3

#VoxxedDaysMinsk
@aiborisov
Thread #X
Thread ...
Thread #7
Thread #6
Thread #5
Thread #4
Thread #3
Thread #2
Thread #1
Thread pool
of size X
1
2
3
X
. . .

#VoxxedDaysMinsk
@aiborisov
Thread #X
Thread ...
Thread #7
Thread #6
Thread #5
Thread #4
Thread #3
Thread #2
Thread #1
Thread pool
of size X
1
2
3
X
. . .
X + 1

@Override
public interface StreamObserver<WeatherResponse> responseObserver){{
void onNext(WeatherResponse response);
void onCompleted();
void onError(Throwable error);
}
}
}
#VoxxedDaysMinsk
@aiborisov

Non-Blocking API
#VoxxedDaysMinsk
@aiborisov
1
2
3
. . .
X + Y
X
. . .

Non-Blocking API (aka Hollywood Principle)
#VoxxedDaysMinsk
@aiborisov
1
2
3
. . .
X + Y
X
. . .
WillCallYouLater!

Non-Blocking API
#VoxxedDaysMinsk
@aiborisov
1
2
3
. . .
X + Y
X
. . .
onNext(1)
WillCallYouLater!

Non-Blocking API
#VoxxedDaysMinsk
@aiborisov
1
2
3
. . .
X + Y
X
. . .
onNext(3)
WillCallYouLater!

Non-Blocking API
#VoxxedDaysMinsk
@aiborisov
1
2
3
. . .
X + Y
X
. . .
onNext(X)
WillCallYouLater!

@Override
WeatherResponse response = WeatherResponse.newBuilder()
.setTemperature(Temperature.newBuilder().setUnits(CELSUIS).setDegrees(20.f))
.setHumidity(Humidity.newBuilder().setValue(.65f))
.build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
#VoxxedDaysMinsk
@aiborisov

Start gRPC Server
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(new WeatherService()).build()
.start();
#VoxxedDaysMinsk
@aiborisov

gRPC Clients
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherServiceStub client = WeatherServiceGrpc.newStub(grpcChannel);
#VoxxedDaysMinsk
@aiborisov

gRPC Clients
WeatherServiceStub client = WeatherServiceGrpc.newStub(grpcChannel);
WeatherServiceBlockingStub blockingClient = WeatherServiceGrpc.newBlockingStub(grpcChannel);
WeatherServiceFutureStub futureClient = WeatherServiceGrpc.newFutureStub(grpcChannel);
#VoxxedDaysMinsk
@aiborisov

gRPC Sync Client
WeatherRequest request = WeatherRequest.newBuilder()
.setCoordinates(Coordinates.newBuilder().setLatitude(420000000)
.setLongitude(-720000000)).build();
WeatherResponse response = blockingClient.getCurrent(request);
logger.info("Current weather for {}: {}", request, response);
#VoxxedDaysMinsk
@aiborisov

gRPC Async Client
WeatherRequest request = WeatherRequest.newBuilder()
.setCoordinates(Coordinates.newBuilder().setLatitude(504316220)
.setLongitude(305166450)).build();
client.getCurrent(request, new StreamObserver<WeatherResponse>() {
@Override
public void onNext(WeatherResponse response) {
logger.info("Current weather for {}: {}", request, response);
}
@Override
public void onError(Throwable t) { logger.info("Cannot get weather for {}", request); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
#VoxxedDaysMinsk
@aiborisov

public class WeatherService extends WeatherServiceGrpc.WeatherServiceImplBase {
@Override
.setTemperature(Temperature.newBuilder().setUnits(CELSUIS).setDegrees(20.f))
.setHumidity(Humidity.newBuilder().setValue(.65f))
.build();
}
}
#VoxxedDaysMinsk
@aiborisov

Adding New [Micro]Services
Weather
Weather
Temp
Wind
Humidity
#VoxxedDaysMinsk
@aiborisov

Service Definitions
}
service TemperatureService {
rpc GetCurrent(Coordinates) returns (Temperature);
}
service HumidityService {
rpc GetCurrent(Coordinates) returns (Humidity);
}
service WindService {
rpc GetCurrent(Coordinates) returns (Wind);
}
#VoxxedDaysMinsk
@aiborisov

Service Definition - Response Message
float humidity = 2;
}
#VoxxedDaysMinsk
@aiborisov

Adding New Field
float humidity = 2;
Wind wind = 3;
}
message Wind {
Speed speed = 1;
float direction = 2;
}
message Speed {
float value = 1;
Units units = 2;
enum Units {
MPH = 0;
MPS = 1;
KNOTS = 2;
KMH = 3;
}
}
#VoxxedDaysMinsk
@aiborisov

private final TemperatureServiceBlockingStub temperatureService;
private final HumidityServiceBlockingStub humidityService;
private final WindServiceBlockingStub windService;
public WeatherService(TemperatureServiceBlockingStub temperatureService,
HumidityServiceBlockingStub humidityService,
WindServiceBlockingStub windService) {
this.temperatureService = temperatureService;
this.humidityService = humidityService;
this.windService = windService;
}
...
Blocking Stub Dependencies
#VoxxedDaysMinsk
@aiborisov

...
@Override
Temperature temperature = temperatureService.getCurrent(request.getCoordinates());
Humidity humidity = humidityService.getCurrent(request.getCoordinates());
Wind wind = windService.getCurrent(request.getCoordinates());
.setTemperature(temperature).setHumidity(humidity).setWind(wind).build();
}
}
Blocking Stub Dependencies
#VoxxedDaysMinsk
@aiborisov

54
Blocking Stubs
Temp
Humidity
Wind
#VoxxedDaysMinsk
@aiborisov

private final TemperatureServiceFutureStub tempService;
private final HumidityServiceFutureStub humidityService;
private final WindServiceFutureStub windService;
public WeatherService(TemperatureServiceFutureStub tempService,
HumidityServiceFutureStub humidityService,
WindServiceFutureStub windService) {
this.tempService = tempService;
this.humidityService = humidityService;
this.windService = windService;
}
...
Async Future Stub Dependencies
#VoxxedDaysMinsk
@aiborisov

@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates),
(Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity).build())
);
...
#VoxxedDaysMinsk
@aiborisov

...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
@Override
#VoxxedDaysMinsk
@aiborisov

Netty Transport
#VoxxedDaysMinsk
@aiborisov

Netty Transport
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(new WeatherService()).build().start();
#VoxxedDaysMinsk
@aiborisov

Netty: Asynchronous and Non-Blocking IO
Netty-based gRPC transport:
• Multiplexes connections on the event loops.
• Decouples I/O waiting from threads.
Request
Event
Loop
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Event
Loop
Event
Loop
Event
Loop
Callback
Request
Callback
N
e
t
w
o
r
k
N
e
t
w
o
r
k
#VoxxedDaysMinsk
@aiborisov

@Override
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
#VoxxedDaysMinsk
@aiborisov

Panta rhei, "everything flows".
Heraclitus of Ephesus
~2,400 years Before HTTP/1.x
https://en.wikipedia.org/wiki/Heraclitus
“ ”
#VoxxedDaysMinsk
@aiborisov

Streaming Service Definitions
service WeatherStreamingService {
rpc GetCurrent(WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureStreamingService {
rpc GetCurrent(Coordinates) returns (stream Temperature);
}
service HumidityStreamingService {
rpc GetCurrent(Coordinates) returns (stream Humidity);
}
service WindStreamingService {
rpc GetCurrent(Coordinates) returns (stream Wind);
}
#VoxxedDaysMinsk
@aiborisov

Bidirectional Streaming Service Definitions
rpc GetCurrent(stream WeatherRequest) returns (stream WeatherResponse);
}
rpc GetCurrent(stream Coordinates) returns (stream Temperature);
}
rpc GetCurrent(stream Coordinates) returns (stream Humidity);
}
rpc GetCurrent(stream Coordinates) returns (stream Wind);
}
#VoxxedDaysMinsk
@aiborisov

Bidirectional Streaming Service Definitions
rpc Observe(stream WeatherRequest) returns (stream WeatherResponse);
}
rpc Observe(stream Coordinates) returns (stream Temperature);
}
rpc Observe(stream Coordinates) returns (stream Humidity);
}
rpc Observe(stream Coordinates) returns (stream Wind);
}
#VoxxedDaysMinsk
@aiborisov

Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
#VoxxedDaysMinsk
@aiborisov

...
@Override
StreamObserver<Coordinates> temperatureClientStream =
temperatureService.observe(new StreamObserver<Temperature>() {
@Override
public void onNext(Temperature temperature) {
WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build()
responseObserver.onNext(resp);
}
@Override
public void onError(Throwable t) { responseObserver.onError(t); }
@Override
public void onCompleted() { responseObserver.onCompleted(); } });
...
}
#VoxxedDaysMinsk
@aiborisov

return new StreamObserver<WeatherRequest>() {
@Override
public void onNext(WeatherRequest request) {
temperatureClientStream.onNext(request.getCoordinates());
}
@Override
public void onError(Throwable t) { temperatureClientStream.onError(t); }
@Override
public void onCompleted() { temperatureClientStream.onCompleted(); }
};
}
...
@Override
StreamObserver<Coordinates> temperatureClientStream = ...
#VoxxedDaysMinsk
@aiborisov

74
Messaging applications.
Games / multiplayer tournaments.
Moving objects.
Sport results.
Stock market quotes.
Smart home devices.
You name it!
Streaming Use-Cases
#VoxxedDaysMinsk
@aiborisov

#VoxxedDaysMinsk
@aiborisov
32 core VMs, streaming throughput

gRPC Speaks Your Language
● Java
● Go
● C/C++
● C#
● Node.js
● PHP
● Ruby
● Python
● Objective-C
● JavaScript
● MacOS
● Linux
● Windows
● Android
● iOS
Service definitions and client libraries Platforms supported
#VoxxedDaysMinsk
@aiborisov

Interoperability
Java
Service
Python
Service
GoLang
Service
C++
Service
gRPC
Service
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Service
gRPC
Service
gRPC
Service
gRPC
Stub
#VoxxedDaysMinsk
@aiborisov

#VoxxedDaysMinsk
@aiborisov
Fault Tolerance?

Use time-outs!
And it will be fine.
“ ”
#VoxxedDaysMinsk
@aiborisov

Timeouts?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
?
?
?
? ?
? ?
? ?
#VoxxedDaysMinsk
@aiborisov

Default Timeout For Every Service?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
200 ms
200 ms
200 ms
200 ms 200 ms
200 ms 200 ms
200 ms 200 ms
#VoxxedDaysMinsk
@aiborisov

Gateway
20 ms 10 ms 10 ms
FastFastFast
30 ms 40 ms 20 ms
130 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#VoxxedDaysMinsk
@aiborisov

Gateway
30 ms 80 ms 100 ms
SlowFair
210 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#VoxxedDaysMinsk
@aiborisov

30 ms 80 ms 100 ms
SlowFair
210 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#VoxxedDaysMinsk
@aiborisov

80 ms 100 ms
SlowFair
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
Still working!
#VoxxedDaysMinsk
@aiborisov

BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
#VoxxedDaysMinsk
@aiborisov

BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
R
e
t
r
y
#VoxxedDaysMinsk
@aiborisov

200 ms
timeout
200 ms
timeout
200 ms
timeout
R
e
t
r
y
#VoxxedDaysMinsk
@aiborisov

Tune Timeout For Every Service?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
190 ms
190 ms
190 ms
110 ms 50 ms
110 ms 50 ms
110 ms 50 ms
#VoxxedDaysMinsk
@aiborisov

Gateway
200 ms
timeout
190 ms
timeout
110 ms
timeout
30 ms 80 ms 25 ms
FastFair
55 ms
Fast
30 ms
50 ms
timeout
#VoxxedDaysMinsk
@aiborisov

Gateway
200 ms
timeout
190 ms
timeout
110 ms
timeout
30 ms 80 ms 25 ms
Fair
55 ms
Fast
30 ms
50 ms
timeout
#VoxxedDaysMinsk
@aiborisov

200 ms
timeout
190 ms
timeout
110 ms
timeout
80 ms 25 ms
Fair
55 ms
Fast
30 ms
50 ms
timeout
#JokerConf @aiborisov
30 ms
Waiting
for an RPC
response

Timeouts for Real World?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
#VoxxedDaysMinsk
@aiborisov

Timeouts for Real World?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
2,500 ms
140 ms
#VoxxedDaysMinsk
@aiborisov

98
Timeouts in gRPC
#VoxxedDaysMinsk
@aiborisov

99
gRPC Java does not support timeouts.
Timeouts in gRPC
#VoxxedDaysMinsk
@aiborisov

100
gRPC Java does not support timeouts.
gRPC supports deadlines instead!
gRPC Deadlines
WeatherResponse response =
client.withDeadlineAfter(200, MILLISECONDS)
.getCurrent(request);
#VoxxedDaysMinsk
@aiborisov

101
First-class feature in gRPC.
Deadline is an absolute point in time.
Deadline indicates to the server how
long the client is willing to wait for an
answer.
RPC will fail with DEADLINE_EXCEEDED
status code when deadline reached.
gRPC Deadlines
#VoxxedDaysMinsk
@aiborisov

gRPC Deadline Propagation
Gateway
Now =
1476600000000
Deadline =
1476600000200
Remaining = 200
withDeadlineAfter(200, MILLISECONDS)
DEADLINE_EXCEEDED DEADLINE_EXCEEDED
60 ms
DEADLINE_EXCEEDED
90 ms
DEADLINE_EXCEEDED
60 ms
Now =
1476600000060
Deadline =
1476600000200
Remaining = 140
Now =
1476600000150
Deadline =
1476600000200
Remaining = 50
Now =
1476600000210
Deadline =
1476600000200
Remaining = -10
#VoxxedDaysMinsk
@aiborisov

103
Deadlines are automatically propagated!
Can be accessed by the receiver!
gRPC Deadlines
Context context = Context.current();
context.getDeadline().isExpired();
context.getDeadline()
.timeRemaining(MILLISECONDS);
context.getDeadline().runOnExpiration(() ->
logger.info("Deadline exceeded!"), exec);
#VoxxedDaysMinsk
@aiborisov

104
Deadlines are expected.
What about unpredictable cancellations?
• User cancelled request.
• Caller is not interested in the result any
more.
• etc
Cancellation?
#VoxxedDaysMinsk
@aiborisov

Cancellation?
GW
Busy Busy Busy
Busy Busy Busy
Busy Busy Busy
RPC
Active RPC Active RPC
Active RPC
Active RPC Active RPCActive RPC
Active RPC
#VoxxedDaysMinsk
@aiborisov

Cancellation?
GW
Busy Busy Busy
Busy Busy Busy
Busy Busy Busy
Active RPC
Active RPC Active RPCActive RPC
Active RPC
#VoxxedDaysMinsk
@aiborisov

Cancellation Propagation
GW
Idle Idle Idle
Idle Idle Idle
Idle Idle Idle
#VoxxedDaysMinsk
@aiborisov

108
Automatically propagated.
RPC fails with CANCELLED status code.
Cancellation status can be accessed by
the receiver.
Server (receiver) always knows if RPC is
valid!
gRPC Cancellation
#VoxxedDaysMinsk
@aiborisov

gRPC Cancellation
public class WeatherService extends WeatherGrpc.WeatherServiceImplBase {
...
@Override
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.isCancelled();
...
#VoxxedDaysMinsk
@aiborisov

gRPC Cancellation
public class WeatherService extends WeatherGrpc.WeatherServiceImplBase {
...
@Override
streamObserver.setOnCancelHandler(() -> {
cleanupResources();
logger.info("Call cancelled by client!");
});
...
#VoxxedDaysMinsk
@aiborisov

More Control?
#VoxxedDaysMinsk
@aiborisov

BiDi Streaming - Slow Client
Fast Server
Request
Responses
Slow Client
CANCELLED
UNAVAILABLE
RESOURCE_EXHAUSTED
#VoxxedDaysMinsk
@aiborisov

BiDi Streaming - Slow Server
Slow Server
Request
Response
Fast Client
CANCELLED
UNAVAILABLE
RESOURCE_EXHAUSTED
Requests
#VoxxedDaysMinsk
@aiborisov

Client-Side Flow-Control
Server
Request, count = 4
Client
2 Responses
4 Responses
Count = 2
#VoxxedDaysMinsk
@aiborisov

Server-Side Flow-Control
Server
Request
Client
Response
count = 2
2 Requests
count = 3
3 Requests
#VoxxedDaysMinsk
@aiborisov

Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#VoxxedDaysMinsk
@aiborisov

Flow-Control (Client-Side)
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreaminServicegImplBase {
...
@Override
streamObserver.setOnReadyHandler(() -> {
if (streamObserver.isReady()) {
streamObserver.onNext(calculateWeather());
}
});
...
#VoxxedDaysMinsk
@aiborisov

122
Helps to balance computing power
and network capacity between client
and server.
gRPC supports both client- and
server-side flow control.
Disabled by default.
Flow-Control
#VoxxedDaysMinsk
@aiborisov

What else?
#VoxxedDaysMinsk
@aiborisov

Microservices?! But...
#JokerConf @aiborisov
Performance Hit?
Service Discovery?
Load Balancing?
Fault Tolerance?
Monitoring?
Tracing?
Testing?

Service Discovery & Load Balancing
HTTP/2
RPC Server-side App
Service Definition
(extends generated definition)
ServerCall handler
TransportTransport
ServerCall
RPC Client-Side App
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
#VoxxedDaysMinsk
@aiborisov

HTTP/2
RPC Client-Side App
Instance
#1
Instance
#N
Instance
#2
RPC Server-side Apps
Transport
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
#VoxxedDaysMinsk
@aiborisov

HTTP/2
RPC Client-Side App
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
RPC Server-side Apps
Tran #1 Tran #2 Tran #N
Service Definition
(extends generated definition)
ServerCall handler
Transport
ServerCall
NameResolver LoadBalancer
Pluggable
Load
Balancing
and
Service
Discovery
#VoxxedDaysMinsk
@aiborisov

String host = System.getenv("weather_host");
int post = Integer.valueOf(System.getenv("weather_port"));
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress(host, port)
.build();
#VoxxedDaysMinsk
@aiborisov

ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv")
.build();
#VoxxedDaysMinsk
@aiborisov

.nameResolverFactory(new DnsNameResolverProvider())
.loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance())
.build();
#VoxxedDaysMinsk
@aiborisov

.nameResolverFactory(new MyCustomNameResolverProviderFactory())
.loadBalancerFactory(new MyMoonPhaseLoadBalancerFactory())
.build();
#VoxxedDaysMinsk
@aiborisov

Testing Support
In-process server transport and client-side channel
StreamRecorder - StreamObserver that records all the observed values and errors.
MetadataUtils for testing metadata headers and trailer.
grpc-testing - utility functions for unit and integration testing:
https://github.com/grpc/grpc-java/tree/master/testing
#VoxxedDaysMinsk
@aiborisov

Testing Support - InProcess
In-process transport: fully-featured, high performance, and useful in testing.
WeatherServiceAsync weatherService =
new WeatherServiceAsync(tempService, humidityService, windService);
Server grpcServer = InProcessServerBuilder.forName("weather")
.addService(weatherService).build();
Channel grpcChannel = InProcessChannelBuilder.forName("weather").build();
WeatherServiceBlockingStub stub =
WeatherServiceGrpc.newBlockingStub(grpcChannel).withDeadlineAfter(100, MILLISECONDS);
#VoxxedDaysMinsk
@aiborisov

More Features
Distributed tracing support:
• OpenTracing, Zipkin / Brave support
Interceptors to add cross-cutting behavior:
• Client- and server-side
Monitoring:
• OpenCensus, gRPC Prometheus; grpcz-monitoring.
Built-in authentication mechanisms:
• SSL/TLS; token-based authentication with Google; authentication API.
Payload compression: https://github.com/grpc/grpc/blob/master/doc/compression.md
#VoxxedDaysMinsk
@aiborisov

Growing Community and Ecosystem
Your
company could
be here
#VoxxedDaysMinsk
@aiborisov

Try It Out!
http://grpc.io
gRPC on Github: https://github.com/grpc
Demo: https://github.com/alxbnet/voxxeddays-grpc-demo
Google group: grpc-io@googlegroups.com
gRPC Java quickstart: http://www.grpc.io/docs/quickstart/java.html
gRPC Java tutorial: http://www.grpc.io/docs/tutorials/basic/java.html
gRPC contribution: https://github.com/grpc/grpc-contrib
#VoxxedDaysMinsk
@aiborisov

Takeaways
gRPC (http://grpc.io):
• HTTP/2 transport based, open source, general purpose
standards-based, feature-rich RPC framework.
• Bidirectional streaming over one single TCP connection.
• Netty transport provides asynchronous and non-blocking I/O.
• Deadline and cancellations propagation.
• Client and server-side flow-control.
• Pluggable service discovery and load-balancing
• Generated client libraries
• Supports 10+ programming languages.
• Build-in testing support.
• Production-ready and growing ecosystem.
#VoxxedDaysMinsk
@aiborisov

Questions?
#VoxxedDaysMinsk
@aiborisov

Thank You
#VoxxedDaysMinsk
@aiborisov

Images Used
Special thanks for the provided photos:
• Andrey Borisenko
• Stanislav Vedmid
Photo from https://www.google.com/about/datacenters page:
• https://www.google.com/about/datacenters/gallery/#/tech/12
Photos licenced by Creative Commons 2.0 https://creativecommons.org/licenses/by/2.0/ :
• https://www.flickr.com/photos/garryknight/5754661212/ by Garry Knight
• https://www.flickr.com/photos/sodaigomi/21128888345/ by sodai gomi
• https://www.flickr.com/photos/zengame/15972170944/ by Zengame
• https://www.flickr.com/photos/londonmatt/18496249193/ by Matt Brown
• https://www.flickr.com/photos/gazeronly/8058105980/ by torbakhopper
#VoxxedDaysMinsk
@aiborisov

"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition

More Related Content

What's hot (20)

Similar to "Enabling Googley microservices with gRPC" VoxxedDays Minsk edition (20)

Recently uploaded (20)

"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition