SlideShare a Scribd company logo
Enabling Googley microservices with gRPC.
A high performance, open source, HTTP/2-based RPC framework.
April 7, 2017
Alex Borysov @aiborisov
#DevoxxFR
Google Cloud Platform
Alex Borysov
• Software Engineer at Google
• Active gRPC user.
@aiborisov
#DevoxxFR
Google Cloud Platform
What is gRPC?
Au commencement était la Stubby.
#DevoxxFR @aiborisov
Google Cloud Platform
What is gRPC?
Au commencement était la Stubby
gRPC stands for gRPC Remote Procedure Calls.
A high performance, open source, general purpose
standards-based, feature-rich RPC framework.
Open sourced version of Stubby RPC used in Google.
Actively developed and production-ready, current
version is 1.2.0.
#DevoxxFR @aiborisov
Google Cloud Platform
Why gRPC?
Photo taken by Andrey Borisenko.
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC vs JSON/HTTP for Google Cloud Pub/Sub
3x increase in throughput
https://cloud.google.com/blog/big-data/2016/03/announcing-grpc-alpha-for-google-cloud-pubsub
11x difference per CPU
#DevoxxFR @aiborisov
Google Cloud Platform
Google ~O(1010
) QPS.
#DevoxxFR @aiborisov
Google Cloud Platform
Google ~O(1010
) QPS.
Your project QPS?
#DevoxxFR @aiborisov
Google Cloud Platform
Continuous Performance Benchmarking
http://www.grpc.io/docs/guides/benchmarking.html
8 core VMs, unary
#DevoxxFR @aiborisov
32 core VMs, unary
Google Cloud Platform
• Single TCP connection.
• No Head-of-line blocking.
• Binary framing layer.
• Header Compression.
HTTP/2 in One Slide
Transport(TCP)
Application (HTTP/2)
Network (IP)
Session (TLS) [optional]
Binary Framing
HEADERS Frame
DATA Frame
HTTP/2
POST: /upload
HTTP/1.1
Host: www.javaday.org.ua
Content-Type: application/json
Content-Length: 27
HTTP/1.x
{“msg”: “Welcome to 2017!”}
#DevoxxFR @aiborisov
Google Cloud Platform
HTTP/1.x vs HTTP/2
http://www.http2demo.io/
https://http2.golang.org/gophertiles
#DevoxxFR @aiborisov
HTTP/1.1 HTTP/2
Google Cloud Platform
Okay, but how?
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definition
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definition (weather.proto)
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definition (weather.proto)
syntax = "proto3";
option java_multiple_files = true;
option java_package = "fr.devoxx.grpcexample";
package grpcexample;
service Weather {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definition (weather.proto)
syntax = "proto3";
option java_multiple_files = true;
option java_package = "fr.devoxx.grpcexample";
package grpcexample;
service Weather {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definition (weather.proto)
syntax = "proto3";
service Weather {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
message WeatherRequest {
Coordinates coordinates = 1;
message Coordinates {
fixed64 latitude = 1;
fixed64 longitude = 2;
}
}
message WeatherResponse {
Temperature temperature = 1;
float humidity = 2;
}
message Temperature {
float degrees = 1;
Units units = 2;
enum Units {
FAHRENHEIT = 0;
CELSIUS = 1;
KELVIN = 2;
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
Generated Classes
$ ls -1 build/generated/source/proto/main/java/fr/devoxx/grpcexample/
Temperature.java
TemperatureOrBuilder.java
WeatherProto.java
WeatherRequest.java
WeatherRequestOrBuilder.java
WeatherResponse.java
WeatherResponseOrBuilder.java
#DevoxxFR @aiborisov
Google Cloud Platform
Generated Classes
$ ls -1 build/generated/source/proto/main/java/fr/devoxx/grpcexample/
Temperature.java
TemperatureOrBuilder.java
WeatherProto.java
WeatherRequest.java
WeatherRequestOrBuilder.java
WeatherResponse.java
WeatherResponseOrBuilder.java
$ ls -1 build/generated/source/proto/main/grpc/fr/devoxx/grpcexample/
WeatherGrpc.java
#DevoxxFR @aiborisov
Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
}
}
Abstract class
#DevoxxFR @aiborisov
Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request,
public interface StreamObserver<WeatherResponse> responseObserver){{
void onNext(WeatherResponse response);
void onCompleted();
void onError(Throwable error);
}
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
WeatherResponse response = WeatherResponse.newBuilder()
.setTemperature(Temperature.newBuilder().setDegrees(70).setUnits(FAHRENHEIT))
.setHumidity(.65f).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
Start gRPC Server
void start() throws IOException {
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(new WeatherService()).build()
.start();
Runtime.getRuntime().addShutdownHook(new Thread(grpcServer::shutdown));
grpcServer.awaitTerminaton();
}
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Clients
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel);
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Clients
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel);
WeatherGrpc.WeatherBlockingStub blockingClient = WeatherGrpc.newBlockingStub(grpcChannel);
WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel);
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Sync Client
WeatherRequest request = WeatherRequest.newBuilder()
.setCoordinates(Coordinates.newBuilder().setLatitude(420000000)
.setLongitude(-720000000)).build();
WeatherResponse response = blockingClient.getCurrent(request);
logger.info("Current weather for {}: {}", request, response);
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Async Client
WeatherRequest request = WeatherRequest.newBuilder()
.setCoordinates(Coordinates.newBuilder().setLatitude(420000000)
.setLongitude(-720000000)).build();
client.getCurrent(request, new StreamObserver<WeatherResponse>() {
@Override
public void onNext(WeatherResponse response) {
logger.info("Current weather for {}: {}", request, response);
}
@Override
public void onError(Throwable t) { logger.info("Cannot get weather for {}", request); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
#DevoxxFR @aiborisov
Google Cloud Platform
Implement gRPC Service
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
WeatherResponse response = WeatherResponse.newBuilder()
.setTemperature(Temperature.newBuilder().setDegrees(70).setUnits(FAHRENHEIT))
.setHumidity(.65f).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
Adding New [Micro]Services
Weather
Weather
Temp
Wind
Humidity
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definitions
service Weather {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
service TemperatureService {
rpc GetCurrent(Coordinates) returns (Temperature);
}
service HumidityService {
rpc GetCurrent(Coordinates) returns (Humidity);
}
service WindService {
rpc GetCurrent(Coordinates) returns (Wind);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definition - Response Message
message WeatherResponse {
Temperature temperature = 1;
float humidity = 2;
}
#DevoxxFR @aiborisov
Google Cloud Platform
Adding New Field
message WeatherResponse {
Temperature temperature = 1;
float humidity = 2;
Wind wind = 3;
}
message Wind {
Speed speed = 1;
float direction = 2;
}
message Speed {
float value = 1;
Units units = 2;
enum Units {
MPH = 0;
MPS = 1;
KNOTS = 2;
KMH = 3;
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
private final TemperatureServiceBlockingStub temperatureService;
private final HumidityServiceBlockingStub humidityService;
private final WindServiceBlockingStub windService;
public WeatherService(TemperatureServiceBlockingStub temperatureService,
HumidityServiceBlockingStub humidityService,
WindServiceBlockingStub windService) {
this.temperatureService = temperatureService;
this.humidityService = humidityService;
this.windService = windService;
}
...
Blocking Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
#DevoxxFR @aiborisov
Google Cloud Platform
...
@Override
public void getCurrent(WeatherRequest request,
StreamObserver<WeatherResponse> responseObserver) {
Temperature temperature = temperatureService.getCurrent(request.getCoordinates());
Humidity humidity = humidityService.getCurrent(request.getCoordinates());
Wind wind = windService.getCurrent(request.getCoordinates());
WeatherResponse response = WeatherResponse.newBuilder().setTemperature(temperature)
.setHumidity(humidity.getValue()).setWind(wind).build();
responseObserver.onNext(response);
responseObserver.onCompleted();
}
}
Blocking Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
#DevoxxFR @aiborisov
35
Blocking Stubs
Temp
Humidity
Wind
#DevoxxFR @aiborisov
Google Cloud Platform
private final TemperatureServiceFutureStub tempService;
private final HumidityServiceFutureStub humidityService;
private final WindServiceFutureStub windService;
public WeatherService(TemperatureServiceFutureStub tempService,
HumidityServiceFutureStub humidityService,
WindServiceFutureStub windService) {
this.tempService = tempService;
this.humidityService = humidityService;
this.windService = windService;
}
...
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
#DevoxxFR @aiborisov
Google Cloud Platform
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates), (Humidity humidity) ->
WeatherResponse.newBuilder().setHumidity(humidity.getValue()).build())
);
...
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
#DevoxxFR @aiborisov
Google Cloud Platform
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates), (Humidity humidity) ->
WeatherResponse.newBuilder().setHumidity(humidity.getValue()).build())
);
...
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
#DevoxxFR @aiborisov
Google Cloud Platform
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Coordinates coordinates = request.getCoordinates();
ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList(
Futures.transform(tempService.getCurrent(coordinates),
(Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()),
Futures.transform(windService.getCurrent(coordinates),
(Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()),
Futures.transform(humidityService.getCurrent(coordinates), (Humidity humidity) ->
WeatherResponse.newBuilder().setHumidity(humidity.getValue()).build())
);
...
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
#DevoxxFR @aiborisov
Google Cloud Platform
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
#DevoxxFR @aiborisov
Google Cloud Platform
Netty Transport
ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build();
WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel);
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(new WeatherService()).build().start();
#DevoxxFR @aiborisov
Google Cloud Platform
Netty: Asynchronous and Non-Blocking IO
Netty-based transport:
• Multiplexes conn-s on the event loops: EpollEventLoopGroup, NioEventLoopGroup.
• Decouples I/O waiting from threads.
• gRPC uses Netty event loops for both client and server transports.
Request
Event
Loop
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Worker thread
Event
Loop
Event
Loop
Event
Loop
Callback
Request
Callback
N
e
t
w
o
r
k
N
e
t
w
o
r
k
#DevoxxFR @aiborisov
Google Cloud Platform
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
Async Future Stub Dependencies
public class WeatherService extends WeatherGrpc.WeatherImplBase {
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
...
Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() {
@Override
public void onSuccess(@Nullable List<WeatherResponse> results) {
WeatherResponse.Builder response = WeatherResponse.newBuilder();
results.forEach(response::mergeFrom);
responseObserver.onNext(response.build());
responseObserver.onCompleted();
}
@Override
public void onFailure(Throwable t) { responseObserver.onError(t); }
});
}
}
#DevoxxFR @aiborisov
Google Cloud Platform
Panta rhei, "everything flows".
Heraclitus of Ephesus
~2,400 years Before HTTP/1.x
https://en.wikipedia.org/wiki/Heraclitus
“ ”
#DevoxxFR @aiborisov
Google Cloud Platform
Service Definitions
service Weather {
rpc GetCurrent(WeatherRequest) returns (WeatherResponse);
}
service TemperatureService {
rpc GetCurrent(Coordinates) returns (Temperature);
}
service HumidityService {
rpc GetCurrent(Coordinates) returns (Humidity);
}
service WindService {
rpc GetCurrent(Coordinates) returns (Wind);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Streaming Service Definitions
service WeatherStreaming {
rpc GetCurrent(WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureServiceStreaming {
rpc GetCurrent(Coordinates) returns (stream Temperature);
}
service HumidityServiceStreaming {
rpc GetCurrent(Coordinates) returns (stream Humidity);
}
service WindServiceStreaming {
rpc GetCurrent(Coordinates) returns (stream Wind);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Bidirectional Streaming Service Definitions
service WeatherStreaming {
rpc GetCurrent(stream WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureServiceStreaming {
rpc GetCurrent(stream Coordinates) returns (stream Temperature);
}
service HumidityServiceStreaming {
rpc GetCurrent(stream Coordinates) returns (stream Humidity);
}
service WindServiceStreaming {
rpc GetCurrent(stream Coordinates) returns (stream Wind);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Bidirectional Streaming Service Definitions
service WeatherStreaming {
rpc Observe(stream WeatherRequest) returns (stream WeatherResponse);
}
service TemperatureServiceStreaming {
rpc Observe(stream Coordinates) returns (stream Temperature);
}
service HumidityServiceStreaming {
rpc Observe(stream Coordinates) returns (stream Humidity);
}
service WindServiceStreaming {
rpc Observe(stream Coordinates) returns (stream Wind);
}
#DevoxxFR @aiborisov
Google Cloud Platform
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
#DevoxxFR @aiborisov
Google Cloud Platform
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
StreamObserver<Coordinates> temperatureClientStream =
temperatureService.observe(new StreamObserver<Temperature>() {
@Override
public void onNext(Temperature temperature) {
WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build()
responseObserver.onNext(resp);
}
@Override
public void onError(Throwable t) { responseObserver.onError(t); }
@Override
public void onCompleted() { responseObserver.onCompleted(); } });
...
}
#DevoxxFR @aiborisov
Google Cloud Platform
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
StreamObserver<Coordinates> temperatureClientStream =
temperatureService.observe(new StreamObserver<Temperature>() {
@Override
public void onNext(Temperature temperature) {
WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build()
responseObserver.onNext(resp);
}
@Override
public void onError(Throwable t) { responseObserver.onError(t); }
@Override
public void onCompleted() { responseObserver.onCompleted(); } });
...
}
#DevoxxFR @aiborisov
Google Cloud Platform
return new StreamObserver<WeatherRequest>() {
@Override
public void onNext(WeatherRequest request) {
temperatureClientStream.onNext(request.getCoordinates());
}
@Override
public void onError(Throwable t) { temperatureClientStream.onError(t); }
@Override
public void onCompleted() { temperatureClientStream.onCompleted(); }
};
}
Bidirectional Streaming Service
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
StreamObserver<Coordinates> temperatureClientStream = ...
#DevoxxFR @aiborisov
54
Messaging applications.
Games / multiplayer tournaments.
Moving objects.
Sport results.
Stock market quotes.
Smart home devices.
You name it!
Streaming Use-Cases
#DevoxxFR @aiborisov
Google Cloud Platform
Continuous Performance Benchmarking
http://www.grpc.io/docs/guides/benchmarking.html
8 core VMs, streaming
#DevoxxFR @aiborisov
32 core VMs, streaming
Google Cloud Platform
gRPC Speaks Your Language
● Java
● Go
● C/C++
● C#
● Node.js
● PHP
● Ruby
● Python
● Objective-C
● MacOS
● Linux
● Windows
● Android
● iOS
Service definitions and client libraries Platforms supported
#DevoxxFR @aiborisov
Google Cloud Platform
Interoperability
Java
Service
Python
Service
GoLang
Service
C++
Service
gRPC
Service
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Stub
gRPC
Service
gRPC
Service
gRPC
Service
gRPC
Stub
#DevoxxFR @aiborisov
58
Things will break!
Don’t pretend you can eliminate every
possible source of failure.
Integration points are #1 killer of
software systems.
Fault Tolerance?
#DevoxxFR @aiborisov
Google Cloud Platform
Use time-outs!
Because your code cannot wait forever.
Captain
“ ”* Original photo taken by Stanislav Vedmid
.
#DevoxxFR @aiborisov
Google Cloud Platform
Timeouts?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
?
?
?
? ?
? ?
? ?
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
200 ms
200 ms
200 ms
200 ms 200 ms
200 ms 200 ms
200 ms 200 ms
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
Gateway
20 ms 10 ms 10 ms
FastFastFast
30 ms 40 ms 20 ms
130 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
Gateway
30 ms 80 ms 100 ms
SlowFair
210 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
30 ms 80 ms 100 ms
SlowFair
210 ms
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
80 ms 100 ms
SlowFair
200 ms
timeout
200 ms
timeout
200 ms
timeout
200 ms
timeout
Still working!
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
R
e
t
r
y
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
BusyBusy
200 ms
timeout
200 ms
timeout
200 ms
timeout
Idle Busy
R
e
t
r
y
#DevoxxFR @aiborisov
Google Cloud Platform
Default Timeout For Every Service?
200 ms
timeout
200 ms
timeout
200 ms
timeout
R
e
t
r
y
#DevoxxFR @aiborisov
Google Cloud Platform
Tune Timeout For Every Service?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
190 ms
190 ms
190 ms
110 ms 50 ms
110 ms 50 ms
110 ms 50 ms
#DevoxxFR @aiborisov
Google Cloud Platform
Tune Timeout For Every Service?
Gateway
200 ms
timeout
190 ms
timeout
110 ms
timeout
30 ms 80 ms 25 ms
FastFair
55 ms
Fast
30 ms
50 ms
timeout
#DevoxxFR @aiborisov
Google Cloud Platform
Tune Timeout For Every Service?
Gateway
200 ms
timeout
190 ms
timeout
110 ms
timeout
30 ms 80 ms 25 ms
Fair
55 ms
Fast
30 ms
50 ms
timeout
#DevoxxFR @aiborisov
Google Cloud Platform
Timeouts?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
?
?
?
? ?
? ?
? ?
#DevoxxFR @aiborisov
Google Cloud Platform
Timeouts for Real World?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
#DevoxxFR @aiborisov
Google Cloud Platform
Timeouts for Real World?
GW
A1 A2 A3
B1 B2 B3
C1 C2 C3
200 ms
2,500 ms
140 ms
#DevoxxFR @aiborisov
76
Timeouts in gRPC
#DevoxxFR @aiborisov
77
gRPC Java does not support timeouts.
Timeouts in gRPC
#DevoxxFR @aiborisov
78
gRPC Java does not support timeouts.
gRPC supports deadlines instead!
gRPC Deadlines
WeatherResponse response =
client.withDeadlineAfter(200, MILLISECONDS)
.getCurrent(request);
#DevoxxFR @aiborisov
79
First-class feature in gRPC.
Deadline is an absolute point in time.
Deadline indicates to the server how
long the client is willing to wait for an
answer.
RPC will fail with DEADLINE_EXCEEDED
status code when deadline reached.
gRPC Deadlines
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Deadline Propagation
Gateway
90 ms
Now =
1476600000000
Deadline =
1476600000200
Remaining = 200
40 ms
20 ms
20 ms 60 ms
withDeadlineAfter(200, MILLISECONDS)
DEADLINE_EXCEEDED DEADLINE_EXCEEDED DEADLINE_EXCEEDED DEADLINE_EXCEEDED
Now =
1476600000040
Deadline =
1476600000200
Remaining = 160
Now =
1476600000150
Deadline =
1476600000200
Remaining = 50
Now =
1476600000230
Deadline =
1476600000200
Remaining = -30
#DevoxxFR @aiborisov
81
Deadlines are automatically propagated!
Can be accessed by the receiver!
gRPC Deadlines
Context context = Context.current();
context.getDeadline().isExpired();
context.getDeadline()
.timeRemaining(MILLISECONDS);
context.getDeadline().runOnExpiration(() ->
logger.info("Deadline exceeded!"), exec);
#DevoxxFR @aiborisov
82
Deadlines are expected.
What about unpredictable cancellations?
• User cancelled request.
• Caller is not interested in the result any
more.
• etc
Cancellation?
#DevoxxFR @aiborisov
Google Cloud Platform
Cancellation?
GW
Busy Busy Busy
Busy Busy Busy
Busy Busy Busy
RPC
Active RPC Active RPC
Active RPC
Active RPC Active RPCActive RPC
Active RPC Active RPC
Active RPC
#DevoxxFR @aiborisov
Google Cloud Platform
Cancellation?
GW
Busy Busy Busy
Busy Busy Busy
Busy Busy Busy
Active RPC Active RPC
Active RPC
Active RPC Active RPCActive RPC
Active RPC Active RPC
Active RPC
#DevoxxFR @aiborisov
Google Cloud Platform
Cancellation Propagation
GW
Idle Idle Idle
Idle Idle Idle
Idle Idle Idle
#DevoxxFR @aiborisov
86
Automatically propagated.
RPC fails with CANCELLED status code.
Cancellation status can be accessed by
the receiver.
Server (receiver) always knows if RPC is
valid!
gRPC Cancellation
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Cancellation
public class WeatherService extends WeatherGrpc.WeatherImplBase {
...
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.isCancelled();
...
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Cancellation
public class WeatherService extends WeatherGrpc.WeatherImplBase {
...
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.setOnCancelHandler(() -> {
cleanupResources();
logger.info("Call cancelled by client!");
});
...
#DevoxxFR @aiborisov
Google Cloud Platform
gRPC Context
public class WeatherService extends WeatherGrpc.WeatherImplBase {
...
@Override
public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) {
Context.current().getDeadline();
Context.current().isCancelled();
Context.current().cancellationCause();
Context.current().addListener(context -> {
cleanupResources();
logger.info("Call cancelled by client!"); }, executor)
...
#DevoxxFR @aiborisov
Google Cloud Platform
More Control?
#DevoxxFR @aiborisov
Google Cloud Platform
BiDi Streaming - Slow Client
Fast Server
Request
Responses
Slow Client
CANCELLED
UNAVAILABLE
RESOURCE_EXHAUSTED
#DevoxxFR @aiborisov
Google Cloud Platform
BiDi Streaming - Slow Server
Slow Server
Request
Response
Fast Client
CANCELLED
UNAVAILABLE
RESOURCE_EXHAUSTED
Requests
#DevoxxFR @aiborisov
Google Cloud Platform
Client-Side Flow-Control
Server
Request, count = 4
Client
2 Responses
4 Responses
Count = 2
#DevoxxFR @aiborisov
Google Cloud Platform
Server-Side Flow-Control
Server
Request
Client
Response
count = 2
2 Requests
count = 3
3 Requests
#DevoxxFR @aiborisov
Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#DevoxxFR @aiborisov
Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#DevoxxFR @aiborisov
Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#DevoxxFR @aiborisov
Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#DevoxxFR @aiborisov
Google Cloud Platform
Flow-Control (Client-Side)
CallStreamObserver<WeatherRequest> requestStream =
(CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() {
@Override
public void beforeStart(ClientCallStreamObserver outboundStream) {
outboundStream.disableAutoInboundFlowControl();
}
@Override
public void onNext(WeatherResponse response) { processResponse(response); }
@Override
public void onError(Throwable e) { logger.error("Error on weather request.", e); }
@Override
public void onCompleted() { logger.info("Stream completed."); }
});
requestStream.onNext(request);
requestStream.request(3); // Request up to 3 responses from server
#DevoxxFR @aiborisov
Google Cloud Platform
Flow-Control (Client-Side)
public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase {
...
@Override
public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) {
ServerCallStreamObserver<WeatherResponse> streamObserver =
(ServerCallStreamObserver<WeatherResponse>) responseObserver;
streamObserver.setOnReadyHandler(() -> {
if (streamObserver.isReady()) {
streamObserver.onNext(calculateWeather());
}
});
...
#DevoxxFR @aiborisov
101
Flow-control helps to balance
computing power and network
capacity between client and server.
gRPC supports both client- and
server-side flow control.
Disabled by default.
Flow-Control
Photo taken by Andrey Borisenko.
#DevoxxFR @aiborisov
Google Cloud Platform
What else?
#DevoxxFR @aiborisov
103
Service Discovery
Load Balancing
Tracing, debugging
Monitoring
Testing
Microservices?
#DevoxxFR @aiborisov
Google Cloud Platform
Service Discovery & Load Balancing
HTTP/2
RPC Server-side App
Service Definition
(extends generated definition)
ServerCall handler
TransportTransport
ServerCall
RPC Client-Side App
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
#DevoxxFR @aiborisov
Google Cloud Platform
HTTP/2
Service Discovery & Load Balancing
RPC Client-Side App
Instance
#1
Instance
#N
Instance
#2
RPC Server-side Apps
Transport
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
#DevoxxFR @aiborisov
Google Cloud Platform
Service Discovery & Load Balancing
HTTP/2
RPC Client-Side App
Channel
Stub
Future
Stub
Blocking
Stub
ClientCall
RPC Server-side Apps
Tran #1 Tran #2 Tran #N
Service Definition
(extends generated definition)
ServerCall handler
Transport
ServerCall
NameResolver LoadBalancer
Pluggable
Load
Balancing
and
Service
Discovery
#DevoxxFR @aiborisov
Google Cloud Platform
Service Discovery & Load Balancing
ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv")
.nameResolverFactory(new DnsNameResolverProvider())
.loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance())
.build();
#DevoxxFR @aiborisov
Google Cloud Platform
Service Discovery & Load Balancing
ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv")
.nameResolverFactory(new DnsNameResolverProvider())
.loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance())
.build();
Design documents:
• Load Balancing in gRPC:
https://github.com/grpc/grpc/blob/master/doc/load-balancing.md
• gRPC Java Name Resolution and Load Balancing v2: http://tiny.cc/grpc-java-lb-v2
#DevoxxFR @aiborisov
Google Cloud Platform
Testing Support
In-process server transport: fully-featured, high performance, and useful in testing.
In-process client side channel: fully-featured, high performance, and useful in
testing.
#DevoxxFR @aiborisov
Google Cloud Platform
Testing Support - InProcess
WeatherServiceAsync weatherService =
new WeatherServiceAsync(tempService, humidityService, windService);
Server grpcServer = InProcessServerBuilder.forName("weather")
.addService(weatherService).build();
#DevoxxFR @aiborisov
Google Cloud Platform
Testing Support - InProcess
WeatherServiceAsync weatherService =
new WeatherServiceAsync(tempService, humidityService, windService);
Server grpcServer = InProcessServerBuilder.forName("weather")
.addService(weatherService).build();
Channel grpcChannel = InProcessChannelBuilder.forName("weather").build();
WeatherBlockingStub stub =
WeatherGrpc.newBlockingStub(grpcChannel).withDeadlineAfter(100, MILLISECONDS);
#DevoxxFR @aiborisov
Google Cloud Platform
Testing Support
In-process server transport and client-side channel
StreamRecorder - StreamObserver that records all the observed values and errors.
MetadataUtils for testing metadata headers and trailer.
grpc-testing - utility functions for unit and integration testing:
https://github.com/grpc/grpc-java/tree/master/testing
#DevoxxFR @aiborisov
Google Cloud Platform
Distributed Tracing
A B C
D E F
G H J
#DevoxxFR @aiborisov
Google Cloud Platform
Distributed Tracing
A B C
D E F
G H J
#DevoxxFR @aiborisov
Google Cloud Platform
Distributed Tracing with Zipkin/Brave
https://github.com/openzipkin/brave/tree/master/brave-grpc
// server-side
Server grpcServer = NettyServerBuilder.forPort(8090)
.addService(ServerInterceptors.intercept(new WeatherService(),
new BraveGrpcServerInterceptor(brave("wind"))))
.build();
// client-side
Channel windChannel = NettyChannelBuilder.forAddress(windserverAddress()).build();
WindServiceStub windClient = WindServiceGrpc.newStub(
ClientInterceptors.intercept(windChannel,
new BraveGrpcClientInterceptor(brave("weather_to_wind"))));
#DevoxxFR @aiborisov
Google Cloud Platform
More Features
Interceptors to add cross-cutting behavior:
• Client- and server-side
Monitoring:
• gRPC Prometheus.
• grpcz-monitoring
• More Monitoring APIs are coming.
Built-in authentication mechanisms:
• SSL/TLS.
• Token-based authentication with Google.
• Authentication API.
Payload compression: https://github.com/grpc/grpc/blob/master/doc/compression.md
#DevoxxFR @aiborisov
Google Cloud Platform
Growing Community and Ecosystem
https://github.com/grpc-ecosystem
Polyglot is a universal grpc command line client.
grpc-gateway generates a reverse-proxy server which translates a RESTful JSON API into gRPC.
OpenTracing is a set of consistent, expressive, vendor-neutral APIs for distributed tracing and
context propagation.
Prometheus monitoring support for grpc-java and grpc-go.
#DevoxxFR @aiborisov
Google Cloud Platform
Try It Out!
http://grpc.io
Twitter @grpcio
gRPC on Github: https://github.com/grpc
Google group: grpc-io@googlegroups.com
gRPC Java quickstart: http://www.grpc.io/docs/quickstart/java.html
gRPC Java tutorial: http://www.grpc.io/docs/tutorials/basic/java.html
gRPC contribution: https://github.com/grpc/grpc-contrib
#DevoxxFR @aiborisov
Google Cloud Platform
Takeaways
Microservices are not a free lunch!
gRPC (http://grpc.io):
• HTTP/2 transport based, open source, general purpose
standards-based, feature-rich RPC framework.
• Bidirectional streaming over one single TCP connection.
• Netty transport provides asynchronous and non-blocking I/O.
• Deadline and cancellations propagation.
• Client and server-side flow-control.
• Pluggable service discovery and load-balancing
• Supports 10 programming languages.
• Build-in testing support.
• Production-ready (current version is 1.2.0) and growing ecosystem.
#DevoxxFR @aiborisov
Thank You
#DevoxxFR @aiborisov
Google Cloud Platform
Images Used
Special thanks for the provided photos:
• Andrey Borisenko
• Stanislav Vedmid
Photo from https://www.google.com/about/datacenters page:
• https://www.google.com/about/datacenters/gallery/#/tech/12
Photos licenced by Creative Commons 2.0 https://creativecommons.org/licenses/by/2.0/ :
• https://www.flickr.com/photos/garryknight/5754661212/ by Garry Knight
• https://www.flickr.com/photos/sodaigomi/21128888345/ by sodai gomi
• https://www.flickr.com/photos/zengame/15972170944/ by Zengame
• https://www.flickr.com/photos/londonmatt/18496249193/ by Matt Brown
• https://www.flickr.com/photos/gazeronly/8058105980/ by torbakhopper
#DevoxxFR @aiborisov

More Related Content

PDF
"Enabling Googley microservices with gRPC" at JDK.IO 2017
PDF
"Enabling Googley microservices with gRPC" at JEEConf 2017
PDF
Enabling Googley microservices with HTTP/2 and gRPC.
PDF
gRPC: Beyond REST
PPTX
Microservices summit talk 1/31
PDF
gRPC and Microservices
PDF
gRPC & Kubernetes
PDF
Robert Kubis - gRPC - boilerplate to high-performance scalable APIs - code.t...
"Enabling Googley microservices with gRPC" at JDK.IO 2017
"Enabling Googley microservices with gRPC" at JEEConf 2017
Enabling Googley microservices with HTTP/2 and gRPC.
gRPC: Beyond REST
Microservices summit talk 1/31
gRPC and Microservices
gRPC & Kubernetes
Robert Kubis - gRPC - boilerplate to high-performance scalable APIs - code.t...

What's hot (20)

PPTX
Building your First gRPC Service
PDF
gRPC - RPC rebirth?
PDF
Generating Unified APIs with Protocol Buffers and gRPC
PDF
Bringing Learnings from Googley Microservices with gRPC - Varun Talwar, Google
PPTX
HTTP2 and gRPC
PPTX
Introduction to gRPC
PDF
HTTP2 in action - Piet Van Dongen - Codemotion Amsterdam 2017
PDF
Pygrib documentation
PDF
KubeCon EU 2016: Getting the Jobs Done With Kubernetes
PDF
How happy they became with H2O/mruby and the future of HTTP
PDF
Promise of Push (HTTP/2 Web Performance)
PPTX
Dockerizing the Hard Services: Neutron and Nova
PPTX
gRPC on .NET Core - NDC Sydney 2019
PDF
Developing the fastest HTTP/2 server
PDF
Beating Python's GIL to Max Out Your CPUs
PDF
Make gRPC great again
PDF
Cilium – Kernel Native Security & DDOS Mitigation for Microservices with BPF
PDF
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
PDF
H2O - the optimized HTTP server
Building your First gRPC Service
gRPC - RPC rebirth?
Generating Unified APIs with Protocol Buffers and gRPC
Bringing Learnings from Googley Microservices with gRPC - Varun Talwar, Google
HTTP2 and gRPC
Introduction to gRPC
HTTP2 in action - Piet Van Dongen - Codemotion Amsterdam 2017
Pygrib documentation
KubeCon EU 2016: Getting the Jobs Done With Kubernetes
How happy they became with H2O/mruby and the future of HTTP
Promise of Push (HTTP/2 Web Performance)
Dockerizing the Hard Services: Neutron and Nova
gRPC on .NET Core - NDC Sydney 2019
Developing the fastest HTTP/2 server
Beating Python's GIL to Max Out Your CPUs
Make gRPC great again
Cilium – Kernel Native Security & DDOS Mitigation for Microservices with BPF
KubeCon EU 2016: Leveraging ephemeral namespaces in a CI/CD pipeline
H2O - the optimized HTTP server
Ad

Similar to "Enabling Googley microservices with gRPC." at Devoxx France 2017 (20)

PDF
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
PDF
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
PDF
Usable APIs at Scale
PPTX
CocoaConf: The Language of Mobile Software is APIs
PDF
Fast and Reliable Swift APIs with gRPC
PDF
gRPC Design and Implementation
PDF
Microservices in GO - Massimiliano Dessì - Codemotion Rome 2017
PDF
REST API vs gRPC, which one should you use in breaking a monolith [Dev conf 2...
PDF
Up and Running with gRPC & Cloud Career [GDG-Cloud-Dhaka-IO/2022}
PDF
Building high performance microservices in finance with Apache Thrift
PDF
gRPC Overview
PDF
REST in Peace. Long live gRPC!
PDF
gRPC Microservices in Go (MEAP V08) Hüseyin Babal
PDF
Building microservices with grpc
PDF
Build microservice with gRPC in golang
PPTX
What is gRPC introduction gRPC Explained
PDF
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
PDF
Microservices Communication Patterns with gRPC
PPTX
gRPC - Fastest Data Transfer Protocol
PDF
Build your next REST API with gRPC
"Enabling Googley microservices with gRPC" Riga DevDays 2018 edition
"Enabling Googley microservices with gRPC" VoxxedDays Minsk edition
Usable APIs at Scale
CocoaConf: The Language of Mobile Software is APIs
Fast and Reliable Swift APIs with gRPC
gRPC Design and Implementation
Microservices in GO - Massimiliano Dessì - Codemotion Rome 2017
REST API vs gRPC, which one should you use in breaking a monolith [Dev conf 2...
Up and Running with gRPC & Cloud Career [GDG-Cloud-Dhaka-IO/2022}
Building high performance microservices in finance with Apache Thrift
gRPC Overview
REST in Peace. Long live gRPC!
gRPC Microservices in Go (MEAP V08) Hüseyin Babal
Building microservices with grpc
Build microservice with gRPC in golang
What is gRPC introduction gRPC Explained
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
Microservices Communication Patterns with gRPC
gRPC - Fastest Data Transfer Protocol
Build your next REST API with gRPC
Ad

More from Alex Borysov (14)

PDF
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
PDF
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
PDF
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
PDF
DevNexus 2020 "Break me if you can: practical guide to building fault-toleran...
PDF
"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
PDF
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
PDF
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
PDF
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
PDF
Break me if you can: practical guide to building fault-tolerant systems (with...
PDF
"gRPC vs REST: let the battle begin!" OSCON 2018 edition
PDF
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
PDF
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
PDF
gRPC vs REST: let the battle begin!
PDF
gRPC vs REST: let the battle begin!
CloudExpo Frankfurt 2023 "Break me if you can: practical guide to building fa...
Devoxx Belgium 2022 gRPC Cornerstone: HTTP/2… or HTTP/3?
Cloud Expo Europe 2022 "Break me if you can: practical guide to building faul...
DevNexus 2020 "Break me if you can: practical guide to building fault-toleran...
"gRPC-Web: It’s All About Communication": Devoxx Belgium 2019
"gRPC-Web: It’s All About Communication": Devoxx Ukraine 2019
OSCON 2019 "Break me if you can: practical guide to building fault-tolerant s...
Devoxx Ukraine 2018 "Break me if you can: practical guide to building fault-t...
Break me if you can: practical guide to building fault-tolerant systems (with...
"gRPC vs REST: let the battle begin!" OSCON 2018 edition
"gRPC vs REST: let the battle begin!" DevoxxUK 2018 edition
"gRPC vs REST: let the battle begin!" GeeCON Krakow 2018 edition
gRPC vs REST: let the battle begin!
gRPC vs REST: let the battle begin!

Recently uploaded (20)

PDF
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
PDF
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
PDF
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
PPTX
Reimagine Home Health with the Power of Agentic AI​
PDF
top salesforce developer skills in 2025.pdf
PDF
Upgrade and Innovation Strategies for SAP ERP Customers
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
PPTX
Operating system designcfffgfgggggggvggggggggg
PPTX
L1 - Introduction to python Backend.pptx
PPTX
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
PDF
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
PDF
Design an Analysis of Algorithms I-SECS-1021-03
PDF
Softaken Excel to vCard Converter Software.pdf
PDF
Nekopoi APK 2025 free lastest update
PDF
medical staffing services at VALiNTRY
PPTX
Transform Your Business with a Software ERP System
PDF
Understanding Forklifts - TECH EHS Solution
PPTX
ai tools demonstartion for schools and inter college
EN-Survey-Report-SAP-LeanIX-EA-Insights-2025.pdf
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
Adobe Premiere Pro 2025 (v24.5.0.057) Crack free
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Reimagine Home Health with the Power of Agentic AI​
top salesforce developer skills in 2025.pdf
Upgrade and Innovation Strategies for SAP ERP Customers
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
Operating system designcfffgfgggggggvggggggggg
L1 - Introduction to python Backend.pptx
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
Claude Code: Everyone is a 10x Developer - A Comprehensive AI-Powered CLI Tool
Design an Analysis of Algorithms I-SECS-1021-03
Softaken Excel to vCard Converter Software.pdf
Nekopoi APK 2025 free lastest update
medical staffing services at VALiNTRY
Transform Your Business with a Software ERP System
Understanding Forklifts - TECH EHS Solution
ai tools demonstartion for schools and inter college

"Enabling Googley microservices with gRPC." at Devoxx France 2017

  • 1. Enabling Googley microservices with gRPC. A high performance, open source, HTTP/2-based RPC framework. April 7, 2017 Alex Borysov @aiborisov #DevoxxFR
  • 2. Google Cloud Platform Alex Borysov • Software Engineer at Google • Active gRPC user. @aiborisov #DevoxxFR
  • 3. Google Cloud Platform What is gRPC? Au commencement était la Stubby. #DevoxxFR @aiborisov
  • 4. Google Cloud Platform What is gRPC? Au commencement était la Stubby gRPC stands for gRPC Remote Procedure Calls. A high performance, open source, general purpose standards-based, feature-rich RPC framework. Open sourced version of Stubby RPC used in Google. Actively developed and production-ready, current version is 1.2.0. #DevoxxFR @aiborisov
  • 5. Google Cloud Platform Why gRPC? Photo taken by Andrey Borisenko. #DevoxxFR @aiborisov
  • 6. Google Cloud Platform gRPC vs JSON/HTTP for Google Cloud Pub/Sub 3x increase in throughput https://cloud.google.com/blog/big-data/2016/03/announcing-grpc-alpha-for-google-cloud-pubsub 11x difference per CPU #DevoxxFR @aiborisov
  • 7. Google Cloud Platform Google ~O(1010 ) QPS. #DevoxxFR @aiborisov
  • 8. Google Cloud Platform Google ~O(1010 ) QPS. Your project QPS? #DevoxxFR @aiborisov
  • 9. Google Cloud Platform Continuous Performance Benchmarking http://www.grpc.io/docs/guides/benchmarking.html 8 core VMs, unary #DevoxxFR @aiborisov 32 core VMs, unary
  • 10. Google Cloud Platform • Single TCP connection. • No Head-of-line blocking. • Binary framing layer. • Header Compression. HTTP/2 in One Slide Transport(TCP) Application (HTTP/2) Network (IP) Session (TLS) [optional] Binary Framing HEADERS Frame DATA Frame HTTP/2 POST: /upload HTTP/1.1 Host: www.javaday.org.ua Content-Type: application/json Content-Length: 27 HTTP/1.x {“msg”: “Welcome to 2017!”} #DevoxxFR @aiborisov
  • 11. Google Cloud Platform HTTP/1.x vs HTTP/2 http://www.http2demo.io/ https://http2.golang.org/gophertiles #DevoxxFR @aiborisov HTTP/1.1 HTTP/2
  • 12. Google Cloud Platform Okay, but how? #DevoxxFR @aiborisov
  • 13. Google Cloud Platform Service Definition #DevoxxFR @aiborisov
  • 14. Google Cloud Platform Service Definition (weather.proto) #DevoxxFR @aiborisov
  • 15. Google Cloud Platform Service Definition (weather.proto) syntax = "proto3"; option java_multiple_files = true; option java_package = "fr.devoxx.grpcexample"; package grpcexample; service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } #DevoxxFR @aiborisov
  • 16. Google Cloud Platform Service Definition (weather.proto) syntax = "proto3"; option java_multiple_files = true; option java_package = "fr.devoxx.grpcexample"; package grpcexample; service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } #DevoxxFR @aiborisov
  • 17. Google Cloud Platform Service Definition (weather.proto) syntax = "proto3"; service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } message WeatherRequest { Coordinates coordinates = 1; message Coordinates { fixed64 latitude = 1; fixed64 longitude = 2; } } message WeatherResponse { Temperature temperature = 1; float humidity = 2; } message Temperature { float degrees = 1; Units units = 2; enum Units { FAHRENHEIT = 0; CELSIUS = 1; KELVIN = 2; } } #DevoxxFR @aiborisov
  • 18. Google Cloud Platform Generated Classes $ ls -1 build/generated/source/proto/main/java/fr/devoxx/grpcexample/ Temperature.java TemperatureOrBuilder.java WeatherProto.java WeatherRequest.java WeatherRequestOrBuilder.java WeatherResponse.java WeatherResponseOrBuilder.java #DevoxxFR @aiborisov
  • 19. Google Cloud Platform Generated Classes $ ls -1 build/generated/source/proto/main/java/fr/devoxx/grpcexample/ Temperature.java TemperatureOrBuilder.java WeatherProto.java WeatherRequest.java WeatherRequestOrBuilder.java WeatherResponse.java WeatherResponseOrBuilder.java $ ls -1 build/generated/source/proto/main/grpc/fr/devoxx/grpcexample/ WeatherGrpc.java #DevoxxFR @aiborisov
  • 20. Google Cloud Platform Implement gRPC Service public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { } } Abstract class #DevoxxFR @aiborisov
  • 21. Google Cloud Platform Implement gRPC Service public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, public interface StreamObserver<WeatherResponse> responseObserver){{ void onNext(WeatherResponse response); void onCompleted(); void onError(Throwable error); } } } #DevoxxFR @aiborisov
  • 22. Google Cloud Platform Implement gRPC Service public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { WeatherResponse response = WeatherResponse.newBuilder() .setTemperature(Temperature.newBuilder().setDegrees(70).setUnits(FAHRENHEIT)) .setHumidity(.65f).build(); responseObserver.onNext(response); responseObserver.onCompleted(); } } #DevoxxFR @aiborisov
  • 23. Google Cloud Platform Start gRPC Server void start() throws IOException { Server grpcServer = NettyServerBuilder.forPort(8090) .addService(new WeatherService()).build() .start(); Runtime.getRuntime().addShutdownHook(new Thread(grpcServer::shutdown)); grpcServer.awaitTerminaton(); } #DevoxxFR @aiborisov
  • 24. Google Cloud Platform gRPC Clients ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel); #DevoxxFR @aiborisov
  • 25. Google Cloud Platform gRPC Clients ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherStub client = WeatherGrpc.newStub(grpcChannel); WeatherGrpc.WeatherBlockingStub blockingClient = WeatherGrpc.newBlockingStub(grpcChannel); WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel); #DevoxxFR @aiborisov
  • 26. Google Cloud Platform gRPC Sync Client WeatherRequest request = WeatherRequest.newBuilder() .setCoordinates(Coordinates.newBuilder().setLatitude(420000000) .setLongitude(-720000000)).build(); WeatherResponse response = blockingClient.getCurrent(request); logger.info("Current weather for {}: {}", request, response); #DevoxxFR @aiborisov
  • 27. Google Cloud Platform gRPC Async Client WeatherRequest request = WeatherRequest.newBuilder() .setCoordinates(Coordinates.newBuilder().setLatitude(420000000) .setLongitude(-720000000)).build(); client.getCurrent(request, new StreamObserver<WeatherResponse>() { @Override public void onNext(WeatherResponse response) { logger.info("Current weather for {}: {}", request, response); } @Override public void onError(Throwable t) { logger.info("Cannot get weather for {}", request); } @Override public void onCompleted() { logger.info("Stream completed."); } }); #DevoxxFR @aiborisov
  • 28. Google Cloud Platform Implement gRPC Service public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { WeatherResponse response = WeatherResponse.newBuilder() .setTemperature(Temperature.newBuilder().setDegrees(70).setUnits(FAHRENHEIT)) .setHumidity(.65f).build(); responseObserver.onNext(response); responseObserver.onCompleted(); } } #DevoxxFR @aiborisov
  • 29. Google Cloud Platform Adding New [Micro]Services Weather Weather Temp Wind Humidity #DevoxxFR @aiborisov
  • 30. Google Cloud Platform Service Definitions service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } service TemperatureService { rpc GetCurrent(Coordinates) returns (Temperature); } service HumidityService { rpc GetCurrent(Coordinates) returns (Humidity); } service WindService { rpc GetCurrent(Coordinates) returns (Wind); } #DevoxxFR @aiborisov
  • 31. Google Cloud Platform Service Definition - Response Message message WeatherResponse { Temperature temperature = 1; float humidity = 2; } #DevoxxFR @aiborisov
  • 32. Google Cloud Platform Adding New Field message WeatherResponse { Temperature temperature = 1; float humidity = 2; Wind wind = 3; } message Wind { Speed speed = 1; float direction = 2; } message Speed { float value = 1; Units units = 2; enum Units { MPH = 0; MPS = 1; KNOTS = 2; KMH = 3; } } #DevoxxFR @aiborisov
  • 33. Google Cloud Platform private final TemperatureServiceBlockingStub temperatureService; private final HumidityServiceBlockingStub humidityService; private final WindServiceBlockingStub windService; public WeatherService(TemperatureServiceBlockingStub temperatureService, HumidityServiceBlockingStub humidityService, WindServiceBlockingStub windService) { this.temperatureService = temperatureService; this.humidityService = humidityService; this.windService = windService; } ... Blocking Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { #DevoxxFR @aiborisov
  • 34. Google Cloud Platform ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Temperature temperature = temperatureService.getCurrent(request.getCoordinates()); Humidity humidity = humidityService.getCurrent(request.getCoordinates()); Wind wind = windService.getCurrent(request.getCoordinates()); WeatherResponse response = WeatherResponse.newBuilder().setTemperature(temperature) .setHumidity(humidity.getValue()).setWind(wind).build(); responseObserver.onNext(response); responseObserver.onCompleted(); } } Blocking Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { #DevoxxFR @aiborisov
  • 36. Google Cloud Platform private final TemperatureServiceFutureStub tempService; private final HumidityServiceFutureStub humidityService; private final WindServiceFutureStub windService; public WeatherService(TemperatureServiceFutureStub tempService, HumidityServiceFutureStub humidityService, WindServiceFutureStub windService) { this.tempService = tempService; this.humidityService = humidityService; this.windService = windService; } ... Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { #DevoxxFR @aiborisov
  • 37. Google Cloud Platform @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Coordinates coordinates = request.getCoordinates(); ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList( Futures.transform(tempService.getCurrent(coordinates), (Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()), Futures.transform(windService.getCurrent(coordinates), (Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()), Futures.transform(humidityService.getCurrent(coordinates), (Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity.getValue()).build()) ); ... Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { #DevoxxFR @aiborisov
  • 38. Google Cloud Platform @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Coordinates coordinates = request.getCoordinates(); ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList( Futures.transform(tempService.getCurrent(coordinates), (Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()), Futures.transform(windService.getCurrent(coordinates), (Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()), Futures.transform(humidityService.getCurrent(coordinates), (Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity.getValue()).build()) ); ... Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { #DevoxxFR @aiborisov
  • 39. Google Cloud Platform @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Coordinates coordinates = request.getCoordinates(); ListenableFuture<List<WeatherResponse>> responsesFuture = Futures.allAsList( Futures.transform(tempService.getCurrent(coordinates), (Temperature temp) -> WeatherResponse.newBuilder().setTemperature(temp).build()), Futures.transform(windService.getCurrent(coordinates), (Wind wind) -> WeatherResponse.newBuilder().setWind(wind).build()), Futures.transform(humidityService.getCurrent(coordinates), (Humidity humidity) -> WeatherResponse.newBuilder().setHumidity(humidity.getValue()).build()) ); ... Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { #DevoxxFR @aiborisov
  • 40. Google Cloud Platform ... Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() { @Override public void onSuccess(@Nullable List<WeatherResponse> results) { WeatherResponse.Builder response = WeatherResponse.newBuilder(); results.forEach(response::mergeFrom); responseObserver.onNext(response.build()); responseObserver.onCompleted(); } @Override public void onFailure(Throwable t) { responseObserver.onError(t); } }); } } Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { #DevoxxFR @aiborisov
  • 41. Google Cloud Platform Netty Transport ManagedChannel grpcChannel = NettyChannelBuilder.forAddress("localhost", 8090).build(); WeatherGrpc.WeatherFutureStub futureClient = WeatherGrpc.newFutureStub(grpcChannel); Server grpcServer = NettyServerBuilder.forPort(8090) .addService(new WeatherService()).build().start(); #DevoxxFR @aiborisov
  • 42. Google Cloud Platform Netty: Asynchronous and Non-Blocking IO Netty-based transport: • Multiplexes conn-s on the event loops: EpollEventLoopGroup, NioEventLoopGroup. • Decouples I/O waiting from threads. • gRPC uses Netty event loops for both client and server transports. Request Event Loop Worker thread Worker thread Worker thread Worker thread Worker thread Worker thread Worker thread Event Loop Event Loop Event Loop Callback Request Callback N e t w o r k N e t w o r k #DevoxxFR @aiborisov
  • 43. Google Cloud Platform Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ... Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() { @Override public void onSuccess(@Nullable List<WeatherResponse> results) { WeatherResponse.Builder response = WeatherResponse.newBuilder(); results.forEach(response::mergeFrom); responseObserver.onNext(response.build()); responseObserver.onCompleted(); } @Override public void onFailure(Throwable t) { responseObserver.onError(t); } }); } } #DevoxxFR @aiborisov
  • 44. Google Cloud Platform Async Future Stub Dependencies public class WeatherService extends WeatherGrpc.WeatherImplBase { @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ... Futures.addCallback(responsesFuture, new FutureCallback<List<WeatherResponse>>() { @Override public void onSuccess(@Nullable List<WeatherResponse> results) { WeatherResponse.Builder response = WeatherResponse.newBuilder(); results.forEach(response::mergeFrom); responseObserver.onNext(response.build()); responseObserver.onCompleted(); } @Override public void onFailure(Throwable t) { responseObserver.onError(t); } }); } } #DevoxxFR @aiborisov
  • 45. Google Cloud Platform Panta rhei, "everything flows". Heraclitus of Ephesus ~2,400 years Before HTTP/1.x https://en.wikipedia.org/wiki/Heraclitus “ ” #DevoxxFR @aiborisov
  • 46. Google Cloud Platform Service Definitions service Weather { rpc GetCurrent(WeatherRequest) returns (WeatherResponse); } service TemperatureService { rpc GetCurrent(Coordinates) returns (Temperature); } service HumidityService { rpc GetCurrent(Coordinates) returns (Humidity); } service WindService { rpc GetCurrent(Coordinates) returns (Wind); } #DevoxxFR @aiborisov
  • 47. Google Cloud Platform Streaming Service Definitions service WeatherStreaming { rpc GetCurrent(WeatherRequest) returns (stream WeatherResponse); } service TemperatureServiceStreaming { rpc GetCurrent(Coordinates) returns (stream Temperature); } service HumidityServiceStreaming { rpc GetCurrent(Coordinates) returns (stream Humidity); } service WindServiceStreaming { rpc GetCurrent(Coordinates) returns (stream Wind); } #DevoxxFR @aiborisov
  • 48. Google Cloud Platform Bidirectional Streaming Service Definitions service WeatherStreaming { rpc GetCurrent(stream WeatherRequest) returns (stream WeatherResponse); } service TemperatureServiceStreaming { rpc GetCurrent(stream Coordinates) returns (stream Temperature); } service HumidityServiceStreaming { rpc GetCurrent(stream Coordinates) returns (stream Humidity); } service WindServiceStreaming { rpc GetCurrent(stream Coordinates) returns (stream Wind); } #DevoxxFR @aiborisov
  • 49. Google Cloud Platform Bidirectional Streaming Service Definitions service WeatherStreaming { rpc Observe(stream WeatherRequest) returns (stream WeatherResponse); } service TemperatureServiceStreaming { rpc Observe(stream Coordinates) returns (stream Temperature); } service HumidityServiceStreaming { rpc Observe(stream Coordinates) returns (stream Humidity); } service WindServiceStreaming { rpc Observe(stream Coordinates) returns (stream Wind); } #DevoxxFR @aiborisov
  • 50. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { #DevoxxFR @aiborisov
  • 51. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = temperatureService.observe(new StreamObserver<Temperature>() { @Override public void onNext(Temperature temperature) { WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build() responseObserver.onNext(resp); } @Override public void onError(Throwable t) { responseObserver.onError(t); } @Override public void onCompleted() { responseObserver.onCompleted(); } }); ... } #DevoxxFR @aiborisov
  • 52. Google Cloud Platform Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = temperatureService.observe(new StreamObserver<Temperature>() { @Override public void onNext(Temperature temperature) { WeatherResponse resp = WeatherResponse.newBuilder().setTemperature(temperature).build() responseObserver.onNext(resp); } @Override public void onError(Throwable t) { responseObserver.onError(t); } @Override public void onCompleted() { responseObserver.onCompleted(); } }); ... } #DevoxxFR @aiborisov
  • 53. Google Cloud Platform return new StreamObserver<WeatherRequest>() { @Override public void onNext(WeatherRequest request) { temperatureClientStream.onNext(request.getCoordinates()); } @Override public void onError(Throwable t) { temperatureClientStream.onError(t); } @Override public void onCompleted() { temperatureClientStream.onCompleted(); } }; } Bidirectional Streaming Service public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { StreamObserver<Coordinates> temperatureClientStream = ... #DevoxxFR @aiborisov
  • 54. 54 Messaging applications. Games / multiplayer tournaments. Moving objects. Sport results. Stock market quotes. Smart home devices. You name it! Streaming Use-Cases #DevoxxFR @aiborisov
  • 55. Google Cloud Platform Continuous Performance Benchmarking http://www.grpc.io/docs/guides/benchmarking.html 8 core VMs, streaming #DevoxxFR @aiborisov 32 core VMs, streaming
  • 56. Google Cloud Platform gRPC Speaks Your Language ● Java ● Go ● C/C++ ● C# ● Node.js ● PHP ● Ruby ● Python ● Objective-C ● MacOS ● Linux ● Windows ● Android ● iOS Service definitions and client libraries Platforms supported #DevoxxFR @aiborisov
  • 58. 58 Things will break! Don’t pretend you can eliminate every possible source of failure. Integration points are #1 killer of software systems. Fault Tolerance? #DevoxxFR @aiborisov
  • 59. Google Cloud Platform Use time-outs! Because your code cannot wait forever. Captain “ ”* Original photo taken by Stanislav Vedmid . #DevoxxFR @aiborisov
  • 60. Google Cloud Platform Timeouts? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms ? ? ? ? ? ? ? ? ? #DevoxxFR @aiborisov
  • 61. Google Cloud Platform Default Timeout For Every Service? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms 200 ms #DevoxxFR @aiborisov
  • 62. Google Cloud Platform Default Timeout For Every Service? Gateway 20 ms 10 ms 10 ms FastFastFast 30 ms 40 ms 20 ms 130 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout #DevoxxFR @aiborisov
  • 63. Google Cloud Platform Default Timeout For Every Service? Gateway 30 ms 80 ms 100 ms SlowFair 210 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout #DevoxxFR @aiborisov
  • 64. Google Cloud Platform Default Timeout For Every Service? 30 ms 80 ms 100 ms SlowFair 210 ms 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout #DevoxxFR @aiborisov
  • 65. Google Cloud Platform Default Timeout For Every Service? 80 ms 100 ms SlowFair 200 ms timeout 200 ms timeout 200 ms timeout 200 ms timeout Still working! #DevoxxFR @aiborisov
  • 66. Google Cloud Platform Default Timeout For Every Service? BusyBusy 200 ms timeout 200 ms timeout 200 ms timeout Idle Busy #DevoxxFR @aiborisov
  • 67. Google Cloud Platform Default Timeout For Every Service? BusyBusy 200 ms timeout 200 ms timeout 200 ms timeout Idle Busy R e t r y #DevoxxFR @aiborisov
  • 68. Google Cloud Platform Default Timeout For Every Service? BusyBusy 200 ms timeout 200 ms timeout 200 ms timeout Idle Busy R e t r y #DevoxxFR @aiborisov
  • 69. Google Cloud Platform Default Timeout For Every Service? 200 ms timeout 200 ms timeout 200 ms timeout R e t r y #DevoxxFR @aiborisov
  • 70. Google Cloud Platform Tune Timeout For Every Service? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms 190 ms 190 ms 190 ms 110 ms 50 ms 110 ms 50 ms 110 ms 50 ms #DevoxxFR @aiborisov
  • 71. Google Cloud Platform Tune Timeout For Every Service? Gateway 200 ms timeout 190 ms timeout 110 ms timeout 30 ms 80 ms 25 ms FastFair 55 ms Fast 30 ms 50 ms timeout #DevoxxFR @aiborisov
  • 72. Google Cloud Platform Tune Timeout For Every Service? Gateway 200 ms timeout 190 ms timeout 110 ms timeout 30 ms 80 ms 25 ms Fair 55 ms Fast 30 ms 50 ms timeout #DevoxxFR @aiborisov
  • 73. Google Cloud Platform Timeouts? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms ? ? ? ? ? ? ? ? ? #DevoxxFR @aiborisov
  • 74. Google Cloud Platform Timeouts for Real World? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms #DevoxxFR @aiborisov
  • 75. Google Cloud Platform Timeouts for Real World? GW A1 A2 A3 B1 B2 B3 C1 C2 C3 200 ms 2,500 ms 140 ms #DevoxxFR @aiborisov
  • 77. 77 gRPC Java does not support timeouts. Timeouts in gRPC #DevoxxFR @aiborisov
  • 78. 78 gRPC Java does not support timeouts. gRPC supports deadlines instead! gRPC Deadlines WeatherResponse response = client.withDeadlineAfter(200, MILLISECONDS) .getCurrent(request); #DevoxxFR @aiborisov
  • 79. 79 First-class feature in gRPC. Deadline is an absolute point in time. Deadline indicates to the server how long the client is willing to wait for an answer. RPC will fail with DEADLINE_EXCEEDED status code when deadline reached. gRPC Deadlines #DevoxxFR @aiborisov
  • 80. Google Cloud Platform gRPC Deadline Propagation Gateway 90 ms Now = 1476600000000 Deadline = 1476600000200 Remaining = 200 40 ms 20 ms 20 ms 60 ms withDeadlineAfter(200, MILLISECONDS) DEADLINE_EXCEEDED DEADLINE_EXCEEDED DEADLINE_EXCEEDED DEADLINE_EXCEEDED Now = 1476600000040 Deadline = 1476600000200 Remaining = 160 Now = 1476600000150 Deadline = 1476600000200 Remaining = 50 Now = 1476600000230 Deadline = 1476600000200 Remaining = -30 #DevoxxFR @aiborisov
  • 81. 81 Deadlines are automatically propagated! Can be accessed by the receiver! gRPC Deadlines Context context = Context.current(); context.getDeadline().isExpired(); context.getDeadline() .timeRemaining(MILLISECONDS); context.getDeadline().runOnExpiration(() -> logger.info("Deadline exceeded!"), exec); #DevoxxFR @aiborisov
  • 82. 82 Deadlines are expected. What about unpredictable cancellations? • User cancelled request. • Caller is not interested in the result any more. • etc Cancellation? #DevoxxFR @aiborisov
  • 83. Google Cloud Platform Cancellation? GW Busy Busy Busy Busy Busy Busy Busy Busy Busy RPC Active RPC Active RPC Active RPC Active RPC Active RPCActive RPC Active RPC Active RPC Active RPC #DevoxxFR @aiborisov
  • 84. Google Cloud Platform Cancellation? GW Busy Busy Busy Busy Busy Busy Busy Busy Busy Active RPC Active RPC Active RPC Active RPC Active RPCActive RPC Active RPC Active RPC Active RPC #DevoxxFR @aiborisov
  • 85. Google Cloud Platform Cancellation Propagation GW Idle Idle Idle Idle Idle Idle Idle Idle Idle #DevoxxFR @aiborisov
  • 86. 86 Automatically propagated. RPC fails with CANCELLED status code. Cancellation status can be accessed by the receiver. Server (receiver) always knows if RPC is valid! gRPC Cancellation #DevoxxFR @aiborisov
  • 87. Google Cloud Platform gRPC Cancellation public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ServerCallStreamObserver<WeatherResponse> streamObserver = (ServerCallStreamObserver<WeatherResponse>) responseObserver; streamObserver.isCancelled(); ... #DevoxxFR @aiborisov
  • 88. Google Cloud Platform gRPC Cancellation public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { ServerCallStreamObserver<WeatherResponse> streamObserver = (ServerCallStreamObserver<WeatherResponse>) responseObserver; streamObserver.setOnCancelHandler(() -> { cleanupResources(); logger.info("Call cancelled by client!"); }); ... #DevoxxFR @aiborisov
  • 89. Google Cloud Platform gRPC Context public class WeatherService extends WeatherGrpc.WeatherImplBase { ... @Override public void getCurrent(WeatherRequest request, StreamObserver<WeatherResponse> responseObserver) { Context.current().getDeadline(); Context.current().isCancelled(); Context.current().cancellationCause(); Context.current().addListener(context -> { cleanupResources(); logger.info("Call cancelled by client!"); }, executor) ... #DevoxxFR @aiborisov
  • 90. Google Cloud Platform More Control? #DevoxxFR @aiborisov
  • 91. Google Cloud Platform BiDi Streaming - Slow Client Fast Server Request Responses Slow Client CANCELLED UNAVAILABLE RESOURCE_EXHAUSTED #DevoxxFR @aiborisov
  • 92. Google Cloud Platform BiDi Streaming - Slow Server Slow Server Request Response Fast Client CANCELLED UNAVAILABLE RESOURCE_EXHAUSTED Requests #DevoxxFR @aiborisov
  • 93. Google Cloud Platform Client-Side Flow-Control Server Request, count = 4 Client 2 Responses 4 Responses Count = 2 #DevoxxFR @aiborisov
  • 94. Google Cloud Platform Server-Side Flow-Control Server Request Client Response count = 2 2 Requests count = 3 3 Requests #DevoxxFR @aiborisov
  • 95. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver outboundStream) { outboundStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3); // Request up to 3 responses from server #DevoxxFR @aiborisov
  • 96. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver outboundStream) { outboundStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3); // Request up to 3 responses from server #DevoxxFR @aiborisov
  • 97. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver outboundStream) { outboundStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3); // Request up to 3 responses from server #DevoxxFR @aiborisov
  • 98. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver outboundStream) { outboundStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3); // Request up to 3 responses from server #DevoxxFR @aiborisov
  • 99. Google Cloud Platform Flow-Control (Client-Side) CallStreamObserver<WeatherRequest> requestStream = (CallStreamObserver) client.observe(new ClientResponseObserver<WeatherRequest, WeatherResponse>() { @Override public void beforeStart(ClientCallStreamObserver outboundStream) { outboundStream.disableAutoInboundFlowControl(); } @Override public void onNext(WeatherResponse response) { processResponse(response); } @Override public void onError(Throwable e) { logger.error("Error on weather request.", e); } @Override public void onCompleted() { logger.info("Stream completed."); } }); requestStream.onNext(request); requestStream.request(3); // Request up to 3 responses from server #DevoxxFR @aiborisov
  • 100. Google Cloud Platform Flow-Control (Client-Side) public class WeatherStreamingService extends WeatherStreamingGrpc.WeatherStreamingImplBase { ... @Override public StreamObserver<WeatherRequest> observe(StreamObserver<WeatherResponse> responseObserver) { ServerCallStreamObserver<WeatherResponse> streamObserver = (ServerCallStreamObserver<WeatherResponse>) responseObserver; streamObserver.setOnReadyHandler(() -> { if (streamObserver.isReady()) { streamObserver.onNext(calculateWeather()); } }); ... #DevoxxFR @aiborisov
  • 101. 101 Flow-control helps to balance computing power and network capacity between client and server. gRPC supports both client- and server-side flow control. Disabled by default. Flow-Control Photo taken by Andrey Borisenko. #DevoxxFR @aiborisov
  • 102. Google Cloud Platform What else? #DevoxxFR @aiborisov
  • 103. 103 Service Discovery Load Balancing Tracing, debugging Monitoring Testing Microservices? #DevoxxFR @aiborisov
  • 104. Google Cloud Platform Service Discovery & Load Balancing HTTP/2 RPC Server-side App Service Definition (extends generated definition) ServerCall handler TransportTransport ServerCall RPC Client-Side App Channel Stub Future Stub Blocking Stub ClientCall #DevoxxFR @aiborisov
  • 105. Google Cloud Platform HTTP/2 Service Discovery & Load Balancing RPC Client-Side App Instance #1 Instance #N Instance #2 RPC Server-side Apps Transport Channel Stub Future Stub Blocking Stub ClientCall #DevoxxFR @aiborisov
  • 106. Google Cloud Platform Service Discovery & Load Balancing HTTP/2 RPC Client-Side App Channel Stub Future Stub Blocking Stub ClientCall RPC Server-side Apps Tran #1 Tran #2 Tran #N Service Definition (extends generated definition) ServerCall handler Transport ServerCall NameResolver LoadBalancer Pluggable Load Balancing and Service Discovery #DevoxxFR @aiborisov
  • 107. Google Cloud Platform Service Discovery & Load Balancing ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv") .nameResolverFactory(new DnsNameResolverProvider()) .loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance()) .build(); #DevoxxFR @aiborisov
  • 108. Google Cloud Platform Service Discovery & Load Balancing ManagedChannel grpcChannel = NettyChannelBuilder.forTarget("WeatherSrv") .nameResolverFactory(new DnsNameResolverProvider()) .loadBalancerFactory(RoundRobinLoadBalancerFactory.getInstance()) .build(); Design documents: • Load Balancing in gRPC: https://github.com/grpc/grpc/blob/master/doc/load-balancing.md • gRPC Java Name Resolution and Load Balancing v2: http://tiny.cc/grpc-java-lb-v2 #DevoxxFR @aiborisov
  • 109. Google Cloud Platform Testing Support In-process server transport: fully-featured, high performance, and useful in testing. In-process client side channel: fully-featured, high performance, and useful in testing. #DevoxxFR @aiborisov
  • 110. Google Cloud Platform Testing Support - InProcess WeatherServiceAsync weatherService = new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(weatherService).build(); #DevoxxFR @aiborisov
  • 111. Google Cloud Platform Testing Support - InProcess WeatherServiceAsync weatherService = new WeatherServiceAsync(tempService, humidityService, windService); Server grpcServer = InProcessServerBuilder.forName("weather") .addService(weatherService).build(); Channel grpcChannel = InProcessChannelBuilder.forName("weather").build(); WeatherBlockingStub stub = WeatherGrpc.newBlockingStub(grpcChannel).withDeadlineAfter(100, MILLISECONDS); #DevoxxFR @aiborisov
  • 112. Google Cloud Platform Testing Support In-process server transport and client-side channel StreamRecorder - StreamObserver that records all the observed values and errors. MetadataUtils for testing metadata headers and trailer. grpc-testing - utility functions for unit and integration testing: https://github.com/grpc/grpc-java/tree/master/testing #DevoxxFR @aiborisov
  • 113. Google Cloud Platform Distributed Tracing A B C D E F G H J #DevoxxFR @aiborisov
  • 114. Google Cloud Platform Distributed Tracing A B C D E F G H J #DevoxxFR @aiborisov
  • 115. Google Cloud Platform Distributed Tracing with Zipkin/Brave https://github.com/openzipkin/brave/tree/master/brave-grpc // server-side Server grpcServer = NettyServerBuilder.forPort(8090) .addService(ServerInterceptors.intercept(new WeatherService(), new BraveGrpcServerInterceptor(brave("wind")))) .build(); // client-side Channel windChannel = NettyChannelBuilder.forAddress(windserverAddress()).build(); WindServiceStub windClient = WindServiceGrpc.newStub( ClientInterceptors.intercept(windChannel, new BraveGrpcClientInterceptor(brave("weather_to_wind")))); #DevoxxFR @aiborisov
  • 116. Google Cloud Platform More Features Interceptors to add cross-cutting behavior: • Client- and server-side Monitoring: • gRPC Prometheus. • grpcz-monitoring • More Monitoring APIs are coming. Built-in authentication mechanisms: • SSL/TLS. • Token-based authentication with Google. • Authentication API. Payload compression: https://github.com/grpc/grpc/blob/master/doc/compression.md #DevoxxFR @aiborisov
  • 117. Google Cloud Platform Growing Community and Ecosystem https://github.com/grpc-ecosystem Polyglot is a universal grpc command line client. grpc-gateway generates a reverse-proxy server which translates a RESTful JSON API into gRPC. OpenTracing is a set of consistent, expressive, vendor-neutral APIs for distributed tracing and context propagation. Prometheus monitoring support for grpc-java and grpc-go. #DevoxxFR @aiborisov
  • 118. Google Cloud Platform Try It Out! http://grpc.io Twitter @grpcio gRPC on Github: https://github.com/grpc Google group: grpc-io@googlegroups.com gRPC Java quickstart: http://www.grpc.io/docs/quickstart/java.html gRPC Java tutorial: http://www.grpc.io/docs/tutorials/basic/java.html gRPC contribution: https://github.com/grpc/grpc-contrib #DevoxxFR @aiborisov
  • 119. Google Cloud Platform Takeaways Microservices are not a free lunch! gRPC (http://grpc.io): • HTTP/2 transport based, open source, general purpose standards-based, feature-rich RPC framework. • Bidirectional streaming over one single TCP connection. • Netty transport provides asynchronous and non-blocking I/O. • Deadline and cancellations propagation. • Client and server-side flow-control. • Pluggable service discovery and load-balancing • Supports 10 programming languages. • Build-in testing support. • Production-ready (current version is 1.2.0) and growing ecosystem. #DevoxxFR @aiborisov
  • 121. Google Cloud Platform Images Used Special thanks for the provided photos: • Andrey Borisenko • Stanislav Vedmid Photo from https://www.google.com/about/datacenters page: • https://www.google.com/about/datacenters/gallery/#/tech/12 Photos licenced by Creative Commons 2.0 https://creativecommons.org/licenses/by/2.0/ : • https://www.flickr.com/photos/garryknight/5754661212/ by Garry Knight • https://www.flickr.com/photos/sodaigomi/21128888345/ by sodai gomi • https://www.flickr.com/photos/zengame/15972170944/ by Zengame • https://www.flickr.com/photos/londonmatt/18496249193/ by Matt Brown • https://www.flickr.com/photos/gazeronly/8058105980/ by torbakhopper #DevoxxFR @aiborisov