SlideShare a Scribd company logo

JDD 2016 - Michał Balinski, Oleksandr Goldobin - Practical Non Blocking Microservices in Java 8

PROIDEA, profile picture
PROIDEA

The document discusses the implementation of practical non-blocking microservices in Java 8, focusing on use cases for Over-the-Air (OTA) gateways in IoT environments that require scalability and resilience. It compares blocking and non-blocking approaches to processing script submissions, detailing the challenges and benefits of each methodology, particularly in terms of resource management and performance. The authors emphasize that while non-blocking processing can enhance efficiency, it is crucial to assess the specific needs and implications before adoption.

Technology
1 of 46
Download to read offline
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
Practical non-blocking microservices in Java 8 Michał Baliński System Architect Oleksandr Goldobin System Architect
Cloud of microservices for secure IoT gateway backing service core service gateway gateway core service backing service
The demand and characteristics of our domain Crowded IoT environment Slow, long lived connections Big amount of concurrent connections Scalability and resilience Rather I/O intensive than CPU intensive
OTA Gateway – Use Case TSM Trusted Service Manager
OTA Gateway – Use Case TSM Trusted Service Manager Security Module
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 3. store scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 4. submition response 3. store scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 4. submition response HTTP 5. poll scripts 3. store scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 3. store scripts 4. submition response HTTP 5. poll scripts 6. search scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 3. store scripts 4. submition response HTTP 5. poll scripts 6. search scripts 7. response with scripts
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP HTTP HTTP TCP
OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB Log Storage HTTP HTTP HTTP TCP File I/OMonitoring System HTTP
Let’s test blocking approach!
OTA Gateway Blocking - technologies 17 TSM DB Log Storage HTTP TCP STDOUT OTA Gateway JAX-RS Logback appender Security Module JAX-RS Jedis JAX-RS Client
OTA Gateway – Blocking - test OTA (Over-the-Air) Gateway send 2 scripts per request Security Module DB Log Storage HTTP HTTP HTTP TCP File I/O emulated latency 200 ms expected latency < 450 ms verified with throughput over 7k req/s
Blocking – 1k threads OTA (Over-the-Air) Gateway send 2 scripts per request Security Module DB Log Storage HTTP HTTP HTTP TCP File I/O emulated latency 200 ms max 1000 connections max 1000 threads expected latency < 450 ms
Blocking – 1k threads 500 req/s 1000 req/s 1500 req/s 2000 req/s
The drawbacks of classic synchronous I/O One thread per connection Threads waiting instead of running Context switches Resource wasting (~1 MB per thread - 64bit)
Let’s switch from blocking to non-blocking
OTA Gateway Non-blocking - technologies 23 TSM DB Log Storage HTTP TCP STDOUT OTA Gateway JAX-RS 2.0 Logback async appender Async Http Client 2 (Netty) Security Module JAX-RS 2.0 Lettuce (Netty)
Non-blocking – 16 threads OTA (Over-the-Air) Gateway send 2 scripts per request Security Module DB Log Storage HTTP HTTP HTTP TCP File I/O emulated latency 200 ms no limit for connections max 16 threads expected latency < 450 ms
Non-blocking – 16 threads 1k req/s 2k req/s 2.5k req/s 3k req/s
Blocking Non-blocking 1000 threads 56 threads 1.2 GB 0.5 GB ~1.2k r/s 2.5k r/s
Let’s talk about challenges
OTA Gateway Blocking – sequence diagram OTA Gateway TSM Security Module DB Logs loop 1. submit scripts 2. encrypt script 3a. store script 4. submition response 3b. count scripts
OTA Gateway Non-blocking – sequence diagram OTA Gateway TSM Security Module DB Logs loop 1. submit scripts 2. encrypt script 3a. store script 4. submition response 3b. count scripts
OTA Non-blocking – realityOTA Gateway TSM Security Module DB Logs „loopedprocessingchain” 1. submit scripts 4. submition response 3b. count scripts HTTP Server 2. encrypt script 3a. store script Logging DB Client Security Client
Code. Bird view 3110 October 2016 package org.demo.ota.blocking.rest; @Path("se") public class ScriptSubmissionResource extends Application { private static final Logger log = LoggerFactory.getLogger(ScriptSubmissionResource.class); private static final ResourceMetrics METRICS = new ResourceMetrics("ota_submission"); private SecureModuleClient secureModuleClient = SecureModuleClient.instance(); private ScriptStorageClient scriptStorageClient = ScriptStorageClient.instance(); @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public long submitScripts(@PathParam("seId") String seId, List<Script> scripts) { MDC.put("flow", "submission"); MDC.put("se", seId); return METRICS.instrument(() -> { log.debug("Processing {} scripts submission", scripts.size(), seId); for (int i = 0; i < scripts.size(); i++) { final Script script = scripts.get(i); log.debug("Encrypting {} script", i); final String encryptedPayload = secureModuleClient.encrypt(seId, script.getPayload()); script.setPayload(encryptedPayload); log.debug("Storing encrypted script {}", i); scriptStorageClient.storeScript(seId, script); } long numberOfScripts = scriptStorageClient.numberOfScriptsForSe(seId); log.debug("Request processed", seId); return numberOfScripts; }); } @Override public Set<Object> getSingletons() { return Collections.singleton(this); } } package org.demo.ota.nonblocking.rest; @Path("se") public class ScriptSubmissionResource extends Application { private static final Logger log = LoggerFactory.getLogger(ScriptSubmissionResource.class); private static final ResourceMetrics METRICS = new ResourceMetrics("ota_submission"); private SecureModuleClient secureModuleClient = SecureModuleClient.instance(); private ScriptStorageClient scriptStorageClient = ScriptStorageClient.instance(); @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public void submitScripts( @PathParam("seId") String seId, List<Script> scripts, @Suspended final AsyncResponse asyncResponse ) { final DiagnosticContext diagnosticContext = new DiagnosticContext("submission", seId); METRICS.instrumentStage(() -> { log.debug("{} Processing {} scripts submission", diagnosticContext, scripts.size()); return encryptAndStoreAllScripts(diagnosticContext, seId, scripts) .thenCompose( ignore -> scriptStorageClient.numberOfScriptsForSe(seId) ); }) .whenComplete((numberOfScripts, e) -> { if (e != null) { asyncResponse.resume(e); } else { log.debug("{} Request processed", diagnosticContext); asyncResponse.resume(numberOfScripts); } }); } private CompletionStage<Void> encryptAndStoreAllScripts( final DiagnosticContext diagnosticContext, final String seId, final List<Script> scripts ) { CompletionStage<Void> stage = null; // <- non final field, potential concurrent access bug! for (int i = 0; i < scripts.size(); i++) { final int scriptIndex = i; final Script script = scripts.get(scriptIndex); if (stage == null) { stage = encryptAndStoreSingleScript(diagnosticContext, seId, scriptIndex, script); } else { stage = stage.thenCompose(ignore -> encryptAndStoreSingleScript(diagnosticContext, seId, scriptIndex, script)); } } return stage; } private CompletionStage<Void> encryptAndStoreSingleScript( final DiagnosticContext diagnosticContext, final String seId, final int scriptIndex, final Script script ) { log.debug("{} Encrypting script {}", diagnosticContext, scriptIndex); return secureModuleClient .encrypt(seId, script.getPayload()) .thenCompose( encryptedPayload -> { log.debug("{} Storing encrypted script {}", diagnosticContext, scriptIndex); return scriptStorageClient.storeScript(seId, new Script(encryptedPayload)); } ); } @Override public Set<Object> getSingletons() { return new HashSet<>(Collections.singletonList(this)); } }
Code. Blocking. Submission 1 10 October 2016 32 @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public long submitScripts(@PathParam("seId") String seId, List<Script> scripts) { MDC.put("flow", "submission"); // ß Setting diagnostic context MDC.put("se", seId); return METRICS.instrument(() -> { // ß Instrumenting with metrics //...
Code. Blocking. Submission 2 10 October 2016 33 log.debug("Processing {} scripts submission", scripts.size(), seId); for (int i = 0; i < scripts.size(); i++) { ß Cycle through the scripts final Script script = scripts.get(i); log.debug("Encrypting {} script", i); final String encryptedPayload = secureModuleClient .encrypt(seId, script.getPayload()); ß Encrypting the script script.setPayload(encryptedPayload); log.debug("Storing encrypted script {}", i); scriptStorageClient.storeScript(seId, script); ß Saving the script into DB } long numberOfScripts = scriptStorageClient.numberOfScriptsForSe(seId); ß Getting current number of scripts in DB log.debug("Request processed", seId); return numberOfScripts;
Code. Non-blocking. Submission 1 10 October 2016 34 @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public void submitScripts( @PathParam("seId") String seId, List<Script> scripts, @Suspended final AsyncResponse asyncResponse ) { final DiagnosticContext diagnosticContext = new DiagnosticContext("submission", seId); ß Creating diagnostic context METRICS.instrumentStage(() -> { ß Instrumenting with metrics
Code. Non-blocking. Submission 1 10 October 2016 35 @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public void submitScripts( @PathParam("seId") String seId, List<Script> scripts, @Suspended final AsyncResponse asyncResponse ) { final DiagnosticContext diagnosticContext = new DiagnosticContext("submission", seId); ß Creating diagnostic context METRICS.instrumentStage(() -> { ß Instrumenting with metrics
Code. Non-blocking. Submission 2 10 October 2016 36 METRICS.instrumentStage(() -> { log.debug( "{} Processing {} scripts submission", diagnosticContext, scripts.size()); return encryptAndStoreAllScripts(diagnosticContext, seId, scripts) .thenCompose( ignore -> scriptStorageClient.numberOfScriptsForSe(seId) ); }) .whenComplete((numberOfScripts, e) -> { if (e != null) { asyncResponse.resume(e); } else { log.debug("{} Request processed", diagnosticContext); asyncResponse.resume(numberOfScripts); } });
Code. Non-blocking. Submission 3 10 October 2016 37 private CompletionStage<Void> encryptAndStoreAllScripts( final DiagnosticContext diagnosticContext, final String seId, final List<Script> scripts ) { CompletionStage<Void> stage = null; // <- non final field, potential // concurrent access bug! for (int i = 0; i < scripts.size(); i++) { ß Cycle through the scripts final int scriptIndex = i; final Script script = scripts.get(scriptIndex); if (stage == null) { stage = encryptAndStoreSingleScript( diagnosticContext, seId, scriptIndex, script); } else { stage = stage.thenCompose(ignore -> encryptAndStoreSingleScript( diagnosticContext, seId, scriptIndex, script)); } } return stage; }
Code. Non-blocking. Submission 4 10 October 2016 38 private CompletionStage<Void> encryptAndStoreSingleScript( final DiagnosticContext diagnosticContext, final String seId, final int scriptIndex, final Script script ) { log.debug("{} Encrypting script {}", diagnosticContext, scriptIndex); return secureModuleClient .encrypt(seId, script.getPayload()) ß Encrypting the script .thenCompose( encryptedPayload -> { log.debug( "{} Storing encrypted script {}", diagnosticContext, scriptIndex); return scriptStorageClient.storeScript( ß Saving the script into seId, the DB new Script(encryptedPayload)); } ); }
Code. Blocking. Integration 10 October 2016 39 private final JedisPool pool; public void storeScript(String seId, Script script) { try (Jedis jedis = pool.getResource()) { jedis.rpush(seId, script.getPayload()); } } public long numberOfScriptsForSe(String seId) { try (Jedis jedis = pool.getResource()) { return jedis.llen(seId); } } public Optional<Script> nextScript(String seId) { try (Jedis jedis = pool.getResource()) { return Optional.ofNullable(jedis.lpop(seId)).map(Script::new); } } public String encrypt(String keyDiversifier, String payload) { // ... }
Code. Non-Blocking. Integration 10 October 2016 40 private final RedisAsyncCommands<String, String> commands; public CompletionStage<Void> storeScript(String seId, Script script) { return commands .rpush(seId, script.getPayload()) .thenApply(ignore -> null); } public CompletionStage<Long> numberOfScriptsForSe(String seId) { return commands.llen(seId); } public CompletionStage<Optional<String>> nextScript(String seId) { return commands.lpop(seId).thenApply(Optional::ofNullable); } public CompletionStage<String> encrypt(String keyDiversifier, String payload) { // ... }
Diagnostics in non-blocking systems No clear stack traces, need for good logs Name your threads properly MDC becomes useless (thread locals) Explicitly pass debug context to trace flows Be prepared for debuging non-obvious errors
NIO technology landscape JDK 1.4 NIO JDK 1.7 NIO.2 JAX-RS 2.x Servlet API 3.x
Lessons learned, part 1 Vanila Java 8 for NIO µ-services Netty best for custom protocols in NIO Unit tests should be synchronous Load/stress testing is a must Make bulkheading and plan your resources
Lessons learned, part 2 Functional programming patterns for readability Immutability as 1-st class citizen Scala may be good choice ;-)
Conclusion Non-blocking processing can really save your resources (== money) But! Weight all pros and cons and use non-blocking processing only if you really need it.
Thank you. Michał Baliński m.balinski@oberthur.com Oleksandr Goldobin o.goldobin@oberthur.com goldobin @goldobin balonus @MichalBalinski Readings: • https://github.com/balonus/blocking-vs-nonblocking-demo • C10k Problem, C10M Problem, Asynchronous I/O • Boost application performance using asynchronous I/O (M. Tim Jones, 2006) • Zuul 2 : The Netflix Journey to Asynchronous, Non-Blocking Systems (Netflix, 2016) • Thousands of Threads and Blocking I/O: The Old Way to Write Java Servers Is New Again (and Way Better) (Paul Tyma, 2008) • Why Non-Blocking? (Bozhidar Bozhanov, 2011)

More Related Content

PDF
Control Plane: Security Rationale for Istio (DevSecOps - London Gathering, Ja...
Michael Man
 
PDF
Chris Rutter: Avoiding The Security Brick
Michael Man
 
PDF
DAST в CI/CD, Ольга Свиридова
Mail.ru Group
 
PPTX
Security in NodeJS applications
Daniel Garcia (a.k.a cr0hn)
 
PPT
Securing Network Access with Open Source solutions
Nick Owen
 
PDF
London HUG 19/5 - Kubernetes and vault
London HashiCorp User Group
 
PDF
Onward15
sarah_nadi
 
PPTX
Do WAFs dream of static analyzers
Vladimir Kochetkov
 
Control Plane: Security Rationale for Istio (DevSecOps - London Gathering, Ja...
Michael Man
 
Chris Rutter: Avoiding The Security Brick
Michael Man
 
DAST в CI/CD, Ольга Свиридова
Mail.ru Group
 
Security in NodeJS applications
Daniel Garcia (a.k.a cr0hn)
 
Securing Network Access with Open Source solutions
Nick Owen
 
London HUG 19/5 - Kubernetes and vault
London HashiCorp User Group
 
Onward15
sarah_nadi
 
Do WAFs dream of static analyzers
Vladimir Kochetkov
 

What's hot (20)

PPTX
Toni de la Fuente - Automate or die! How to survive to an attack in the Cloud...
RootedCON
 
PPTX
[OPD 2019] Side-Channels on the Web:
Attacks and Defenses
OWASP
 
PPTX
New and smart way to develop microservice for istio with micro profile
Emily Jiang
 
ODP
Secure coding in C#
Siddharth Bezalwar
 
PDF
Microservices with Micronaut
QAware GmbH
 
PDF
Understanding Windows Access Token Manipulation
Justin Bui
 
PPTX
Cloud nativeworkshop
Emily Jiang
 
PDF
OWASP Poland Day 2018 - Amir Shladovsky - Crypto-mining
OWASP
 
PDF
[Wroclaw #9] The purge - dealing with secrets in Opera Software
OWASP
 
PDF
Hardening cassandra for compliance or paranoia
zznate
 
PDF
SSL/TLS for Mortals (JavaZone)
Maarten Mulders
 
PDF
[OPD 2019] Attacking JWT tokens
OWASP
 
PDF
Codetainer: a Docker-based browser code 'sandbox'
Jen Andre
 
PDF
iOS Keychain by 흰, 민디
MINJICHO20
 
PDF
OpenSSL programming (still somewhat initial version)
Shteryana Shopova
 
PDF
SSL/TLS for Mortals (GOTO Berlin)
Maarten Mulders
 
PDF
Spring Security Patterns
VMware Tanzu
 
PDF
Athens IoT meetup #7 - Create the Internet of your Things - Laurent Ellerbach...
Athens IoT Meetup
 
PPT
На страже ваших денег и данных
Positive Hack Days
 
KEY
Openssl
psychesnet Hsieh
 
Toni de la Fuente - Automate or die! How to survive to an attack in the Cloud...
RootedCON
 
[OPD 2019] Side-Channels on the Web:
Attacks and Defenses
OWASP
 
New and smart way to develop microservice for istio with micro profile
Emily Jiang
 
Secure coding in C#
Siddharth Bezalwar
 
Microservices with Micronaut
QAware GmbH
 
Understanding Windows Access Token Manipulation
Justin Bui
 
Cloud nativeworkshop
Emily Jiang
 
OWASP Poland Day 2018 - Amir Shladovsky - Crypto-mining
OWASP
 
[Wroclaw #9] The purge - dealing with secrets in Opera Software
OWASP
 
Hardening cassandra for compliance or paranoia
zznate
 
SSL/TLS for Mortals (JavaZone)
Maarten Mulders
 
[OPD 2019] Attacking JWT tokens
OWASP
 
Codetainer: a Docker-based browser code 'sandbox'
Jen Andre
 
iOS Keychain by 흰, 민디
MINJICHO20
 
OpenSSL programming (still somewhat initial version)
Shteryana Shopova
 
SSL/TLS for Mortals (GOTO Berlin)
Maarten Mulders
 
Spring Security Patterns
VMware Tanzu
 
Athens IoT meetup #7 - Create the Internet of your Things - Laurent Ellerbach...
Athens IoT Meetup
 
На страже ваших денег и данных
Positive Hack Days
 
Openssl
psychesnet Hsieh
 
Ad

Viewers also liked (15)

PDF
JDD 2016 - Bartosz Majsak - Meet The Assertable Chaos Monkeys
PROIDEA
 
PPT
JDD 2016 - Jacek Bukowski - "Flying To Clouds" - Can It Be Easy?
PROIDEA
 
PDF
JDD 2016 - Ondrej Mihalyi - How to bake reactive behavior into your Java EE ...
PROIDEA
 
PDF
JDD 2016 - Jakub Kubrynski - Jpa - beyond copy-paste
PROIDEA
 
PDF
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PROIDEA
 
PDF
Blazing Fast Feedback Loops in the Java Universe
Michał Kordas
 
PDF
How to write your database: the story about Event Store
Victor Haydin
 
PDF
JDD 2016 - Pawel Byszewski - Kotlin, why?
PROIDEA
 
PDF
JDD 2016 - Christin Gorman - Concurrency in Java
PROIDEA
 
PPTX
2016 - Daniel Lebrero - REPL driven development
PROIDEA
 
PDF
JDD 2016 - Pawel Szulc - Writing Your Wwn RDD For Fun And Profit
PROIDEA
 
PDF
JDD 2016 - Grzegorz Piwowarek - Davaslang - Functional Java Done Right
PROIDEA
 
PDF
JDD 2016 - Grzegorz Rozniecki - Java 8 What Could Possibly Go Wrong
PROIDEA
 
PDF
JDD 2016 - Marcin Stozek - Docker. Przewodnik dla poczatkujacych
PROIDEA
 
PPTX
JDD 2016 - Wojciech Oczkowski - Testowanie Wydajnosci Za Pomoca Narzedzia JMH
PROIDEA
 
JDD 2016 - Bartosz Majsak - Meet The Assertable Chaos Monkeys
PROIDEA
 
JDD 2016 - Jacek Bukowski - "Flying To Clouds" - Can It Be Easy?
PROIDEA
 
JDD 2016 - Ondrej Mihalyi - How to bake reactive behavior into your Java EE ...
PROIDEA
 
JDD 2016 - Jakub Kubrynski - Jpa - beyond copy-paste
PROIDEA
 
JDD 2016 - Tomasz Borek - DB for next project? Why, Postgres, of course
PROIDEA
 
Blazing Fast Feedback Loops in the Java Universe
Michał Kordas
 
How to write your database: the story about Event Store
Victor Haydin
 
JDD 2016 - Pawel Byszewski - Kotlin, why?
PROIDEA
 
JDD 2016 - Christin Gorman - Concurrency in Java
PROIDEA
 
2016 - Daniel Lebrero - REPL driven development
PROIDEA
 
JDD 2016 - Pawel Szulc - Writing Your Wwn RDD For Fun And Profit
PROIDEA
 
JDD 2016 - Grzegorz Piwowarek - Davaslang - Functional Java Done Right
PROIDEA
 
JDD 2016 - Grzegorz Rozniecki - Java 8 What Could Possibly Go Wrong
PROIDEA
 
JDD 2016 - Marcin Stozek - Docker. Przewodnik dla poczatkujacych
PROIDEA
 
JDD 2016 - Wojciech Oczkowski - Testowanie Wydajnosci Za Pomoca Narzedzia JMH
PROIDEA
 
Ad

Similar to JDD 2016 - Michał Balinski, Oleksandr Goldobin - Practical Non Blocking Microservices in Java 8 (20)

PPTX
Practical non blocking microservices in java 8
Michal Balinski
 
PDF
10 Excellent Ways to Secure Spring Boot Applications - Okta Webinar 2020
Matt Raible
 
PPTX
Top Ten Java Defense for Web Applications v2
Jim Manico
 
PDF
2 years with python and serverless
Hector Canto
 
PDF
Securing Microservices using Play and Akka HTTP
Rafal Gancarz
 
PDF
Ria Spring Blaze Ds
Skills Matter
 
PPTX
OpenStack Security Project
Travis McPeak
 
PPTX
Containerless in the Cloud with AWS Lambda
Ryan Cuprak
 
PDF
Positive Technologies - S4 - Scada under x-rays
qqlan
 
PPTX
Scala at Netflix
Manish Pandit
 
PDF
WSO2 SOA with C and C++
WSO2
 
PPTX
OWASP_Top_Ten_Proactive_Controls_v2.pptx
cgt38842
 
PPT
SQL Server Security - Attack
webhostingguy
 
PPTX
OWASP_Top_Ten_Proactive_Controls version 2
ssuser18349f1
 
PPTX
OWASP_Top_Ten_Proactive_Controls_v2.pptx
azida3
 
PPTX
OWASP_Top_Ten_Proactive_Controls_v2.pptx
johnpragasam1
 
PPT
Shmoocon 2013 - OpenStack Security Brief
openfly
 
PPTX
Building Your Own IoT Platform using FIWARE GEis
FIWARE
 
PPTX
OWASP_Top_Ten_Proactive_Controls_v32.pptx
nmk42194
 
KEY
Using Apache as an Application Server
Phil Windley
 
Practical non blocking microservices in java 8
Michal Balinski
 
10 Excellent Ways to Secure Spring Boot Applications - Okta Webinar 2020
Matt Raible
 
Top Ten Java Defense for Web Applications v2
Jim Manico
 
2 years with python and serverless
Hector Canto
 
Securing Microservices using Play and Akka HTTP
Rafal Gancarz
 
Ria Spring Blaze Ds
Skills Matter
 
OpenStack Security Project
Travis McPeak
 
Containerless in the Cloud with AWS Lambda
Ryan Cuprak
 
Positive Technologies - S4 - Scada under x-rays
qqlan
 
Scala at Netflix
Manish Pandit
 
WSO2 SOA with C and C++
WSO2
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
cgt38842
 
SQL Server Security - Attack
webhostingguy
 
OWASP_Top_Ten_Proactive_Controls version 2
ssuser18349f1
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
azida3
 
OWASP_Top_Ten_Proactive_Controls_v2.pptx
johnpragasam1
 
Shmoocon 2013 - OpenStack Security Brief
openfly
 
Building Your Own IoT Platform using FIWARE GEis
FIWARE
 
OWASP_Top_Ten_Proactive_Controls_v32.pptx
nmk42194
 
Using Apache as an Application Server
Phil Windley
 

Recently uploaded (20)

PPTX
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
PDF
Encapsulation theory and applications.pdf
gurumoop
 
PDF
KodekX | Application Modernization Development
KodekX
 
PDF
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
PPTX
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
PDF
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
PDF
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
PDF
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
PDF
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
PDF
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
PPTX
Cloud computing and distributed systems.
kkkkkhan
 
PDF
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
PPTX
Spectroscopy.pptx food analysis technology
nawahassan45
 
PPT
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
PDF
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
PDF
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
PPTX
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
PDF
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
PDF
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
DOCX
The AUB Centre for AI in Media Proposal.docx
GAONE9
 
Big Data Technologies - Introduction.pptx
kuthubussaman1
 
Encapsulation theory and applications.pdf
gurumoop
 
KodekX | Application Modernization Development
KodekX
 
Peak of Data & AI Encore- AI for Metadata and Smarter Workflows
Safe Software
 
20250228 LYD VKU AI Blended-Learning.pptx
DatVoNgoc
 
The Rise and Fall of 3GPP – Time for a Sabbatical?
3G4G
 
Chapter 3 Spatial Domain Image Processing.pdf
Getnet Tigabie Askale -(GM)
 
Machine learning based COVID-19 study performance prediction
IAESIJAI
 
cuic standard and advanced reporting.pdf
SimoneTitaniumTomase
 
7 ChatGPT Prompts to Help You Define Your Ideal Customer Profile.pdf
SOFTTECHHUB
 
Cloud computing and distributed systems.
kkkkkhan
 
Blue Purple Modern Animated Computer Science Presentation.pdf.pdf
Akar16
 
Spectroscopy.pptx food analysis technology
nawahassan45
 
“AI and Expert System Decision Support & Business Intelligence Systems”
ZubinRadhakrishnan
 
Building Integrated photovoltaic BIPV_UPV.pdf
SandeepSingh286037
 
Reach Out and Touch Someone: Haptics and Empathic Computing
Mark Billinghurst
 
Effective Security Operations Center (SOC) A Modern, Strategic, and Threat-In...
ReZa AdineH
 
Empathic Computing: Creating Shared Understanding
Mark Billinghurst
 
Optimiser vos workloads AI/ML sur Amazon EC2 et AWS Graviton
Julien SIMON
 
The AUB Centre for AI in Media Proposal.docx
GAONE9
 

JDD 2016 - Michał Balinski, Oleksandr Goldobin - Practical Non Blocking Microservices in Java 8

  • 1. Practical non-blocking microservices in Java 8 Michał Baliński System Architect Oleksandr Goldobin System Architect
  • 2. Cloud of microservices for secure IoT gateway backing service core service gateway gateway core service backing service
  • 3. The demand and characteristics of our domain Crowded IoT environment Slow, long lived connections Big amount of concurrent connections Scalability and resilience Rather I/O intensive than CPU intensive
  • 4. OTA Gateway – Use Case TSM Trusted Service Manager
  • 5. OTA Gateway – Use Case TSM Trusted Service Manager Security Module
  • 6. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB
  • 7. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts
  • 8. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts
  • 9. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 3. store scripts
  • 10. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 4. submition response 3. store scripts
  • 11. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 4. submition response HTTP 5. poll scripts 3. store scripts
  • 12. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 3. store scripts 4. submition response HTTP 5. poll scripts 6. search scripts
  • 13. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP 1. submit scripts HTTP 2. encrypt scripts TCP 3. store scripts 4. submition response HTTP 5. poll scripts 6. search scripts 7. response with scripts
  • 14. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB HTTP HTTP HTTP TCP
  • 15. OTA Gateway – Use Case OTA (Over-the-Air) Gateway TSM Trusted Service Manager Security Module DB Log Storage HTTP HTTP HTTP TCP File I/OMonitoring System HTTP
  • 17. OTA Gateway Blocking - technologies 17 TSM DB Log Storage HTTP TCP STDOUT OTA Gateway JAX-RS Logback appender Security Module JAX-RS Jedis JAX-RS Client
  • 18. OTA Gateway – Blocking - test OTA (Over-the-Air) Gateway send 2 scripts per request Security Module DB Log Storage HTTP HTTP HTTP TCP File I/O emulated latency 200 ms expected latency < 450 ms verified with throughput over 7k req/s
  • 19. Blocking – 1k threads OTA (Over-the-Air) Gateway send 2 scripts per request Security Module DB Log Storage HTTP HTTP HTTP TCP File I/O emulated latency 200 ms max 1000 connections max 1000 threads expected latency < 450 ms
  • 20. Blocking – 1k threads 500 req/s 1000 req/s 1500 req/s 2000 req/s
  • 21. The drawbacks of classic synchronous I/O One thread per connection Threads waiting instead of running Context switches Resource wasting (~1 MB per thread - 64bit)
  • 22. Let’s switch from blocking to non-blocking
  • 23. OTA Gateway Non-blocking - technologies 23 TSM DB Log Storage HTTP TCP STDOUT OTA Gateway JAX-RS 2.0 Logback async appender Async Http Client 2 (Netty) Security Module JAX-RS 2.0 Lettuce (Netty)
  • 24. Non-blocking – 16 threads OTA (Over-the-Air) Gateway send 2 scripts per request Security Module DB Log Storage HTTP HTTP HTTP TCP File I/O emulated latency 200 ms no limit for connections max 16 threads expected latency < 450 ms
  • 25. Non-blocking – 16 threads 1k req/s 2k req/s 2.5k req/s 3k req/s
  • 27. Let’s talk about challenges
  • 28. OTA Gateway Blocking – sequence diagram OTA Gateway TSM Security Module DB Logs loop 1. submit scripts 2. encrypt script 3a. store script 4. submition response 3b. count scripts
  • 29. OTA Gateway Non-blocking – sequence diagram OTA Gateway TSM Security Module DB Logs loop 1. submit scripts 2. encrypt script 3a. store script 4. submition response 3b. count scripts
  • 30. OTA Non-blocking – realityOTA Gateway TSM Security Module DB Logs „loopedprocessingchain” 1. submit scripts 4. submition response 3b. count scripts HTTP Server 2. encrypt script 3a. store script Logging DB Client Security Client
  • 31. Code. Bird view 3110 October 2016 package org.demo.ota.blocking.rest; @Path("se") public class ScriptSubmissionResource extends Application { private static final Logger log = LoggerFactory.getLogger(ScriptSubmissionResource.class); private static final ResourceMetrics METRICS = new ResourceMetrics("ota_submission"); private SecureModuleClient secureModuleClient = SecureModuleClient.instance(); private ScriptStorageClient scriptStorageClient = ScriptStorageClient.instance(); @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public long submitScripts(@PathParam("seId") String seId, List<Script> scripts) { MDC.put("flow", "submission"); MDC.put("se", seId); return METRICS.instrument(() -> { log.debug("Processing {} scripts submission", scripts.size(), seId); for (int i = 0; i < scripts.size(); i++) { final Script script = scripts.get(i); log.debug("Encrypting {} script", i); final String encryptedPayload = secureModuleClient.encrypt(seId, script.getPayload()); script.setPayload(encryptedPayload); log.debug("Storing encrypted script {}", i); scriptStorageClient.storeScript(seId, script); } long numberOfScripts = scriptStorageClient.numberOfScriptsForSe(seId); log.debug("Request processed", seId); return numberOfScripts; }); } @Override public Set<Object> getSingletons() { return Collections.singleton(this); } } package org.demo.ota.nonblocking.rest; @Path("se") public class ScriptSubmissionResource extends Application { private static final Logger log = LoggerFactory.getLogger(ScriptSubmissionResource.class); private static final ResourceMetrics METRICS = new ResourceMetrics("ota_submission"); private SecureModuleClient secureModuleClient = SecureModuleClient.instance(); private ScriptStorageClient scriptStorageClient = ScriptStorageClient.instance(); @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public void submitScripts( @PathParam("seId") String seId, List<Script> scripts, @Suspended final AsyncResponse asyncResponse ) { final DiagnosticContext diagnosticContext = new DiagnosticContext("submission", seId); METRICS.instrumentStage(() -> { log.debug("{} Processing {} scripts submission", diagnosticContext, scripts.size()); return encryptAndStoreAllScripts(diagnosticContext, seId, scripts) .thenCompose( ignore -> scriptStorageClient.numberOfScriptsForSe(seId) ); }) .whenComplete((numberOfScripts, e) -> { if (e != null) { asyncResponse.resume(e); } else { log.debug("{} Request processed", diagnosticContext); asyncResponse.resume(numberOfScripts); } }); } private CompletionStage<Void> encryptAndStoreAllScripts( final DiagnosticContext diagnosticContext, final String seId, final List<Script> scripts ) { CompletionStage<Void> stage = null; // <- non final field, potential concurrent access bug! for (int i = 0; i < scripts.size(); i++) { final int scriptIndex = i; final Script script = scripts.get(scriptIndex); if (stage == null) { stage = encryptAndStoreSingleScript(diagnosticContext, seId, scriptIndex, script); } else { stage = stage.thenCompose(ignore -> encryptAndStoreSingleScript(diagnosticContext, seId, scriptIndex, script)); } } return stage; } private CompletionStage<Void> encryptAndStoreSingleScript( final DiagnosticContext diagnosticContext, final String seId, final int scriptIndex, final Script script ) { log.debug("{} Encrypting script {}", diagnosticContext, scriptIndex); return secureModuleClient .encrypt(seId, script.getPayload()) .thenCompose( encryptedPayload -> { log.debug("{} Storing encrypted script {}", diagnosticContext, scriptIndex); return scriptStorageClient.storeScript(seId, new Script(encryptedPayload)); } ); } @Override public Set<Object> getSingletons() { return new HashSet<>(Collections.singletonList(this)); } }
  • 32. Code. Blocking. Submission 1 10 October 2016 32 @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public long submitScripts(@PathParam("seId") String seId, List<Script> scripts) { MDC.put("flow", "submission"); // ß Setting diagnostic context MDC.put("se", seId); return METRICS.instrument(() -> { // ß Instrumenting with metrics //...
  • 33. Code. Blocking. Submission 2 10 October 2016 33 log.debug("Processing {} scripts submission", scripts.size(), seId); for (int i = 0; i < scripts.size(); i++) { ß Cycle through the scripts final Script script = scripts.get(i); log.debug("Encrypting {} script", i); final String encryptedPayload = secureModuleClient .encrypt(seId, script.getPayload()); ß Encrypting the script script.setPayload(encryptedPayload); log.debug("Storing encrypted script {}", i); scriptStorageClient.storeScript(seId, script); ß Saving the script into DB } long numberOfScripts = scriptStorageClient.numberOfScriptsForSe(seId); ß Getting current number of scripts in DB log.debug("Request processed", seId); return numberOfScripts;
  • 34. Code. Non-blocking. Submission 1 10 October 2016 34 @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public void submitScripts( @PathParam("seId") String seId, List<Script> scripts, @Suspended final AsyncResponse asyncResponse ) { final DiagnosticContext diagnosticContext = new DiagnosticContext("submission", seId); ß Creating diagnostic context METRICS.instrumentStage(() -> { ß Instrumenting with metrics
  • 35. Code. Non-blocking. Submission 1 10 October 2016 35 @POST @Path("/{seId}/scripts") @Consumes(MediaType.APPLICATION_JSON) @Produces(MediaType.TEXT_PLAIN) public void submitScripts( @PathParam("seId") String seId, List<Script> scripts, @Suspended final AsyncResponse asyncResponse ) { final DiagnosticContext diagnosticContext = new DiagnosticContext("submission", seId); ß Creating diagnostic context METRICS.instrumentStage(() -> { ß Instrumenting with metrics
  • 36. Code. Non-blocking. Submission 2 10 October 2016 36 METRICS.instrumentStage(() -> { log.debug( "{} Processing {} scripts submission", diagnosticContext, scripts.size()); return encryptAndStoreAllScripts(diagnosticContext, seId, scripts) .thenCompose( ignore -> scriptStorageClient.numberOfScriptsForSe(seId) ); }) .whenComplete((numberOfScripts, e) -> { if (e != null) { asyncResponse.resume(e); } else { log.debug("{} Request processed", diagnosticContext); asyncResponse.resume(numberOfScripts); } });
  • 37. Code. Non-blocking. Submission 3 10 October 2016 37 private CompletionStage<Void> encryptAndStoreAllScripts( final DiagnosticContext diagnosticContext, final String seId, final List<Script> scripts ) { CompletionStage<Void> stage = null; // <- non final field, potential // concurrent access bug! for (int i = 0; i < scripts.size(); i++) { ß Cycle through the scripts final int scriptIndex = i; final Script script = scripts.get(scriptIndex); if (stage == null) { stage = encryptAndStoreSingleScript( diagnosticContext, seId, scriptIndex, script); } else { stage = stage.thenCompose(ignore -> encryptAndStoreSingleScript( diagnosticContext, seId, scriptIndex, script)); } } return stage; }
  • 38. Code. Non-blocking. Submission 4 10 October 2016 38 private CompletionStage<Void> encryptAndStoreSingleScript( final DiagnosticContext diagnosticContext, final String seId, final int scriptIndex, final Script script ) { log.debug("{} Encrypting script {}", diagnosticContext, scriptIndex); return secureModuleClient .encrypt(seId, script.getPayload()) ß Encrypting the script .thenCompose( encryptedPayload -> { log.debug( "{} Storing encrypted script {}", diagnosticContext, scriptIndex); return scriptStorageClient.storeScript( ß Saving the script into seId, the DB new Script(encryptedPayload)); } ); }
  • 39. Code. Blocking. Integration 10 October 2016 39 private final JedisPool pool; public void storeScript(String seId, Script script) { try (Jedis jedis = pool.getResource()) { jedis.rpush(seId, script.getPayload()); } } public long numberOfScriptsForSe(String seId) { try (Jedis jedis = pool.getResource()) { return jedis.llen(seId); } } public Optional<Script> nextScript(String seId) { try (Jedis jedis = pool.getResource()) { return Optional.ofNullable(jedis.lpop(seId)).map(Script::new); } } public String encrypt(String keyDiversifier, String payload) { // ... }
  • 40. Code. Non-Blocking. Integration 10 October 2016 40 private final RedisAsyncCommands<String, String> commands; public CompletionStage<Void> storeScript(String seId, Script script) { return commands .rpush(seId, script.getPayload()) .thenApply(ignore -> null); } public CompletionStage<Long> numberOfScriptsForSe(String seId) { return commands.llen(seId); } public CompletionStage<Optional<String>> nextScript(String seId) { return commands.lpop(seId).thenApply(Optional::ofNullable); } public CompletionStage<String> encrypt(String keyDiversifier, String payload) { // ... }
  • 41. Diagnostics in non-blocking systems No clear stack traces, need for good logs Name your threads properly MDC becomes useless (thread locals) Explicitly pass debug context to trace flows Be prepared for debuging non-obvious errors
  • 43. Lessons learned, part 1 Vanila Java 8 for NIO µ-services Netty best for custom protocols in NIO Unit tests should be synchronous Load/stress testing is a must Make bulkheading and plan your resources
  • 44. Lessons learned, part 2 Functional programming patterns for readability Immutability as 1-st class citizen Scala may be good choice ;-)
  • 46. Thank you. Michał Baliński m.balinski@oberthur.com Oleksandr Goldobin o.goldobin@oberthur.com goldobin @goldobin balonus @MichalBalinski Readings: • https://github.com/balonus/blocking-vs-nonblocking-demo • C10k Problem, C10M Problem, Asynchronous I/O • Boost application performance using asynchronous I/O (M. Tim Jones, 2006) • Zuul 2 : The Netflix Journey to Asynchronous, Non-Blocking Systems (Netflix, 2016) • Thousands of Threads and Blocking I/O: The Old Way to Write Java Servers Is New Again (and Way Better) (Paul Tyma, 2008) • Why Non-Blocking? (Bozhidar Bozhanov, 2011)