Introduction to Akka-Streams
- 2. PREVIOUS AGE OF COMPUTING SYSTEMS • Monolith software architecture • Managed servers and containers • RDBMS, transactions isolation • Scalability: scale-up by more powerful hardware • Proprietary enterprise solutions
- 3. NEW CHALLENGES • response time: s -> ms • high availability: 3 nines (8h/y) -> 5+ nines (5– m/y) • storage: GBs (10 9 ) -> PBs (10 15 )+ • hardware: spread from mobile phone to 1000+ nodes cluster
- 4. MOORE’S LAW
- 5. EVIL OF CPU PERFORMANCE: MOORE’S LAW DOESN’T WORK ANYMORE FOR A SINGLE CORE
- 6. The more CPU cores a system has, the faster it is. True or false?
- 7. EVIL OF PARALLELISM: AMDAHL’S LAW Gene Amdahl 1967
- 8. STUBBORN AMDAHL’S LAW The speedup of a program using multiple processors in parallel computing is limited by the sequential fraction of the program.
- 9. EVIL OF CONCURRENCY: SHARED MUTABLE STATE
- 10. «Do not communicate by sharing memory. Instead, share memory by communicating» – Effective Go
- 11. v1.0: July 2013 v2.0: Sept 2014 REACTIVE SOFTWARE DESIGN http://www.reactivemanifesto.org/
- 12. • rapid and consistent response times • response in SLA time may be more important than late correct response • problems may be detected quickly and dealt with effectively react to users Responsive
- 13. react to load Elastic • No contention points or central bottlenecks • ability to shard or replicate components and distribute inputs among them • Automatic resource management • scale-up • scale-out • scale-down • scale-in
- 14. • “Let it crash!”: there is no way to think about every single failure point • failure is not dealt with as an error • means that some capacity of the system will be reduced • dealt with as a message react to failures Resilient
- 15. Asynchronous message-passing • loose coupling • isolation of components • location transparency • failures as messages Benefits: • load management • elasticity • flow control (monitoring , back-pressure) react to events Message Driven
- 16. Application should be reactive from top to bottom
- 17. v1.0: July 2013 v2.0: Sept 2014 REACTIVE SOFTWARE DESIGN http://www.reactivemanifesto.org/
- 18. CARL HEWITT, 1973 ACTOR MODEL • Describes: • message processing algorithm • data storage principles • interaction between modules • Erlang • Used in telecommunication systems • High Availability of 9 “nines”
- 19. • Written in Scala • Stable since 2009 • Part of Scala Standard Library since 2013 • Supports Java 8 since 2014 by ex
- 20. WHAT IS AN ACTOR? Actor is a unit of code with a mailbox and an internal state that just responds to messages in a single thread (briefly)
- 21. IDLE TWO STATES OF ACTOR
- 22. IDLE TWO STATES OF ACTOR MESSAGE PROCESSING
- 23. WHAT IS AN ACTOR? • Similar to object in OOP, but message-driven • Even more isolated than object: no explicit access exposed (hidden behind ActorRef) • no shared state • location transparent (can live in different JVMs) • Light-weight: 300B memory footprint (millions per GB) • No threads allocated in idle state • Single-threaded invocation inside, sequential message processing • Supervises children actors
- 24. WHAT CAN ACTOR DO? • If no messages being processed: • Nothing • When message being processed (one at a time): • make local decisions • send messages to other actors (incl. sender() and self()) • do other actions with side-effects (IO, logs, DB access) • change own behavior for next messages • create more actors (and promise to supervise them)
- 25. BENEFITS • You’re not going to have multiple threads modifying a variable at the same time. • Forget about: • Shared state • Threads • Locks, “synchronized” • Concurrent collections • wait/notify/notifyAll • Describe only business behavior in the code. • Akka and app configuration care about everything else.
- 26. ACTOR AND ACTOR MESSAGE purely-immutable serializable
- 27. ACTORSYSTEM AND ENTRY POINT
- 28. OTHER FEATURES OF AKKA • Supervision model • Routing via configuration • Remote actors via configuration • Clustering via configuration • etc.
- 29. WEAKNESS OF AKKA #1: NO DSL FOR DATA FLOW AND TYPE-UNSAFE MESSAGE PASSING
- 30. ACTORS BECAME JUST ANOTHER CONCURRENCY PRIMITIVE AND SHOULD BE HIDDEN BEHIND HIGHER-LEVEL ABSTRACTIONS
- 31. WEAKNESS OF AKKA #2: AGGRESSIVE STRATEGIES OF DEALING WITH BUFFER OVERFLOW
- 33. Asynchronous boundary DON’T PUSH BACK – JUST DON’T ASK BACK-PRESSURE PUBLISHER SUBSCRIBER Demand Data
- 34. HOW CAN PRODUCER DEAL WITH BACK-PRESSURE? • Slow down producing data (if it can) • Buffer (using bounded buffer, of course) • Drop elements • Fail
- 36. PARTIALLY BACK-PRESSURED STREAM THE WHOLE STREAM SHOULD BE BACK-PRESSURED
- 37. REACTIVE STREAMS GOALS • minimalistic • asynchronous • never block (waste time/resources) • safe (back-threads pressure) • purely local abstraction • allow synchronous implementations • compatible with TCP
- 38. REACTIVE STREAMS • 2013 – “reactive non-blocking asynchronous back-pressure” • 2015 – JEP-266: Introduced to OpenJDK 9 by Doug Lea http://openjdk.java.net/jeps/266
- 40. AKKA-STREAMS • Implements Reactive Streams, transparently for the user • Uses Akka under the hood: • Derived actors can be configured using akka configuration • Clustering, networking, HTTP, profiling tools
- 41. DEMO PRINT OUT LIST(“HELLO”, “WORLD”)
- 42. AKKA-STREAMS MAIN BUILDING BLOCKS Source Flow Sink
- 43. AKKA-STREAMS ALGEBRA • Source + Sink = RunnableGraph • Source + Flow = (composite) Source • Flow + Sink = (composite) Sink • Flow + Flow = (composite) Flow
- 44. DEMO PRINT (1 TO 10) MULTIPLICATION BY 2
- 45. MATERIALIZATION
- 46. MATERIALIZATION
- 47. MATERIALIZATION
- 49. DEMO ACTORREF AS MATERIALIZED VALUE SUM OF EVEN NUMBERS
- 50. GRAPH DSL MORE COMPLEX SHAPES Fan-In Fan-Out BidiFlow UniformFanInShape UniformFanOutShape FanInShape1 FanInShape2 FanInShapeN FanOutShape1 FanOutShape2 FanOutShapeN
- 51. PLAIN FLOW DSL VS GRAPH DSL
- 54. SOURCES OUT OF THE BOX
- 55. SINKS OUT OF THE BOX
- 56. OTHER FEATURES OF AKKA-STREAMS • mapAsync for external calls • Custom shapes • Subgraph customisation with Attributes • Custom processing with GraphStage • Operator Fusing • Supervision Strategies via ActorMaterializerSettings • Integration with other RS implementations • Streams TestKit
- 57. RESOURCES Akka: http://akka.io/ http://lightbend.com http://letitcrash.com Principles of Reactive Programming @ Coursera Akka Streams: http://doc.akka.io/docs/akka/current/scala/stream/
- 58. THANKS! Q & A