Streams on top of scala - #lambdaCon
Download as PPTX, PDF4 likes2,250 views
The document discusses the significance of streams in modern data processing, highlighting their relevance due to increasing data sizes and the necessity for real-time actions. It critiques traditional file handling and Java stream management, advocating for more efficient methods like iteratees and scalaz streams, which facilitate better data stream management. The presentation concludes with a call to embrace reactive streams technologies such as Akka Streams for enterprise compatibility and enhanced design.
Streams on top of scala - #lambdaCon
- 1. STREAMS ON TOP OF SCALA BY @WAXZCE – QUENTIN ADAM LAMBDACON 2015 BOLOGNA
- 2. Quentin ADAM from the Clever Cloud @waxzce on twitter – github- soundcloud – instagram …. WHO AM I ?
- 3. MY DAY TO DAY WORK : CLEVER CLOUD, MAKE YOUR APP RUN ALL THE TIME
- 4. And learn a lot of things about your code, apps, and good/bad design… KEEP YOUR APPS ONLINE. MADE WITH NODE.JS, SCALA, JAVA, RUBY, PHP, PYTHON, GO OR DOCKER…
- 5. AND LEARN A LOT OF THINGS ABOUT YOUR CODE, APPS, AND GOOD/BAD DESIGN…
- 6. WHY ARE STREAMS SO TRENDY THESE DAYS?
- 8. DATA IS BECOMING BIGGER AND BIGGER
- 9. WE NEED TO ACT WHILE DATA TRANSFERRING IS OCCURRING
- 10. &
- 11. WE ARE NOW USING LOTS OF PERMANENT CONNECTIONS
- 13. WEBSOCKETS
- 14. HTTP/2
- 15. EXAMPLE
- 16. WHAT IS INSIDE AN HTTP REQUEST ? Verb • The action Resource • The object of the action Headers • The context of the action Body • Optional • The datas
- 17. IN MANY CASES THE REQUEST IS MANIPULATED ALL FROM MEMORY
- 18. UPLOADING A 20 GB FILE ON A POST REQUEST AND STORING IT EXAMPLE
- 19. IT’S A BAD IDEA TO PUT THE BODY PART IN MEMORY
- 20. CREATE A TEMP FILE TO STORE THE DATA
- 23. DO NOT USE THE FILE SYSTEM AS A DATASTORE File systems are POSIX compliant • POSIX is ACID • POSIX is powerful but is a bottleneck • File System is the nightmare of ops • File System creates coupling (host provider/OS/language) • SPOF-free multi tenant File System is a unicorn
- 24. Client uploads file Create a temp file Send a file in a backend Then answer to the client The file is transferring two times
- 25. NOT SO GOOD
- 26. Client uploads file Directly stream it to the backend Then answer to the client
- 27. LET’S ACT ON STREAMS!
- 29. CLASSIC JAVA STREAM MANAGEMENT • Buffers • Buffer management • Buffer exception • Buffer overflow
- 30. CLASSIC JAVA STREAM MANAGEMENT • Low performances if not buffered • Not modular • Thread blocking • Code is ugly • No back pressure • Error handling is bad • i/o management and business code are mixed
- 31. I CAN’T SLEEP AT NIGHT
- 32. WE HAVE TO FIND A BETTER WAY
- 35. Like a recipe Consume the data ITERATEE : HOW TO MANAGE A STREAM
- 36. Produce the data ENUMERATOR : DATA STREAM
- 37. Set of tools to do cool things with Iteratee and Enumerator ENUMERATEE
- 38. SIMPLE EXAMPLE TO START
- 39. WHAT’S OUR GOAL • Get Reactive Manifesto words • Stream it to an iteratee to group it by pair of word • Print it
- 40. AN ENUMERATOR
- 41. AN ITERATEE
- 44. AN ITERATEE
- 45. AN ITERATEE TO MANAGE THE STREAM
- 47. AN ITERATEE
- 52. TYPE SAFE
- 53. COMPOSITION
- 54. WHAT’S TRENDY? BUT ITERATEES ARE GOING TO DIE
- 55. SCALAZ STREAMS
- 56. WHAT’S IN THE BOX ?
- 58. PROCESS IS KEY
- 59. SIMPLY USE THE COLLECTION API
- 60. DEMO
- 61. BINDINGS WITH • http with (netty or http4s) • scodec • …
- 62. BUT WE NEED TO TALK WITH OTHERS…
- 63. LIKE OUR FRIENDS ON TOP OF JVM
- 65. + POLITICS
- 67. AKKA STREAMS CURRENT 1.0-M5 (MILESTONE) (M5 OUT YESTEDAY)
- 68. BASIC EXAMPLE
- 69. BASIC EXAMPLE
- 70. DEMO AGAIN
- 71. MORE POSSIBILITIES Grab from rabbitMQ Parse Event Check event ACL from DB Find reliable WS Send data to WS
- 72. ARE YOU CONVINCE TO USE STREAM ?
- 73. GO NOW Iteratee Scalaz- streams Akka- stream • learn • Act now on play! • Beautiful design • The most scala-ish way to do • Enterprise compatible • Wide ecosystem • Reactive stream ready
- 74. DATA STREAMING PROGRAMMING IS NOW TRENDING
- 75. THX TO @CLEMENTD CLEMENT DELAFARGUE
- 76. I’m @waxzce on twitter I’m the CEO of A PaaS provider, give it a try ;-) THX FOR LISTENING & QUESTIONS TIME Coupon for Clever Cloud trial : lambdacon