Concurrency and Distributed Systems Using JRuby
3 likes842 views
This document discusses concurrency and distributed systems in JRuby. It introduces Java concurrency utilities like Executors for managing thread pools, and blocking queues for producer-consumer patterns. It also discusses actors as an abstraction for concurrency and Mikka for implementing actors in JRuby. Distributed systems like Cassandra and Storm are presented, along with Redstorm for using Storm in JRuby. The key lessons are to separate concurrent and distributed logic from application code, and to leverage tools that handle lower-level concurrency details.
Concurrency and Distributed Systems Using JRuby
- 1. CONCURRENCY AND DISTRIBUTED SYSTEMS IN JRUBY @iconara
- 2. speakerdeck.com/u/iconara (real time!)
- 5. big data with Ruby since 2009
- 6. MAKING THIS
- 7. INTO THIS
- 8. big data with Ruby since 2009
- 9. APRIL 2011 stuck
- 10. MAY 2011 let’s try JRuby
- 11. JUNE 2011 wrapping Java libraries for fun and for profit
- 12. HOTBUNNIES natural Ruby API for RabbitMQ
- 13. JULY 2011 threads are better than processes
- 14. JRUBY
- 15. JRUBY
- 17. SHORTCUT Java Concurrency in Practice
- 18. EXECUTORS configurable thread pools, a better abstraction than using Thread directly
- 19. thread_pool = Executors.new_fixed_thread_pool(16) all_the_things.each do |thing| thread_pool.submit do crunch_crunch_crunch(thing) end end
- 20. BLOCKING QUEUES easy producer/consumer patterns
- 21. J.U.C.ATOMIC optimistic locking primitives, compare-and-swap, thread safe non-blocking counters
- 22. MORE AWESOMES ConcurrentHashMap, CountDownLatch, ForkJoinPool, Google Guava, LMAX distruptor
- 23. TL; DR use j.u.c and avoid mutable state and you’re safe, mostly
- 25. AUGUST 2011 threads are a mess, let’s try actors
- 26. ACTORS a higher level abstraction of concurrency
- 27. AKKA concurrency library for Scala, famous actor implementation
- 28. MIKKA Akka wrapper for JRuby
- 29. DO ALL THE THINGS Programming Concurrency on the JVM
- 30. AUGUST 2011 async systems with no back pressure mechanism will die of memory starvation
- 31. INPUT AGGREGATION TIMER STATE PERSISTENCE OUTPUT
- 32. INPUT AGGREGATION TIMER ! STATE PERSISTENCE OUTPUT
- 33. AUGUST 2011 actors are cool, but not really worth it when all the arrows in your flow chart point in the same direction
- 34. DIVIDE DIVIDE DIVIDE run multiple independent, synchronous workers, tune to the number of CPUs & IO wait
- 35. SEPTEMBER 2011 blocking all the way down, back pressure all the way up
- 36. AUGUST 2011 (short rant about MongoDB’s global write lock)
- 37. NEW SHINY TOYS for building new shiny products
- 38. CASSANDRA distributed database, mostly magic
- 39. ColumnFamilyUpdater<String, String> updater = template.createUpdater("a key"); updater.setString("domain", "www.datastax.com"); updater.setLong("time", System.currentTimeMillis()); try { template.update(updater); } catch (HectorException e) { // hurgh } CASSANDRA very verbose drivers
- 40. new_data = {:domain => 'www.datastax.com', :time => Time.now.to_i} column_family.update('a key', new_data) EURYDICE JRuby nails it
- 41. STORM stream processing framework written in Clojure
- 42. STORM (it does the arrows, you do the blobs)
- 43. REDSTORM Storm for JRuby
- 44. YOUR CODE DON’T DO THIS AT HOME KIDS! the key to building concurrent and distributed systems is to separate what you want to do from the concurrent- and distributed-ness, and let someone else take care of that
- 45. JRUBY IS A SLIPPERY SLOPE TO JBOSS let there be no doubt about it
- 46. SCALING IS HARD go to Srdan’s talk tomorrow
- 47. KTHXBAI @iconara github.com/iconara architecturalatrocities.com burtcorp.com