Loom me up Scotty! Project Loom - What's in it for Me?
0 likes56 views
The document discusses Project Loom, focusing on its implications for concurrency in Java, introducing concepts like virtual threads and structured concurrency to enhance developer productivity. It outlines examples in Kotlin and Java, detailing how to implement these features and manage concurrent tasks effectively. The document concludes by suggesting that while Project Loom is not yet fully production-ready, it could be beneficial for new, IO-bound services requiring high concurrency.
![Hello World
Thread.ofVirtual()
.name("virtual-", 1)
.start {
Thread.sleep(2000)
println("Hello World:"+ Thread.currentThread())
}.join()
Output :
Hello World: VirtualThread[#23,virtual-1]/runnable@ForkJoinPool-1-worker-1](https://image.slidesharecdn.com/loommeupscotty-221121082804-599720bd/85/Loom-me-up-Scotty-Project-Loom-What-s-in-it-for-Me-60-320.jpg)
![Thread dumps
jcmd <pid> Thread.dump_to_file -format=json <File name>
Thread.ofVirtual
executor.submit
scope.fork
Platform Threads
{
"tid": "5604",
"name": "",
"stack": [
"java.base/jdk.internal.vm.Continuation.yield(Continuation.java:357)",
"java.base/java.lang.VirtualThread.yieldContinuation(VirtualThread.java:370)",
"java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:532)",
"java.base/java.lang.VirtualThread.doSleepNanos(VirtualThread.java:713)",
"java.base/java.lang.VirtualThread.sleepNanos(VirtualThread.java:686)",
"java.base/java.lang.Thread.sleep(Thread.java:451)",
"playground.ExecutorTest$vthreadExecutorTest$1$1$1.invoke(ExecutorTest.kt:30)",
"playground.ExecutorTest$vthreadExecutorTest$1$1$1.invoke(ExecutorTest.kt:27)",
"playground.ExecutorTest.vthreadExecutorTest$lambda-1$lambda-0(ExecutorTest.kt:27)",
"java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)",
"java.base/java.util.concurrent.ThreadPerTaskExecutor$ThreadBoundFuture.run(ThreadPerTaskExecutor.ja
va:352)",
"java.base/java.lang.VirtualThread.run(VirtualThread.java:287)",
"java.base/java.lang.VirtualThread$VThreadContinuation.lambda$new$0(VirtualThread.java:174)",
"java.base/jdk.internal.vm.Continuation.enter0(Continuation.java:327)",
"java.base/jdk.internal.vm.Continuation.enter(Continuation.java:320)"
]
},
❌
✅
✅
✅](https://image.slidesharecdn.com/loommeupscotty-221121082804-599720bd/85/Loom-me-up-Scotty-Project-Loom-What-s-in-it-for-Me-88-320.jpg)
Loom me up Scotty! Project Loom - What's in it for Me?
- 2. Loom Haim Yadid @ Next Insurance
- 3. Loom me up Scotty! Haim Yadid @ Next Insurance
- 4. Loom me up Scotty! Project Loom - What's in it for Me? Haim Yadid @ Next Insurance
- 5. Loom me up Scotty! Project Loom - What's in it for Me? Haim Yadid @ Next Insurance Dall E - Star Trek spaceship Enterprise flying inside a loom, Van gogh style
- 10. @lifeyx
- 11. Pro t L o es . 2017 JE -425 JE -428 JE -429
- 13. CONCURRENCY
- 14. request (task)
- 15. getBid(user)
- 17. Running Blocked
- 18. getBid(user) resolve user country Query bidding service 1 for user country Query bidding service 2 for user country
- 19. Example - Single Threaded val country = userClient. findUserByEmail(userEmail).country t1 https://github.com/lifey/loom-playground
- 20. Example - Single Threaded val country = userClient. findUserByEmail(userEmail).country val resultA = serverA.getBid(country) val resultB = serverB.getBid(country) t1
- 21. Example - Single Threaded val country = userClient. findUserByEmail(userEmail).country val resultA = serverA.getBid(country) val resultB = serverB.getBid(country) val lowestBid = if (resultA.price < resultB.price) resultA else resultB t1
- 22. getBid resolve user country Query bidding service 1 for user country Query bidding service 2 for user country fan out is 3
- 25. threads Immutability Deadlocks synchronization Thread pools race conditions contention shared mutable state message passing channels Cache Coherence callbacks semaphores
- 27. Java and Concurrency / Parallelism Java 1.0-1.1 (1995) Threads wait /notify synchronized
- 28. Java and Concurrency / Parallelism Java 1.0-1.1 (1995) Threads wait /notify synchronized Java 1.4 (2002) Non blocking IO SocketChannel Selector
- 29. Java and Concurrency / Parallelism Java 1.0-1.1 (1995) Threads wait /notify synchronized Java 1.4 (2002) Non blocking IO SocketChannel Selector Java 1.5 (2004) j.u.c Executors Future Reentrant Locks Semaphores Latches
- 30. Java and Concurrency / Parallelism Java 1.0-1.1 (1995) Threads wait /notify synchronized Java 1.4 (2002) Non blocking IO SocketChannel Selector Java 1.5 (2004) j.u.c Executors Future Reentrant Locks Semaphores Latches Java 1.7 (2011) ForkJoin Pool Java 1.8 (2014) ParallelStreams CompletableFuture
- 31. Java and Concurrency / Parallelism Java 1.0-1.1 (1995) Threads wait /notify synchronized Java 1.4 (2002) Non blocking IO SocketChannel Selector Java 1.5 (2004) j.u.c Executors Future Reentrant Locks Semaphores Latches Java 1.7 (2011) ForkJoin Pool Java 1.8 (2014) ParallelStreams CompletableFuture Java 19 Loom (2022) VirtualThreads Structured Concurrency
- 32. traditional Java Concurrency Model ● Thread per request (task) style ● Java thread is a wrapper of OS thread ● 1 or more thread to serve the request using Futures ● Executors and thread pools to control concurrency level
- 33. getBid resolve user country Query bidding service 1 for user country Query bidding service 2 for user country fan out is 3
- 34. Example (Kotlin) val userCountry = userClient.findUserByEmail(userEmail).country t1
- 35. Example (Kotlin) val userCountry = userClient.findUserByEmail(userEmail).country executor.invokeAny (listOf( )) t1
- 36. Example (Kotlin) val userCountry = userClient.findUserByEmail(userEmail).country executor.invokeAny (listOf( Callable { clientA.getBid(userCountry) } , Callable { clientB.getBid(userCountry) } )) t1 t2 t3
- 37. Example (Kotlin) val userCountry = userClient.findUserByEmail(userEmail).country val price = executor.invokeAny (listOf( Callable { clientA.getBid(userCountry) } , Callable { clientB.getBid(userCountry) } )) .price t1 t2 t3
- 39. So How much concurrency ? Little’s Law ● λ - throughput (arrival rate) ● L - average concurrency ● W- average latency /duration L= λ * W
- 40. E= M * C2
- 41. getBid(user) resolve user country Query bidding service 1 for user country Query bidding service 2 for user country
- 42. 50% 50% 32 CPU cores 64 (32*2) concurrent tasks
- 43. 5 % 99% 32 CPU cores 3200 (32*100) concurrent tasks
- 44. 5 % 99.9 % 32 CPU cores 32000 (32*1000) concurrent tasks
- 45. getBid resolve user country Query bidding service 1 for user country Query bidding service 2 for user country fan out is 3
- 46. 5 % 99.9 % 32 CPU cores Fan out of 5 150K (32*1000*5) concurrent tasks
- 47. C1M Problem What is needed for a server to hold 1M concurrent connections
- 48. So what is the Problem With OS Threads ● stack: ~1 MB ● scheduling >1µs ● launch a new thread ~0.1ms ● context switch involves kernel meaning returning from future ~5µs ● Usually Linux OS can support thousands of threads
- 49. Asynchronous Programming Style, Callback / Non Blocking / Async Model Event Loop
- 50. Event Loop Asynchronous Programming Style, Callback / Non Blocking / Async Model # of cores
- 51. Async ● Less readable (for most developers ) ● Callback hell anyone ? a diagram from Ron Pressler’s presentation CompletableFuture.supplyAsync(info::getUrl, pool) .thenCompose(url -> getBodyAsync(url, HttpResponse.BodyHandlers.ofString())) .thenApply(info::findImage) .thenCompose(url -> getBodyAsync(url, HttpResponse.BodyHandlers.ofByteArray())) .thenApply(info::setImageData) .thenAccept(this::process) .exceptionally(t -> { t.printStackTrace(); return null; });
- 52. Async is Tooling - Nightmare! ● Where is my call stack ? ● debug ● profile ● troubleshoot ● understanding number of actively running task
- 53. ●
- 54. a slide from Ron Pressler’s presentation
- 55. Project Loom is about Developer Productivity (& Sanity)
- 57. Virtual Thread is: A unit of execution (a task) which can run on on carrier Platform(OS) thread
- 59. Hello World Thread.ofVirtual() .name("virtual-", 1) .start { }.join() Runnable
- 60. Hello World Thread.ofVirtual() .name("virtual-", 1) .start { Thread.sleep(2000) println("Hello World:"+ Thread.currentThread()) }.join() Output : Hello World: VirtualThread[#23,virtual-1]/runnable@ForkJoinPool-1-worker-1
- 61. Example: Executor val executor = Executors. newVirtualThreadPerTaskExecutor()
- 62. Example: Executor val executor = Executors. newVirtualThreadPerTaskExecutor() executor.use { }
- 63. Example: Executor val executor = Executors. newVirtualThreadPerTaskExecutor() executor.use { val future = executor.submit { } } Callable
- 64. Example: Executor val executor = Executors. newVirtualThreadPerTaskExecutor() executor.use { val future = executor.submit { Thread.sleep(2000) println("Hello World: " + Thread.currentThread()) } }
- 65. Virtual thread
- 67. Common Pool Scheduler Virtual thread Platform Threads
- 70. Yield -> Unmounting Scheduler Virtual thread Platform Threads
- 71. Virtual Threads stack frames ● Mounted -> On the carrier thread stack ● UnMounted -> in the Heap ○ pay as you go stack can be very small
- 72. Loom Is JVM Copying Stack frames Changes in GC root scanning JRE Continuation Changes in all IO APIs
- 73. Thread Locals Context (e.g.MDC) TransactionID Shared Heavy Data Structures ❌ ✅ Non thread safe Objects (XML Parser) ❌ Inheritable
- 74. You can prevent this from happening by setting inheritInheritableThreadLocals(false)
- 76. Structured Concurrency ● Makes writing correct concurrent code easier ● Cancellation, error propagation, timeouts ● Observe hierarchy of virtual threads at runtime
- 77. Example: Structured Concurrency val country = userRClient.findUserByEmail(userEmail).country
- 78. Example: Structured Concurrency val country = userRClient.findUserByEmail(userEmail).country val scope = StructuredTaskScope. ShutdownOnSuccess<BiddingResponse>() scope.use { }
- 79. Example: Structured Concurrency val country = userRClient.findUserByEmail(userEmail).country val scope = StructuredTaskScope. ShutdownOnSuccess<BiddingResponse>() scope.use { scope.fork { serverA.getBid(country) } scope.fork { serverB.getBid(country) } }
- 80. Example: Structured Concurrency val country = userRClient.findUserByEmail(userEmail).country val scope = StructuredTaskScope. ShutdownOnSuccess<BiddingResponse>() scope.use { scope.fork { serverA.getBid(country) } scope.fork { serverB.getBid(country) } val firstSuccessfulFuture = scope.join() return firstSuccessfulFuture.result().price }
- 81. Example: Structured Concurrency val country = userRClient.findUserByEmail(userEmail).country val scope = StructuredTaskScope. ShutdownOnSuccess<BiddingResponse>() scope.use { scope.fork { serverA.getBid(country) } scope.fork { serverB.getBid(country) } val firstSuccessfulFuture = scope.joinUntil( Instant.now().plusMillis(3000)) return firstSuccessfulFuture.result().price }
- 83. Tooling
- 87. Thread dumps jstack -l <pid> Shows only platform threads
- 88. Thread dumps jcmd <pid> Thread.dump_to_file -format=json <File name> Thread.ofVirtual executor.submit scope.fork Platform Threads { "tid": "5604", "name": "", "stack": [ "java.base/jdk.internal.vm.Continuation.yield(Continuation.java:357)", "java.base/java.lang.VirtualThread.yieldContinuation(VirtualThread.java:370)", "java.base/java.lang.VirtualThread.parkNanos(VirtualThread.java:532)", "java.base/java.lang.VirtualThread.doSleepNanos(VirtualThread.java:713)", "java.base/java.lang.VirtualThread.sleepNanos(VirtualThread.java:686)", "java.base/java.lang.Thread.sleep(Thread.java:451)", "playground.ExecutorTest$vthreadExecutorTest$1$1$1.invoke(ExecutorTest.kt:30)", "playground.ExecutorTest$vthreadExecutorTest$1$1$1.invoke(ExecutorTest.kt:27)", "playground.ExecutorTest.vthreadExecutorTest$lambda-1$lambda-0(ExecutorTest.kt:27)", "java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:577)", "java.base/java.util.concurrent.ThreadPerTaskExecutor$ThreadBoundFuture.run(ThreadPerTaskExecutor.ja va:352)", "java.base/java.lang.VirtualThread.run(VirtualThread.java:287)", "java.base/java.lang.VirtualThread$VThreadContinuation.lambda$new$0(VirtualThread.java:174)", "java.base/jdk.internal.vm.Continuation.enter0(Continuation.java:327)", "java.base/jdk.internal.vm.Continuation.enter(Continuation.java:320)" ] }, ❌ ✅ ✅ ✅
- 89. Java Flight Recorder (JFR) ● jdk.VirtualThreadStart ● jdk.VirtualThreadEnd ● jdk.VirtualThreadPinned ● jdk.VirtualThreadSubmitFailed
- 90. Summary
- 91. Should I use Project Loom in Production?
- 92. Should I use Project Loom in Production? Of course not
- 93. Not Yet
- 94. But Maybe ● If you are starting to write a new service that is IO bound and requires high level of concurrency (> 10K concurrent connections) ● If you want to play around ○ https://github.com/lifey/loom-playground ○ https://github.com/cmpxchg16/kotlin-loom ● If you want to provide feedback
- 95. supports Virtual Threads Since: v10.0.12 Enable by: QueuedThreadPool.setUseVirtualThread(true)
- 98. Helidon Nima With “dumb” synchronous code on Loom and some simple tuning, quickly got similar or better performance to Netty — Tomas Langer (Helidon)
- 101. Pinned Threads ● When a virtual thread is in a synchronized block (or in JNI call) ● It cannot be unmounted ● Even when blocked on IO -Djdk.tracePinnedThreads=full
- 102. Don’t Po Vir Th e s