![Productize a narrower domain?
• Incremental GC
– We thought something like [Seligman, Grarup]
would suffice.](https://image.slidesharecdn.com/kiev-jrockit-jvm-architecture-160108204237/85/Design-rationales-in-the-JRockit-JVM-23-320.jpg)
![Productize a narrower domain?
• Incremental GC
– We thought something like [Seligman, Grarup]
would suffice.
• Support ourselves on consulting only.
– Nope – needed venture capital](https://image.slidesharecdn.com/kiev-jrockit-jvm-architecture-160108204237/85/Design-rationales-in-the-JRockit-JVM-24-320.jpg)
![Object layout and types
• Object headers should be fixed sized.
• JRockit Object header is 32 + 32 bits
• All platforms with some content variations.
• [Grove] ramblings on object models
• Type tree similar to [Krall, Vitek,
Horspool]](https://image.slidesharecdn.com/kiev-jrockit-jvm-architecture-160108204237/85/Design-rationales-in-the-JRockit-JVM-68-320.jpg)
![Memory management
• Concurrentcollection
– Your basic generational concurrent mark and sweep
collector [Printezis, Detlefs]
– Supports multi generation (>1) young spaces.
• Combats heavy object allocation situations.
• Adaptively balanced against copy overhead
– Write barriers before object writes
– Minimize stopping the world
– Young collections use a variant of stop & copy](https://image.slidesharecdn.com/kiev-jrockit-jvm-architecture-160108204237/85/Design-rationales-in-the-JRockit-JVM-72-320.jpg)
Design rationales in the JRockit JVM
- 4. Agenda • In the beginning… • What did we accomplish / Internals – Code Generation – Memory Management – Threads & Synchronization • Externals – The Java Mission Control suite – A parenthesis on JRockit VE • Q&A
- 9. About the speaker • M. Sc. from KTH, Stockholm – Narrowly escaped doing a PhD on bit security in cryptographic systems • Runtime, OS and compiler engineer since 1999, with some startup breaks • One of the original creators of the JRockit JVM
- 10. In the beginning
- 11. Appeal Virtual Machines • Appeal Software Solutions – Consulting, almost exclusively Java by 1997 • Still the pre-app server era
- 13. Appeal Virtual Machines • We saw that Java would be great on the server side – Shorter development cycles – money in the bank • Buffer overrun protection • Automatic memory management • Write once run everywhere
- 18. JavaOne 1998 • Sun Microsystems presents the HotSpot virtual machine again – “WTF! This is slide-by-slide the exact same presentation as last year!?!” – We can’t wait any longer. Let’s build our own VM. How hard can it be?
- 21. Productize a narrower domain? • Server-side usage only. Headless. – We need to help the early app server vendors get performance and scalability • No interpreter – “startup time doesn’t matter on the server anyway”
- 22. Productize a narrower domain? • Server-side usage only. Headless. – We need to help the early app server vendors get performance and scalability • No interpreter – “startup time doesn’t matter on the server anyway” • Green threads or n x m threads. – Explicit parallelism was all-pervasive.
- 24. Productize a narrower domain? • Incremental GC – We thought something like [Seligman, Grarup] would suffice. • Support ourselves on consulting only. – Nope – needed venture capital
- 25. The Java License • You can’t call yourself “Java” without a Java license • You need to pass the TCK test suite – Not available without license • To get a Java License you need a “value add”
- 30. The Java License • What’s a “value add”? – Superior performance! – What? You didn’t like that?
- 31. The Java License • What’s a “value add”? – Superior performance! – What? You didn’t like that? – OK… Let’s see… Err.. “managability”
- 32. The Java License • Java License was granted 2001 – Helped us partner up with BEA Systems and Intel – BEA acquired us in 2002 – Oracle acquired BEA in 2008 – Oracle acquired Sun in 2010
- 35. • Monitoring and Serviceability – JRockit Mission Control (now Java Mission Control) – Record and introspect production systems with zero overhead. The real value adds turned out to be:
- 36. • Pioneered “Soft realtime” GC – Deterministic GC – Low latency GC The real value adds turned out to be:
- 37. • Virtualization – JRockit Virtual Edition – an operating system for Java – Shorter paths between Java and hardware – Hypervisor required – JRockit VE on virtual hardware outperformed physical Linux! The real value adds turned out to be:
- 39. • JRockit became the default JVM in the Oracle stack in 2008 • ExaLogic … and then
- 41. Code Generation
- 42. Code generation – No Interpreter • Keep test matrix small • Keep operational complexity down • Targeting server side apps – warmup a small issue • “Code caching / AOT can be done later”
- 43. Code generation – One JIT • Keep test matrix small • Keep operation complexity down • Run it in different modes, with maximum code reuse • Same IR throughout – With gradual augmentations
- 44. But… • Startup became a problem – We removed optimizers and added as a “spine” to the normal JIT pipeline. • Lazy code generation through trampolines • Same mechanism for code invalidation • Bookkeeping to identify a program point down to any individual machine instruction
- 47. Optimizations • In and out of SSA • Applied to all levels of IR – Loop peeling, value numbering, String append explosion, Type check removal, sign extension elimination, copy propagation, bounds check removal, virtual to fixed calls, inlining, if short circuiting, straightening, strength reduction, constant propagation, dead code removal, out of loop hoisting, explode objects and array copies, boxing & unboxing removal, local escape analysis, ASM peephole optimization, redundant memory access removal, etc etc etc… • Support for regionalized IRs • Graph Fusion Register Allocator
- 48. Optimization Targets • Thread sampling • Partly taken over by safe point based approach in R28 • Some code instrumentation, for example for inlining path – Not in the general case, e.g invocation counters
- 49. Optimization Targets • Hardware sampling where available – Only good thing about IA64? – Could also match e.g. L2 misses to program points • Bugging the processor manufacturers since 2002 about userland PC sample buffer. • JRockit VE x 1000 more samples – significantly proven shorter warmup
- 51. HotSpot style? • On-stack replacement? • Deoptimization? • Never much cared for any it ;-)
- 52. HotSpot Style OSR and Deoptimization • We’ve never found a practical use case. – So we can’t ever swap out the main function with the microbenchmark loop. Who cares? • An assumption is invalidated – Either patch code directly or use a guard when generating it in the first place • A large assumption – Write a trap in the code and schedule lazy regeneration of entire method • Not strictly true for dynamic languages
- 53. HotSpot Style OSR and Deoptimization • We’ve never found a practical use case. – So we can’t ever swap out the main function with the microbenchmark loop. Who cares? • An assumption is invalidated – Either patch code directly or use a guard when generating it in the first place • A large assumption – Write a trap in the code and schedule lazy regeneration of entire method • Not strictly true for dynamic languages
- 54. HotSpot Style OSR and Deoptimization • We’ve never found a practical use case. – So we can’t ever swap out the main function with the microbenchmark loop. Who cares? • An assumption is invalidated – Either patch code directly or use a guard when generating it in the first place • A large assumption – Write a trap in the code and do regeneration of entire method • Not strictly true for dynamic languages
- 55. HotSpot Style OSR and Deoptimization • We’ve never found a practical use case. – So we can’t ever swap out the main function with the microbenchmark loop. Who cares? • An assumption is invalidated – Either patch code directly or use a guard when generating it in the first place • A large assumption – Write a trap in the code and do regeneration of entire method • Not strictly true for dynamic languages
- 56. “Garbage collecting code” • Code kept in binary tree of code blocks ~ 64M – More if large pages enabled • Class loader unloading à garbage collection • Reference count to active code modified when backpatching • Specialized usage of code blocks. – Trampolines only – Optimized code only
- 58. Bytecode is bad – kill it quickly • What’s with the goto:s? • Why can it express more than Java source code? – OK we understand the multi language concept, we sorta forgive you. – But man, dominators and loop analysis – that’s a lot of compile time
- 59. Bytecode is bad – kill it quickly • …and why is it a stack machine AND a register machine with 65535 registers at the same time!? • Initially tried to reconstruct ASTs – Obfuscators etc made this pretty hopeless. • ~15% of the klocs in JRockit/codegen do flow control analysis on the goto:s
- 60. The IR • Use IR everywhere (or Java) • The IR should ideally reflect any of several pluggable frontends. – We played around with CLR a bit. – These days – dynamic languages :-) • No Sea of Nodes • No HotSpot style “high level IR is low level”
- 62. The IR – Design Rationale • We had some compiler experience – wanted to be on track quickly. Do it the traditional way. • We are not “wrong”. LLVM is very similar.
- 63. The IR – Design Rationale • Tiered: highest tier == always high level • Hardware agnostic. • No architecture specific memory ops • Tiered: lowest tier == always the native architecture instruction for instruction. • A gradual transition. • A CPU has no sea of nodes.
- 64. The IR • Highest IR level may have operations as operands • Intrinsics everywhere – arraycopy, membar, cmpuXX, sse4IndexOf, doubleToLongBits, crypto, Math.sin and so on … • Regret not doing more in SSA form
- 65. The IRInfo “database” • Lazily computable information – Liveness – Dominators – Loop information – Aliases – Type inference – Ranges – Nullness analysis – … – Invalidate on modification. • Not a very stable model.
- 68. Object layout and types • Object headers should be fixed sized. • JRockit Object header is 32 + 32 bits • All platforms with some content variations. • [Grove] ramblings on object models • Type tree similar to [Krall, Vitek, Horspool]
- 69. Livemaps (oopmaps) • Registers and stack slots on the local frame that contain objects. • Nothing strange here. Required for non- conservative garbage collection of any sort. • Internal pointer bit • Forms the root set. • Rollforwarding vs the safepoint approach
- 71. Memory management • Garbage collectors – Concurrent – Parallel – Deterministic • With or without generations
- 72. Memory management • Concurrentcollection – Your basic generational concurrent mark and sweep collector [Printezis, Detlefs] – Supports multi generation (>1) young spaces. • Combats heavy object allocation situations. • Adaptively balanced against copy overhead – Write barriers before object writes – Minimize stopping the world – Young collections use a variant of stop & copy
- 73. Memory management • Can also run with a parallel policy – Stop the world and clean up quickly – Only throughput oriented – No write barriers, as there is no need for a card table
- 75. Mark & Sweep • Two colors – not three. – Object is in one of two sets – Live objects: grey bits (mix of grey & black objects in traditional tri-coloring) – Distinction handled by putting grey objects in thread local queues for each GC thread. – Parallel threads can work on thread local data – Efficient prefetching is possible due to FIFO order.
- 76. No permgen ever!
- 78. Other nice features • No permgen!!! Ever! • Pinned objects. – Fast memory buffers – Also enable non-contiguous heaps
- 79. Other nice features • No permgen!!! Ever! • Pinned objects. – Fast memory buffers. – Also enable non-contiguous heaps. • Compaction – “Internal and external”. – G1 evacuates regions instead with a stop the world-and-copy policy similar to JRockit YC
- 80. Memory management • Concurrent GC has an additional set: live bits – Contains all live objects in the system, including the newly created ones. – JRockit can quickly find objects that have been created during a concurrent mark phase. – Card tables • Not just for generational GC • Also to avoid searching the entire live object graph when a concurrent mark phase cleans up. • Just look at dirty cards at the end of the mark phase.
- 81. Young Collections • A variant of stop and copy is used. – All threads are halted and objects are deleted or promoted – Hierarchical breadth first copy for cache locality • Parallelizes nicely • Many threads always harvest a young space
- 82. Young Collections • Young and old collections may occur at same time. – All bit sets and data structures can be shared as long as the old collection is guaranteed to see all cards that have become dirty during a concurrent phase. (Extra card table to record this “difference” – “modified union set”) – Keep this intact for old collection
- 84. Compressed References • For less than 4 (or 4 * x) GB of maximum heap size • Use 32 bit pointers (or 32 + log2(x) bits) CompRef compress(Ref ref) { return (uint32_t)ref; //truncate reference to 32-bits } Ref decompress(CompRef ref) { return globalHeapBase | ref; }
- 85. Compressed References CompRef compress(Ref ref) { return (uint32_t)ref; //truncate reference to 32-bits } Ref decompress(CompRef ref) { return globalHeapBase | ref; } CompRef compress(Ref ref) { return (uint32_t)(ref >> log 2 (objectAlignment)); } Ref decompress(CompRef ref) { return globalHeapBase | (ref << log 2 (objectAlignment)); }
- 86. Deterministic GC • QoS level for latencies. “No more than X ms” • Down to single digits on modern x86 hardware • Caveat: live data on heap is the main constraint. – Up to 50% of heap live data still feasible
- 87. Deterministic GC
- 88. Deterministic GC – How? • Greedy strategy – Postpone stopping the world for as long as possible. – Maybe the problem goes away and we don’t have to stop the world • Split up everything into work packets – Drop them at any time.
- 89. Deterministic GC – How? • Efficient parallelization. – Mark phase is 90% of GC time • Efficient heuristics – Some more work in e.g. write barriers
- 91. Threads and Synchronization • A java.lang.Thread is a native thread. – Interesting, though: thread pooling and pseudo thin-threads are back, for example in Akka. – Java 8 – Collection.parallelStream – The world is moving towards implicit parallelism in general • Most of the JRockit thread code and adaptivity logic is written in Java
- 92. Threads and Synchronization • Locks are thin or fat – Adaptive inflation and deflation • Lazy locking (biased locking supported) – Adaptive heuristics for banning and retrying the lazy approach.
- 93. Threads and Synchronization public class PseudoSpinlock { private static final int LOCK_FREE = 0; private static final int LOCK_TAKEN = 1; public void lock() { //burn cycles while (cmpxchg(LOCK_TAKEN, &lock) == LOCK_TAKEN) { micropause(); //optional } } public void unlock() { int old = cmpxchg(LOCK_FREE, &lock); //guard against recursive locks assert(old == LOCK_TAKEN); } }
- 94. Threads and Synchronization • Locks are thin when first taken • Time spent in lock and times taken triggers inflation • wait or notify immediately inflates a lock • Fat locks are also deflated when uncontended for too long
- 98. Lock Pairing • Bytecode again – no restriction on matching monitorenter with monitorexit • Not all of them can be analyzed by the JIT
- 99. Lock Pairing • We can store what we know, and make unlocks quick. – Lock tokens (the object OR 3 bits) • Thin, fat, recursive, lazily taken, unmatched – Livemap system contains nesting order.
- 100. Optimizations • A lot of smallish code gen transforms: e.g. Lock fusion • “Fat spin” • Lazy unlocking (biased locking) – Start assuming all locks are lazy. Tag thin locks as lazily locked. – If object already lazily locked • If it’s the same thread: profit • Else – stop the lock holder, detect the “real” lock state by stack walk. Convert to thin lock or forcefully unlock it – Transfer bits – Heuristics: object and class banning. Ageing.
- 103. Mission Control • Use “free” runtime information! – JRockit (Java) Mission Control • JRockit (Java) flight recorder • Memory leak detector (JRockit only) • Management console • $JAVA_HOME/bin/JCMD (used to be JRCMD) • Everything in the VM abstracted into an event that may or may not have a duration • Soon: public API
- 104. Java Flight Recorder • Always on – Excellent for debugging and analysis of crashes – Can be set to record more intrusively for periods in production • E.g. extensive lock profiling • Everything is an event • Buffered recording – the last n seconds available at any crash or when a command is given. • Very fine precision. – Multimedia timers and system hardware support required for e.g. latencies
- 105. Latency Analysis
- 106. The Management Console • Peek into the running production JVM • Add triggers on events • Interact with the VM: force GC etc.
- 110. Is the JVM an OS?
- 111. Is the JVM an OS? • Add a cooperative aspect to thread switching • Zero-copy networking code • Reduce cost of entering OS • Balloon driver • Runs only on hypervisor • Facilitates pauseless GC
- 112. Is the JVM an OS?