JVM Mechanics
3 likes614 views
The document discusses topics related to just-in-time (JIT) compilation in the Java Virtual Machine (JVM). It explains that the JIT compiler is triggered when methods are invoked many times based on invocation and back-edge counters, compiling frequently used code to machine code for improved performance. It also discusses why ahead-of-time compilation is not well-suited for Java due to its dynamic nature, and what can cause compiled code to be deoptimized, such as class initialization failures, null pointer exceptions, and unstable control flow.
![01
public class SimpleProgram {
static int blackhole;
public static void main(String[] args) {
int[] nums = randomInts(5_000);
for ( int i = 0; i < 100; ++i ) {
long startTime = System.nanoTime();
blackhole = sum(nums);
long endTime = System.nanoTime();
System.out.printf("%dt%d%n", i, endTime - startTime);
}
}
…
}](https://image.slidesharecdn.com/jvmmechanics-170928232300/85/JVM-Mechanics-5-320.jpg)
![UNINITIALIZED FOREVER
public class UninitializedForever {
public static void main(String[] args) {
for ( int i = 0; i < 10_000_000; ++i ) {
try {
new Uninitialized();
} catch ( Throwable t ) {
// ignore
}
}
}
}
static class Uninitialized {
static {
if ( true ) {
throw new RuntimeException();
}
}
}
02](https://image.slidesharecdn.com/jvmmechanics-170928232300/85/JVM-Mechanics-32-320.jpg)
![03
public static void main(String[] args) {
// warm-up hotMethod — enough to JIT
for ( int i = 0; i < 20_000; ++i ) {
hotMethod("hello");
}
for ( int i = 0; i < 10; ++i ) {
System.out.printf("tempting fate %d%n", i);
try {
hotMethod(null);
} catch ( NullPointerException e ) { }
}
}
static final void hotMethod(
Object value)
{
value.hashCode();
}
NULL CHECKING](https://image.slidesharecdn.com/jvmmechanics-170928232300/85/JVM-Mechanics-39-320.jpg)
![UNREACHED / UNSTABLE IF
public static volatile Object thing = null;
public static void main(final String[] args) {
for ( int i = 0; i < 20_000; ++i ) {
hotMethod();
}
Thread.sleep(5_000); // wait for JIT
thing = new Object();
for ( int i = 0; i < 20_000; ++i ) {
hotMethod();
}
Thread.sleep(5_000); // wait for JIT again
}
static final void hotMethod() {
if ( thing == null )
System.out.print("");
else
System.out.print("");
}
04](https://image.slidesharecdn.com/jvmmechanics-170928232300/85/JVM-Mechanics-43-320.jpg)
JVM Mechanics
- 1. JVM MECHANICSDOUGLAS Q. HAWKINS LEAD JIT DEVELOPER AZUL SYSTEMS @dougqh dougqh@gmail.com
- 2. (SIMPLIFIED) CODE LIFECYCLE Interpreter 1 34 2 Profile Deoptimize Just-in-Time Compilation
- 3. TOPICS WHAT TRIGGERS THE JUST-IN-TIME COMPILER? WHY NOT AHEAD-OF-TIME? WHAT TRIGGERS THE UN-JIT? IMPACT ON YOU
- 5. 01 public class SimpleProgram { static int blackhole; public static void main(String[] args) { int[] nums = randomInts(5_000); for ( int i = 0; i < 100; ++i ) { long startTime = System.nanoTime(); blackhole = sum(nums); long endTime = System.nanoTime(); System.out.printf("%dt%d%n", i, endTime - startTime); } } … }
- 6. … 0 76484 1 75764 2 72254 3 72189 4 72364 5 72014 6 72177 7 72040 8 71721 9 76635 10 72120 11 72241 12 89045 13 30692 14 14899 … 45 12567 46 46570 47 25096 48 56740 49 14620 50 55286 51 14844 52 60827 53 12384 54 66209 55 9638 56 37767 57 9350 58 2178 59 2255 85 1709 86 1858 87 1762 88 1767 89 1760 90 1735 91 1710 92 1706 93 1710 94 1726 95 1724 96 1729 97 1701 98 1733 99 1725 01
- 7. Interpreter 1st JIT 2nd JIT logns 1 100 10000 0 10 20 30 40 50 60 70 80 90 100 01
- 8. PRINT COMPILATIONjava -XX:+PrintCompilation 61 1 3 java.lang.String::hashCode (55 bytes) 64 2 3 java.lang.String::charAt (29 bytes) 64 4 3 java.lang.String::indexOf (70 bytes) 64 3 3 java.lang.String::length (6 bytes) 65 5 3 java.lang.Object::<init> (1 bytes) … 68 12 3 java.lang.String::getChars (62 bytes) 69 13 1 java.lang.ref.Reference::get (5 bytes) 78 14 3 java.lang.String::indexOf (7 bytes) 78 15 1 java.lang.Object::<init> (1 bytes) 81 16 % 3 example01.SimpleProgram::main @ 15 (48 bytes) 81 17 3 example01.SimpleProgram::main (48 bytes) 81 18 % 4 example01.SimpleProgram::main @ 15 (48 bytes) 01
- 10. LOG COMPILATION XML java -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation <task compile_id='2' level='3' stamp=‘0.126’ method='java/lang/String charAt (I)C' bytes='29' count='1955' iicount='1955' > … <task_done success='1' stamp=‘0.127' nmsize='616' count='2269'/> </task> <task_queued compile_id=‘5’ level=‘3' stamp='0.130' method='java/lang/Object <init> ()V' bytes='1' count='1536' iicount='1536' comment='tiered' hot_count='1536'/>
- 11. TIERED COMPILATION C1: Client C2: Server Compilation Speed Fast Slow (4X) Execution Speed Slow Fast (2X)
- 12. TIERED COMPILATION 0: Interpreter 1-3: 1st JIT 4: 2nd JIT 1: No overhead 2: Counters 3: Counters + Profiling 0 0 0 3 3 31 2 4 4 normal flow tier 4 busy trivial method
- 14. LOOP COMPILATIONS ON STACK REPLACEMENTS (OSR) sum ... ... main sum @ 20 ... ... main interpreter frame compiled frame
- 15. OSR IS IMPORTANT TO OUR EXAMPLE 96 54 % 3 …SimpleProgram::main @ 15 (78 bytes) 12 38362 13 35214 14 37139 15 36598 15 36838 16 36429 17 18046 101 80 % 4 …SimpleProgram::main @ 15 (78 bytes) 62 2831 63 2371 64 2288 01
- 16. BUT WHEN? BACKEDGE (LOOP) COUNTER Counter > Backedge Threshold Hot Loops Counter > Invocation Threshold Hot Methods Invocation + Backedge Counters > Compile Threshold Medium Hot Methods with Medium Hot Loops INVOCATION COUNTER
- 17. BUT WHEN? java -XX:+PrintFlagsFinal JAVA 7 (NON-TIERED) THRESHOLDSJAVA 8 (TIERED) THRESHOLDS intx Tier2BackEdgeThreshold = 0 intx Tier2CompileThreshold = 0 intx Tier3BackEdgeThreshold = 60000 intx Tier3CompileThreshold = 2000 intx Tier3InvocationThreshold = 200 intx Tier3MinInvocationThreshold = 100 intx Tier4BackEdgeThreshold = 40000 intx Tier4CompileThreshold = 15000 intx Tier4InvocationThreshold = 5000 intx Tier4MinInvocationThreshold = 600 intx BackEdgeThreshold = 100000 intx CompileThreshold = 10000
- 20. 0 1000 2000 3000 4000 0s 10s 20s 30s 40s 50s 60s 70s 80s 90s 100s 1 2 3 4 ACTIVE COMPILATIONS
- 22. WARM-UP & RUN ONCE CODE
- 23. JAVA IS A DYNAMIC LANGUAGE DYNAMICALLY LOADED LAZY INITIALIZED RUNTIME CHECKED DYNAMICALLY DISPATCHED FROM MY PERSPECTIVE
- 24. IT CANNOT GET MUCH MORE DYNAMIC.
- 25. DYNAMICALLY & LAZY LOADED WHAT DOES JIT DO WITH AN UNLOADED CLASS? if ( cond ) { return Class1.getStatic(); } else { return Class2.getStatic(); }
- 26. UNLOADED JIT DOESN’T KNOW… Fields Methods Parent Class Interfaces Anything … if ( cond ) { return Class1.getStatic(); } else { return Class2.getStatic(); } if ( cond ) { return Class1.getStatic(); } else { uncommon_trap(:unloaded); } Give Up!
- 27. BAIL TO INTERPRETER Interpreter C1 C2 uncommon trap + bail to interpreter
- 28. LAZY INITIALIZED INITIALIZE CLASS AT FIRST Static Field Access Static Method Call New Initialization of Child ...
- 29. MyClass.getStatic() if ( !vm.is_init(MyClass)) { vm.init(MyClass); } MyClass.getStatic() Bloats size by 20% Slows down by 5-10% INITIALIZATION CHECKING http://joeduffyblog.com/2015/12/19/safe-native-code/
- 30. JIT RUNS LATE MOST CLASSES ARE ALREADY INITIALIZED MyClass.getStatic() MyClass.getStatic() vm.is_init(MyClass) initialized uncommon_trap(:uninitialized) uninitialized At compile time…
- 31. WHAT IF A CLASS FAILS TO INIT?
- 32. UNINITIALIZED FOREVER public class UninitializedForever { public static void main(String[] args) { for ( int i = 0; i < 10_000_000; ++i ) { try { new Uninitialized(); } catch ( Throwable t ) { // ignore } } } } static class Uninitialized { static { if ( true ) { throw new RuntimeException(); } } } 02
- 33. 612 24 % ! 3 …UninitializedForever::main @ 5 (25 bytes) 621 25 ! 3 …UninitializedForever::main (25 bytes) 952 26 % ! 4 …UninitializedForever::main @ 5 (25 bytes) 953 24 % ! 3 …UninitializedForever::main @ -2 (25 bytes) made not entrant 1024 26 % ! 4 …UninitializedForever::main @ -2 (25 bytes) made not entrant 1033 27 % ! 3 …UninitializedForever::main @ 5 (25 bytes) 1405 28 % ! 4 …UninitializedForever::main @ 5 (25 bytes) 1406 27 % ! 3 …UninitializedForever::main @ -2 (25 bytes) made not entrant 1481 28 % ! 4 …UninitializedForever::main @ -2 (25 bytes) made not entrant 1489 29 % ! 3 …UninitializedForever::main @ 5 (25 bytes) 1855 30 % ! 4 …UninitializedForever::main @ 5 (25 bytes) … 7339 55 % ! 3 …UninitializedForever::main @ 5 (25 bytes) 7690 56 % ! 4 …UninitializedForever::main @ 5 (25 bytes) 7690 55 % ! 3 …UninitializedForever::main @ -2 (25 bytes) made not entrant 7761 56 % ! 4 …UninitializedForever::main @ -2 (25 bytes) made not entrant 7769 57 % ! 3 …UninitializedForever::main @ 5 (25 bytes) THAT’S UNFORTUNATE 02
- 34. <task compile_id='26' stamp='1.403' compile_kind='osr' osr_bci='5' method=‘…UninitializedForever main ([Ljava/lang/String;)V' decompiles='1' uninitialized_traps='1' …> ... <bc code='187' bci='5'/> <klass id='835' name=‘…Uninitialized’ flags='8'/> <uncommon_trap bci='5' reason='uninitialized' action='reinterpret' klass='835'/> ... </task> LOG COMPILATION XML java -XX:+UnlockDiagnosticVMOptions -XX:+LogCompilation <uncommon_trap thread='7171' stamp='1.038' compile_id='24' compile_kind='osr' compiler='C2' level='4' reason='uninitialized' action='reinterpret'> ... </uncommon_trap> <make_not_entrant thread='7171' stamp='1.038' compile_id='24' compile_kind='osr' compiler='C2' level='4' /> TRAP INSTALLATION TRIGGERING TRAP
- 35. COMPILING LATER PROVIDES MORE INFO TO SPECULATE.
- 36. C2 C1Interpreter 5x 10-100+x REFINED MENTAL MODEL non-sampling period sampling period full-speed period
- 37. TIERED COMPILATION 0: Interpreter 1-3: 1st JIT 4: 2nd JIT 1: No overhead 2: Counters 3: Counters + Profiling 0 0 0 3 3 31 2 4 4
- 39. 03 public static void main(String[] args) { // warm-up hotMethod — enough to JIT for ( int i = 0; i < 20_000; ++i ) { hotMethod("hello"); } for ( int i = 0; i < 10; ++i ) { System.out.printf("tempting fate %d%n", i); try { hotMethod(null); } catch ( NullPointerException e ) { } } } static final void hotMethod( Object value) { value.hashCode(); } NULL CHECKING
- 40. 81 1 java.lang.String::hashCode (55 bytes) 84 2 …NullCheck::hotMethod (6 bytes) 85 3 % ! …NullCheck::main @ 5 (69 bytes) tempting fate 0 tempting fate 1 tempting fate 2 5089 2 …NullCheck::hotMethod (6 bytes) made not entrant tempting fate 3 tempting fate 4 tempting fate 5 tempting fate 6 tempting fate 7 tempting fate 8 tempting fate 9 03
- 41. IMPLICIT NULL CHECK if ( value == null ) { throw new NullPointerException(); } value.hashCode(); Possible, but improbable SEGV 0x10795f9cc: mov 0x8(%rsi),%r10d ; implicit exception: dispatches to 0x10795fe1d Deref Signal Handler throw NPE
- 42. SLOW WHEN IT HAPPENS, FAST WHEN IT DOESN’T.
- 43. UNREACHED / UNSTABLE IF public static volatile Object thing = null; public static void main(final String[] args) { for ( int i = 0; i < 20_000; ++i ) { hotMethod(); } Thread.sleep(5_000); // wait for JIT thing = new Object(); for ( int i = 0; i < 20_000; ++i ) { hotMethod(); } Thread.sleep(5_000); // wait for JIT again } static final void hotMethod() { if ( thing == null ) System.out.print(""); else System.out.print(""); } 04
- 44. 79 18 3 …Unreached::hotMethod (26 bytes) 83 33 4 …Unreached::hotMethod (26 bytes) 5089 33 4 …Unreached::hotMethod (26 bytes) made not entrant 5089 36 3 …Unreached::hotMethod (26 bytes) 5090 38 4 …Unreached::hotMethod (26 bytes) 04
- 45. static final void hotMethod() { if ( thing == null ) System.out.print(""); else System.out.print(""); } static final void hotMethod() { if ( thing == null ) System.out.print(""); else uncommon_trap(:unreached); } <uncommon_trap thread='7171' stamp=‘5.104’ compile_id='29' compiler='C2' level=‘4' reason='unstable_if' action='reinterpret' > <jvms method=‘…Unreached hotMethod ()V’ bci=‘3’…/> </uncommon_trap> <bc code='199' bci='3'/> <branch target_bci='17' taken='0' not_taken='5800' cnt='5800' prob='never'/> <uncommon_trap bci='3' reason='unstable_if' action=‘reinterpret' comment='taken never'/>
- 46. cond x = 100 x = -1 trap! x > 0
- 47. REVISED MENTAL MODEL non-sampling period sampling period full-speed period C2 C1Interpreter time C1 C2 additional sampling period stable period Trap + Bail to Interpreter 5x 10-100+x
- 48. func.apply(x); if ( func.getClass() == Square.class ) { x * x; } else if ( func.getClass() == Sqrt.class ) { Math.sqrt(x) } else { … } <klass id='780' name='Square' flags='1'/> <klass id='781' name='Sqrt' flags='1'/> <call method='783' count='23161' prof_factor='1' virtual='1' inline='1' receiver='780' receiver_count=‘19901' receiver2='781' receiver2_count='3260'/> <predicted_call bci='3' klass=‘780'/> <predicted_call bci='3' klass=‘781'/> <uncommon_trap bci='3' reason=‘bimorphic' action='maybe_recompile'/>
- 49. 0 12.5 25 37.5 50 0 10 20 30 40 50 60 70 80 90 100 Deoptimizations DEOPTIMIZATIONS
- 51. 25 % SPECULATIVE OPTIMIZATIONS + EVEN MORE INSIDE A LOOP
- 52. BAIL TO INTERPRETER + LOCK Interpreter C1 C2 uncommon trap + bail to interpreter new call after trap bail to interpreter
- 53. HARD TO IDENTIFY DISRUPTION 25-50ms GOING SLOW — NOT STOPPED AND SOME DO STOP THE WORLD
- 54. WHAT’S YOUR GOAL? STARTUP TIME PEAK PERFORMANCE FIRST RESPONSE PERFORMANCE PREDICTABLE PERFORMANCE
- 55. WHAT DOES THAT MEAN FOR A JIT OR AOT?
- 56. STABLE PERFORMANCE PEAK PERFOMANCE PICK YOUR PERFORMANCE GOAL https://www.youtube.com/watch?v=Xybzyv8qbOc
- 57. ORIGINAL RUN SUBSEQUENT RUNS AZUL READYNOW
- 58. EACH APPROACH HAS PROS & CONS STARTUP TIME PEAK PERFORMANCE FIRST RESPONSE PERFORMANCE PREDICTABLE PERFORMANCE
- 59. IT CAN GET MORE DYNAMIC.
- 60. EVAL EVIL==CLASS GENERATION xml.transform AOP Rules Engines ORMs: Hibernate JSON Libraries Reflection Java 8 Lambdas Dynamic Proxies Spring Derby
- 61. THAT’S A CHALLENGE FOR AN AOT.
- 62. (SIMPLIFIED) CODE LIFECYCLE Interpreter 1 34 2 Profile Deoptimize Just-in-Time Compilation
- 63. REFERENCES ALEKSEY SHIPILËV http://shipilev.net/ PSYCHOMATIC LOBOTOMY SAW http://psy-lob-saw.blogspot.com/ JAVA SPECIALIST NEWSLETTER http://www.javaspecialists.eu/