JVM Magic

The JVM MagicBaruch SadogurskyConsultant & Architect, AlphaCSP

AgendaIntroductionGC Magic 101General OptimizationsCompiler OptimizationsWhat can I do?Programming tipsJVM configuration flags2

IntroductionIn the past, JVM was considered by many as Java Achilles’ heelInterpreter?!JVM team improved performance in 300 to 3000 timesJDK 1.6 compared to JDK 1.0Java is measured to be 50% to 100+% the speed of C and C++Jake2 vs Quake2How can it be?

Java Virtual Machines ZooCEE-J Excelsior JETHewlett-PackardJ9 (IBM)JbedJblendJrockitMRJMicroJvmMS JVMOJVMPERCBlackdown JavaCVMGemstoneGolden Code DevelopmentIntentNovellNSIcomCrE-MEChaiVMHotSpotAegisVMApache HarmonyCACAODalvikIcedTeaIKVM.NETJamigaJamVMJaosJCJelatine JVMJESSICAJikes RVMJnodeJOPJuiceJupiterJXKaffeleJOSMika VMMysaifuNanoVMSableVMSquawk virtual machineSuperWabaTinyVMVMkit of Low Level Virtual MachineWonka VMXam5

HotSpot Virtual MachineDeveloped by Longview Technologies back in 1999Contains:Class loaderBytecode interpreter2 Virtual machines7 Garbage collectors2 CompilersRuntime libraries

HotSpot Virtual MachineConfigured by hundreds of –XX flagsReminder -X options are non-standard-XX options have specific system requirements for correct operationsBoth are subject to change without notice

GC Is Slow?GC has bad performance reputationReduces throughputIntroduces pausesUnpredictableUncontrolledPerformance degradation is proportional to objects countJust give me the damn free() and malloc()! I’ll be just fine!Is it so?

Generational CollectorsWeak generational hypothesisMost objects die young (AKA Infant mortality)Few old to young referencesGenerations: regions holding objects of different agesGC is done separately once a generation fillsDifferent GC algorithmsThe young (nursery) generationCollected by “Minor garbage collection”The old (tenured) generationCollected by “Minor garbage collection”

GC Magic 101vsYoung is better than TenuredLet your objects die in young generationWhen possible and makes sense11

GC Magic 10112vsSwapping is badApplication's memory footprint should not exceed the available physical memory

GC Magic 10113vsChoose:Throughput (client)Low-pause (server)

GC Magic 101http://java.sun.com/javase/technologies/hotspot/gc/gc_tuning_6.html14

Tracking Collectors AlgorithmsMark-Sweep collectorMark phase marks each reachable objectSweep phase “sweeps” the heapNon marked objects reclaimed as garbageCopying collectorHeap is divided into two equal spacesWhen active space fills, live objects are copied to the unused spaceOnly live objects are examinedThe roles of the spaces are then flipped

CompactionCompaction: The collector moves all live objects to the bottom of the heapRemaining memory is reclaimedReduces the cost of objects allocationNo potential fragmentationThe drawback is slower completion of GC

The Young generationConsists of Eden + two survivor spaces Objects are initially allocated in EdenAll HotSpot young collectors are stop-the-world copying collectorsDone is parallel for parallel garbage collectorsCollections are relatively fast and proportional to number of live objects

The Tenured generationObjects surviving several GC cycles, are promoted to the tenured generation Use -XX:MaxTenuringThreshold=# to changeCollectors algorithms used are variations of Mark-SweepMore space efficientCharacteristicsLower garbage densityBigger heap space Fewer GC cycles

Garbage First (G1)New in JDK 1.6 u14 (May 29th)All memory is divided to 1MB bucketsCalculates objects liveness in bucketsDrops “dead” bucketsIf a bucket is not total garbage, it’s not droppedCollects the most garbage buckets firstPauses only on “mark”No sweepUser can provide pause time goalsActual seconds or Percentage of runtimeG1 records bucket collection time and can estimate how many buckets to collect during pause

Garbage First (G1)Targets multi-process machines and large heapsG1 will be the long-term replacement for the CMS collectorUnlike CMS, compacts to battle fragmentationA bucket’s space is fully reclaimedBetter throughputPredictable pauses (high probability)Garbage left in buckets with high live ratioMay be collected later

Benefits of G1No imbalance of young-tenured generationGenerations are only logical Generations are merely sets of buckets More predictable GC pausesParallelism and concurrency in collections No fragmentation due to compactionBetter heap utilization Better GC ergonomics

Young GCs in G1Done using evacuation pausesStop-The-World parallel collectionsEvacuates surviving objects between sets of buckets

Old GCs in G1Drops dead bucketsCalculates liveness info per bucketIdentifies best buckets for subsequent eviction pausesCollect them piggy-backed on young GCs

GC ErgonomicsErgonomics goal is to provide good performance with little or no tuningBetter matches the needs of different application typesThe HotSpot, garbage collector and heap size are automatically chosenBased on OS, RAM and no# CPUServer Vs. Client class machineHints the characteristics of the application

GC ErgonomicsWith the parallel collectors, one can specify performance goalsIn contrast to specifying the heap sizeImproves performance for large applicationsMax Pause Time GoalUse -XX:MaxGCPauseMillis=<N>Both generation separatelyOr: Average + VarianceNo pause time goal by default

GC ErgonomicsThroughput GoalUse -XX:GCTimeRatio=<N>The ratio of GC Vs. application time is 1/(1+N)If N=19, GC time goal is 1/(1+19) or 5%Default N is 99, meaning GC time is 1% Minimum Footprint GoalPriority of goalsMaximum pause time goalThroughput goalMinimum footprint goal

GC ErgonomicsPerformance goals may not be metPause time and throughput goals are somewhat contradictingThe pause time goal shrinks the generationThe throughput goal grows the generationStatistics are kept by the GCAdaptive to changes in application behavior

Heap SizeThe larger the heap space, the betterFor both young and old generationLarger space: less frequent GCs, lower GC overhead, objects more likely to become garbageSmaller space: faster GCs (not always! see later)Sometimes max heap size is dictated by available memory and/or max space the JVM can addressYou have to find a good balance between young and old generation size

Heap SizeMaximize the number of objects reclaimed in the young generationApplication's memory footprint should not exceed the available physical memorySwapping is badThe above apply to all our GCs37

Heap Size-Xmx<size> : max heap sizeyoung generation + old generation-Xms<size> : initial heap sizeyoung generation + old generation-Xmn<size> : young generation size-XX:PermSize=<size> : permanent generation initial size-XX:MaxPermSize=<size> : permanent generation max size38

Heap SizeWhen -Xms != -Xmx, heap growth or shrinking requires a Full GCSet -Xms to desired heap size Set –Xmx even higher “just in case”Even full GC is better than OOM crashSame for -XX:PermSize and -XX:MaxPermSizeSame for -XX:NewSize and-XX:MaxNewSize-Xmn Combines both39

TenuringMeasure tenuring with - XX:+PrintTenuringDistributionAvoid tenuring for short or even medium-lived objects!Less promotion into the old generationLess frequent old GCsPromote long-lived objects ASAPYeah, conflict with previous bulletBetter copy more, than promote more-XX:TargetSurvivorRatio=<percent>, e.g., 50How much of the survivor space should be filledTypically leave extra space to deal with “spikes”40

Permanent SpaceClasses aren’t unloaded by default-XX:+CMSClassUnloadingEnabled to enableClassloader should be collectedIt holds references to classesEach object holds reference to classloader41

GC Statistics OptionsGC logging has extremely low / non-existent overheadIt’s very helpful when diagnosing production issuesEnable itIn production too!-XX:+PrintGCPrintGCDetailsPrintGCTimeStampsPrintTenuringDistributionShow this threshold and the ages of objects in the new generation43

GC Is Slow? – The AnswersReduces throughputYou chooseIntroduces pausesYou chooseUnpredictableNot any moreUncontrolledConfigurablePerformance degradation is proportional to objects countNot trueJust give me the damn free() and malloc()! I’ll be just fine!Bad idea (see more later)

HotSpot OptimizationsJIT CompilationCompiler OptimizationsGenerates more performant code that you could write in nativeAdaptive OptimizationSplit Time VerificationClass Data Sharing

Two Virtual Machines?Client VMReducing start-up time and memory footprint-client CL flagServer VMMaximum program execution speed-server CL flagAuto-detectionServer: >1 CPUs & >=2GB of physical memoryWin32 – always detected as clientMany 64bit OSes don’t have client VMs47

Just-In-Time CompilationEveryone knows about JIT!Hot code is compiled to nativeWhat is “hot”?Server VM – 10000 invocationsClient VM – 1500 invocationsUse -XX:CompileThreshold=# to changeMore invocations – better optimizationsLess invocations – shorter warmup time

Just-In-Time CompilationThe code is being optimized by the compilerComing soon…

Adaptive OptimizationAllows HotSpot to uncompile previously compiled codeMuch more aggressive, even speculative optimizations may be performedAnd rolled back if something goes wrong or new data gatheredE.g. classloading might invalidate inlining

Split Time VerificationJava suffers from long boot timeOne of the reasons is bytecode verificationValid flow controlType safetyVisibilityIn order to ease on the weak KVM, J2ME started performing part of the verification in compile timeIt’s good, so now it’s in Java SE 6 too

Class Data SharingHelps improve startup timeDuring JDK installation part of rt.jar is preloaded into shared memory file which is attached in runtimeNo need to reload and reverify those classes every time

Two Types of OptimizationsJava has two compilers:javac bytecode compilerHotSpot VM JIT compilerBoth implement similar optimizationsBytecode compiler is limitedDynamic linkingCan apply only static optimizations

WarningCaution! Don’t try this at home yourself!The source code you are about to see is not real!It’s pseudo assembly codeDon’t writesuch code!Source code should be readable and object-orientedBytecode will become performant automagically55

Optimization RulesMake the common case fastDon't worry about uncommon/infrequent caseDefer optimization decisionsUntil you have dataRevisit decisions if data warrants56

Null check EliminationJava is null-safe languagePointer can’t point to meaningless portion of memoryNull checks are added by the compiler, NullPointerException is thrownJVM’s profiler can eliminate those checks57

Example – Null Check Elimination59

InliningLove Encapsulation?Getters and settersLove clean and simple code?Small methodsUse static code analysis?Small methodsNo penalty for using those!JIT brings the implementation of these methods into a containing methodThis optimization known as “Inlining”

InliningNot just about eliminating call overheadProvides optimizer with bigger blocksEnables other optimizationshoisting, dead code elimination, code motion, strength reduction61

InliningBut wait, all public non-final methods in Java are virtual!HotSpot examines the exact case in placeIn most cases there is only one implementation, which can be inlinedBut wait, more implementations may be loaded later!In such case HotSpot undoes the inliningSpeculative inliningBy default limited to 35 bytes of bytecodeUse -XX:MaxInlineSize=# to change

Example – Source Code Revision64

Example – Source Code Revision65

Code HoistingHoist = to raise or liftSize optimizationEliminate duplicate code in method bodies by hoisting expressions or statementsDuplicate bytecode, not necessarily source code

Bounds Check EliminationJava promises automatic boundary checks for arraysException is thrownIf programmer checks the boundaries of its array by himself, the automatic check can be turned off

Example – Bounds Check Elimination69

Sub-Expression EliminationAvoids redundant memory access70

Loop UnrollingSome loops shouldn’t be loopsIn performance meaning, not code readabilityThose can be unrolled to set of statementsIf the boundaries are dynamic, partial unroll will occur

JVM Magic

More Related Content

What's hot (20)

Similar to JVM Magic (20)

More from Baruch Sadogursky (20)

Recently uploaded (20)

JVM Magic