SlideShare a Scribd company logo

Moving to G1GC

Kirk Pepperdine, profile picture
Kirk Pepperdine

The document provides details about Garbage First Garbage Collector (G1GC) in Java. It discusses how G1GC works, including that it divides the heap into small fixed-size regions, uses mark-sweep collection for young generations and mostly concurrent mark-sweep collection for old generations. It also provides an example of GC log output and configuration options that can be used to capture GC logs.

Software
1 of 51
Downloaded 48 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
Copyright 2015 Kirk Pepperdine Moving to G1GC
Copyright 2015 Kirk Pepperdine About me - Offer performance tuning services and training - created jPDM, a performance tuning methodology - Write and speak about performance tuning - Co-founder of jClarity - engaged in building the first generation of performance diagnostic engines - Java Champion since 2006
Copyright 2015 Kirk Pepperdine Questions To Be Answered - How does the G1GC algorithm work - What does Java heap look like - What are the tools we can use the to help us - want to engage in evidence based tuning - configure the collector to work better with our application - tune our application to work better with the collector - How does it compare to the other collectors
Copyright 2015 Kirk Pepperdine Generational heap is - 1 large contigous reserved space - specified with -mx (not a type-o) - split into 5 (or 4) memory pools - Eden - Survivor (from, to) - Old - Perm (prior to Java 8)
Copyright 2015 Kirk Pepperdine Generational Garbage Collection - Mark-Sweep Copy (evacuation) for Young - eden and survivor spaces - both serial and parallel implementations - Mark-Sweep (in-place) for Old space - Serial and Parallel with compaction - (mostly) Concurrent Mark-Sweep - incremential mode
Copyright 2015 Kirk Pepperdine Why another collector - Scalability - pause time tends to be a function of heap size - Difficult to tune - dozens of parameters some of which are very difficult to understand how to use - -XX:PLABSize=???? - Completely unpredictable - well, maybe but that is a different talk
Copyright 2015 Kirk Pepperdine G1GC - Designed to scale - break the pause time heap size dependency - Easier to tune - fewer configuration options - Predictable - offer pause time goals and have the collector tune it’s self - Does it work? - lets see!!!
Copyright 2015 Kirk Pepperdine G1GC heap is - 1 large contigous reserved space - specified with -mx - split into ~2048 regions - size is 1, 2, 4, 8, 16, 32, or 64m For -mx10G, Region size = 10240M/2048 = 5m Number of regions = 10G/4m = 2560
Copyright 2015 Kirk Pepperdine Regions - Placed in a free list - When used, tagged as - Eden, Survivor, Old, or Humongous - Retuned to free list after being swept
Copyright 2015 Kirk Pepperdine Allocation - mutator threads get a region from region free list - tag region as Eden - allocate object into region - when region is full, get a new regions from free list Eden Eden Eden Eden
Copyright 2015 Kirk Pepperdine Humongous Allocation - allocation is larger than 1/2 a regions size - size of a regions defines what is humongous - allocate into a humoungous region - created from a set of contigous regions Eden Eden Eden Eden Humongous
Copyright 2015 Kirk Pepperdine Garbage Collection Triggers - Alloted number of Eden regions have been consumed - Unable to satisfy a Humongous allocation - Heap is full triggering a Full GC - Metaspace threshold is reached - full discussion beyond the scope of this talk
Copyright 2015 Kirk Pepperdine Garbage Collection - Mark-Sweep/Mark GC combination - Mark-Sweep for young - (mostly) Concurrent-Mark for Old - Sweep evacuates live objects in a region to another region - automatic compaction - no need for fine grain free lists (expensive) - Marked Old regions are swept by the Young collector - Most phases require threads to be at a safe-point
Copyright 2015 Kirk Pepperdine Safepoint - A point in a threads execution where it can safely stop for JVM maintenance - Cooperative effort - JVM signals it must perform some maintenance - mutator threads stop when they reach a safe point - JVM carries out maintenance - mutator threads are restarted - Measure TTSP vs time to perform maintenance
Copyright 2015 Kirk Pepperdine Heap after a Mark/Sweep - all surviving objects are copied into (to) Survivor regions - Eden and (from) Survivor regions are returned to free regions list Humoungous Survivor
Copyright 2015 Kirk Pepperdine Promotion to Old - Data is promoted to old - from survivor when it reaches tenuring threshold - to prevent survivor from being overrun - pre-emptive or reactive Humongous Survivor Old
Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total 47.896: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 33336, predicted base time: 56.24 ms, remaining time: 143.76 ms, target pause time: 200.00 ms] 47.896: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 89 regions, survivors: 13 regions, predicted young region time: 174.09 ms] 47.896: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 89 regions, survivors: 13 regions, old: 0 regions, predicted pause time: 230.33 ms, target pause time: 200.00 ms] 47.943: [SoftReference, 0 refs, 0.0074530 secs]47.951: [WeakReference, 2 refs, 0.0038714 secs]47.955: [FinalReference, 1753 refs, 0.0058685 secs]47.961: [PhantomReference, 65 refs, 0.0025029 secs]47.963: [JNI Weak Reference, 0.0000256 secs], 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Yes, I know, the font is too small that will be fixed in a moment
Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young), 0.0711540 secs] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log
Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young), 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails
Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total , 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails -XX:+PrintTenuringDistribution
Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total 47.943: [SoftReference, 0 refs, 0.0074530 secs]47.951: [WeakReference, 2 refs, 0.0038714 secs]47.955: [FinalReference, 1753 refs, 0.0058685 secs]47.961: [PhantomReference, 65 refs, 0.0025029 secs]47.963: [JNI Weak Reference, 0.0000256 secs], 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC
Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total 47.896: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 33336, predicted base time: 56.24 ms, remaining time: 143.76 ms, target pause time: 200.00 ms] 47.896: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 89 regions, survivors: 13 regions, predicted young region time: 174.09 ms] 47.896: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 89 regions, survivors: 13 regions, old: 0 regions, predicted pause time: 230.33 ms, target pause time: 200.00 ms] 47.943: [SoftReference, 0 refs, 0.0074530 secs]47.951: [WeakReference, 2 refs, 0.0038714 secs]47.955: [FinalReference, 1753 refs, 0.0058685 secs]47.961: [PhantomReference, 65 refs, 0.0025029 secs]47.963: [JNI Weak Reference, 0.0000256 secs], 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC -XX:+PrintAdaptiveSizePolicy
Copyright 2015 Kirk Pepperdine What is this 1 record telling us? - Initial state of the Java heap - what our application did to the heap - Final state of Java heap - what the GC algorithm did to the heap - how long did the whole process take - breakdown of phases - Need to use 100s of these records to make decisions
Copyright 2015 Kirk Pepperdine Scan for Roots - Find all pointers external to the memory pool in question - Use RSet to track all incoming pointers from one region - prevent a heap scan for roots - uses cards, card marking - card is 1 word that tracks references into 512 bytes - mark bit dirty when reference is updated
Copyright 2015 Kirk Pepperdine RSet - Collection card tables - one card table for pointers originating from each region - Dirty cards are placed in a concurrent refinement queue - refinement threads will build the RSet - aim is to reduce the cost of scan for roots
Copyright 2015 Kirk Pepperdine RSet Refinement - Refinement queue is divided into 4 regions - White: no refinement threads are working - Green: number of cards that can be processed without exceeding 10% of pause time - Yellow: all refinement threads are working to keep up - Red: Application threads are involved in refinement
Copyright 2015 Kirk Pepperdine CSets - Set of all regions to be swept - Goal is to keep pauses under MaxGCPauseMillis - controls the size of the CSet - CSet contain - all Young regions - selected Old regions during mixed collections - number / mixed GC ratio
Copyright 2015 Kirk Pepperdine Reclaiming Memory (detailed) - Mark Sweep Copy (Evacuating) Garbage Collection - Capture all mutator threads at a safepoint - Scan for GC Roots - Trace all references from GC roots - mark all data reached during tracing - Copy all marked data into a “to space” - Reset supporting structures - Release all mutator threads
Copyright 2015 Kirk Pepperdine Parallel Phases - external root scanning - updating remembered sets - scan remembered sets - code root scanning - object copy - string dedup
Copyright 2015 Kirk Pepperdine Serial Phases - code root fixup - code root migration - clear CT - choose CSet - Reference processing - redirty cards - free CSet
Copyright 2015 Kirk Pepperdine GC Log Entry Basics 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) …….. , 0.0711540 secs] …….. …….. [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs]
Copyright 2015 Kirk Pepperdine Parallel Phases [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4]
Copyright 2015 Kirk Pepperdine Serial Phases [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms]
Copyright 2015 Kirk Pepperdine Starting a (mostly) Concurrent Cycle - Scheduled when heap occupancy reaches 45% - initial-mark runs inside a Young collection - mark calculates livelyness - used for CSet inclusion decisions Eden Eden Eden Eden Humoungous Survivor Survivor Old OldOld Old Old Old Old Old OldOld Old Old Old Old Old
Copyright 2015 Kirk Pepperdine Starting a (mostly) Concurrent Mark 4167.445: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested] 2015-10-17T21:13:17.981-0400: 4167.445: [GC pause (G1 Evacuation Pause) (young) (initial-mark) Desired survivor size 4194304 bytes, new threshold 1 (max 1) - age 1: 1585104 bytes, 1585104 total 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms] 4167.459: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 1 regions, survivors: 1 regions, predicted young region time: 0.90 ms] 4167.459: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 1 regions, survivors: 1 regions, old: 0 regions, predicted pause time: 4.88 ms, target pause time: 1.00 ms] , 0.0347452 secs] No need to squint, a bigger version is coming -XX:+PrintAdaptiveSizePolicy
Copyright 2015 Kirk Pepperdine Initiating Heap Occupancy Percent reached 4167.445: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested]
Copyright 2015 Kirk Pepperdine Piggyback an initial-mark onto the young collection 4167.445: [GC pause (G1 Evacuation Pause) (young) (initial-mark) …… ,0.0347452 secs]
Copyright 2015 Kirk Pepperdine CSet construction 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms]
Copyright 2015 Kirk Pepperdine CSet construction 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms] 4167.459: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 1 regions, survivors: 1 regions, predicted young region time: 0.90 ms]
Copyright 2015 Kirk Pepperdine CSet construction 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms] 4167.459: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 1 regions, survivors: 1 regions, predicted young region time: 0.90 ms] 4167.459: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 1 regions, survivors: 1 regions, old: 0 regions, predicted pause time: 4.88 ms, target pause time: 1.00 ms]
Copyright 2015 Kirk Pepperdine 2.172: [GC pause (Metadata GC Threshold) (young) (initial-mark) [rest of young record removed] 2.177: [GC concurrent-root-region-scan-start] 2.179: [GC concurrent-root-region-scan-end, 0.0016985 secs] 2.179: [GC concurrent-mark-start] 2.184: [GC concurrent-mark-end, 0.0045815 secs] 2.184: [GC remark 2.184: [Finalize Marking, 0.0001992 secs] 2.184: [GC ref-proc2.184: [SoftReference, 0 refs, 0.0000104 secs]2.184: [WeakReference, 16 refs, 0.0000089 secs]2.184: [FinalReference, 14 refs, 0.0000082 secs]2.184: [PhantomReference, 0 refs, 1 refs, 0.0000116 secs]2.184: [JNI Weak Reference, 0.0000045 secs], 0.0000561 secs] 2.184: [Unloading, 0.0030695 secs], 0.0034609 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 2.187: [GC cleanup 35M->15M(256M), 0.0004233 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2.188: [GC concurrent-cleanup-start] 2.188: [GC concurrent-cleanup-end, 0.0000148 secs]
Copyright 2015 Kirk Pepperdine Common Failure Conditions [GC pause (young) (to-space exhausted), 0.1709670 secs] - Collection ran out of reserved space - protect against temporary overflows - -XX:G1ReservedPercent=10 - Heap is too small [GC concurrent-mark-reset-for-overflow] - Global marking stack filled - Collection started too late
Copyright 2015 Kirk Pepperdine Tools for evidence based tuning - GC logs - Log file parser/visualization tool - HPJMeter - GCViewer - Censum - Flags - dozens of flags - most of them you don’t want to touch!!!!
Copyright 2015 Kirk Pepperdine Flags (you want to use) -XX:+UseG1GC -mx4G -XX:MaxGCPauseMillis=200 -Xloggc:gc.log -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC" -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime"
Copyright 2015 Kirk Pepperdine Flags (you might want to use) -XX:G1HeapRegionSize=1 -XX:InitiatingHeapOccupancyPercent=45 -XX:+UnlockExperimentalVMOptions" -XX:G1NewSizePercent=5 -XX:+UnlockDiagnosticVMOptions -XX:+G1PrintRegionLivenessInfo -XX:SurvivorRatio=6 -XX:MaxTenuringThreshold=15
Copyright 2015 Kirk Pepperdine Flags (you should think twice about using) -XX:G1MixedGCCountTarget=8 -XX:+UnlockExperimentalVMOptions" -XX:G1MixedGCLiveThresholdPercent=85/65
Copyright 2015 Kirk Pepperdine Flags (you should never use) -XX:+UnlockExperimentalVMOptions" -XX:G1OldCSetRegionThresholdPercent=10 -XX:G1MaxNewSizePercent=60 -XX:G1HeapWastePercent=10 -XX:G1RSetUpdatingPauseTimePercent=10
Copyright 2015 Kirk Pepperdine Tuning Cassandra (benchmark) - Out of the box tuned for using CMS - exceptionally complex set of configurations - Reconfigured - to run G1 - given fixed unit of work which should ideally be cleared in 15 minutes Goal: Configure G1 to maximize MMU
Copyright 2015 Kirk Pepperdine Some results 00:12:35 00:14:40 00:16:45 00:18:50 00:20:55 1 2 3 4 5 6 7 8 9 10 11
Copyright 2015 Kirk Pepperdine More results 0 17500 35000 52500 70000 1 2 3 4 5 6 7 8 9 10 11 12
Copyright 2015 Kirk Pepperdine Performance Seminar www.kodewerk.com Java Performance Tuning, May 26-29, Chania Greece

More Related Content

PDF
Gclogs j1
Kirk Pepperdine
 
PDF
Moving to g1 gc by Kirk Pepperdine.
J On The Beach
 
PPT
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Gurpreet Sachdeva
 
PPTX
Garbage First Garbage Collector (G1 GC) - Migration to, Expectations and Adva...
Monica Beckwith
 
PPTX
Hadoop world g1_gc_forh_base_v4
YanpingWang
 
PPTX
Garbage First Garbage Collector (G1 GC): Current and Future Adaptability and ...
Monica Beckwith
 
PDF
-XX:+UseG1GC
Jakub Kubrynski
 
PPTX
G1 collector and tuning and Cassandra
Chris Lohfink
 
Gclogs j1
Kirk Pepperdine
 
Moving to g1 gc by Kirk Pepperdine.
J On The Beach
 
Java Garbage Collectors – Moving to Java7 Garbage First (G1) Collector
Gurpreet Sachdeva
 
Garbage First Garbage Collector (G1 GC) - Migration to, Expectations and Adva...
Monica Beckwith
 
Hadoop world g1_gc_forh_base_v4
YanpingWang
 
Garbage First Garbage Collector (G1 GC): Current and Future Adaptability and ...
Monica Beckwith
 
-XX:+UseG1GC
Jakub Kubrynski
 
G1 collector and tuning and Cassandra
Chris Lohfink
 

What's hot (20)

PPTX
G1GC
koji lin
 
PDF
Game of Performance: A Song of JIT and GC
Monica Beckwith
 
PPTX
G1 Garbage Collector - Big Heaps and Low Pauses?
C2B2 Consulting
 
PPTX
Tuning Java GC to resolve performance issues
Sergey Podolsky
 
PPTX
Pick diamonds from garbage
Tier1 App
 
PDF
Tuning Java for Big Data
Scott Seighman
 
PDF
The Performance Engineer's Guide To HotSpot Just-in-Time Compilation
Monica Beckwith
 
PPTX
Exactly once with spark streaming
Quentin Ambard
 
PPTX
GoodFit: Multi-Resource Packing of Tasks with Dependencies
DataWorks Summit/Hadoop Summit
 
PDF
JVM and Garbage Collection Tuning
Kai Koenig
 
PDF
Introduction of Java GC Tuning and Java Java Mission Control
Leon Chen
 
PDF
Way Improved :) GC Tuning Confessions - presented at JavaOne2015
Monica Beckwith
 
PDF
淺談 Java GC 原理、調教和 新發展
Leon Chen
 
PDF
Fight with Metaspace OOM
Leon Chen
 
PPTX
JVM memory management & Diagnostics
Dhaval Shah
 
PDF
Processing Big Data in Real-Time - Yanai Franchi, Tikal
Codemotion Tel Aviv
 
PDF
JVM Garbage Collection Tuning
ihji
 
PDF
Basics of JVM Tuning
Vladislav Gangan
 
PDF
Thanos - Prometheus on Scale
Bartłomiej Płotka
 
PDF
Monitoring with Ganglia
Fastly
 
G1GC
koji lin
 
Game of Performance: A Song of JIT and GC
Monica Beckwith
 
G1 Garbage Collector - Big Heaps and Low Pauses?
C2B2 Consulting
 
Tuning Java GC to resolve performance issues
Sergey Podolsky
 
Pick diamonds from garbage
Tier1 App
 
Tuning Java for Big Data
Scott Seighman
 
The Performance Engineer's Guide To HotSpot Just-in-Time Compilation
Monica Beckwith
 
Exactly once with spark streaming
Quentin Ambard
 
GoodFit: Multi-Resource Packing of Tasks with Dependencies
DataWorks Summit/Hadoop Summit
 
JVM and Garbage Collection Tuning
Kai Koenig
 
Introduction of Java GC Tuning and Java Java Mission Control
Leon Chen
 
Way Improved :) GC Tuning Confessions - presented at JavaOne2015
Monica Beckwith
 
淺談 Java GC 原理、調教和 新發展
Leon Chen
 
Fight with Metaspace OOM
Leon Chen
 
JVM memory management & Diagnostics
Dhaval Shah
 
Processing Big Data in Real-Time - Yanai Franchi, Tikal
Codemotion Tel Aviv
 
JVM Garbage Collection Tuning
ihji
 
Basics of JVM Tuning
Vladislav Gangan
 
Thanos - Prometheus on Scale
Bartłomiej Płotka
 
Monitoring with Ganglia
Fastly
 
Ad

Similar to Moving to G1GC (20)

PDF
Tuning the g1gc
Kirk Pepperdine
 
PPTX
Am I reading GC logs Correctly?
Tier1 App
 
ODP
Hotspot gc
Michał Warecki
 
PDF
Le guide de dépannage de la jvm
Jean-Philippe BEMPEL
 
PPTX
GC Tuning: Fortune 500 Case Studies on Cutting Costs and Boosting Performance
KumarNagaraju4
 
PPTX
GC Tuning: Fortune 500 Case Studies on Cutting Costs and Boosting Performance
KumarNagaraju4
 
PPTX
Gc crash course (1)
Tier1 app
 
PDF
Disruptive IP Networking with Intel DPDK on Linux
Naoto MATSUMOTO
 
PPTX
Gc and-pagescan-attacks-by-linux
Cuong Tran
 
PPTX
Become a Garbage Collection Hero
Tier1app
 
PPTX
Hadoop Meetup Jan 2019 - Dynamometer and a Case Study in NameNode GC
Erik Krogen
 
PPTX
GC Tuning: A Masterpiece in Performance Engineering
Tier1 app
 
PDF
JDD2015: -XX:+UseG1GC - Jakub Kubryński
PROIDEA
 
PPTX
Become a GC Hero
Tier1app
 
PDF
Static Analysis and Code Optimizations in Glasgow Haskell Compiler
Ilya Sergey
 
PDF
MesosCon 2018
Pablo Delgado
 
PDF
hbaseconasia2017: HBase Practice At XiaoMi
HBaseCon
 
PPTX
Become a Java GC Hero - ConFoo Conference
Tier1app
 
PPT
Taming Java Garbage Collector
Daya Atapattu
 
PPT
Servers and Processes: Behavior and Analysis
dreamwidth
 
Tuning the g1gc
Kirk Pepperdine
 
Am I reading GC logs Correctly?
Tier1 App
 
Hotspot gc
Michał Warecki
 
Le guide de dépannage de la jvm
Jean-Philippe BEMPEL
 
GC Tuning: Fortune 500 Case Studies on Cutting Costs and Boosting Performance
KumarNagaraju4
 
GC Tuning: Fortune 500 Case Studies on Cutting Costs and Boosting Performance
KumarNagaraju4
 
Gc crash course (1)
Tier1 app
 
Disruptive IP Networking with Intel DPDK on Linux
Naoto MATSUMOTO
 
Gc and-pagescan-attacks-by-linux
Cuong Tran
 
Become a Garbage Collection Hero
Tier1app
 
Hadoop Meetup Jan 2019 - Dynamometer and a Case Study in NameNode GC
Erik Krogen
 
GC Tuning: A Masterpiece in Performance Engineering
Tier1 app
 
JDD2015: -XX:+UseG1GC - Jakub Kubryński
PROIDEA
 
Become a GC Hero
Tier1app
 
Static Analysis and Code Optimizations in Glasgow Haskell Compiler
Ilya Sergey
 
MesosCon 2018
Pablo Delgado
 
hbaseconasia2017: HBase Practice At XiaoMi
HBaseCon
 
Become a Java GC Hero - ConFoo Conference
Tier1app
 
Taming Java Garbage Collector
Daya Atapattu
 
Servers and Processes: Behavior and Analysis
dreamwidth
 
Ad

Recently uploaded (20)

PDF
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
TechBuilder
 
PDF
Design an Analysis of Algorithms II-SECS-1021-03
DilanEscobar1
 
PPTX
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
PDF
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
PDF
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ghrom2211g
 
PPTX
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
PPTX
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
PPTX
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
PDF
System and Network Administration Chapter 2
jaramharo
 
PDF
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
PPTX
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
PDF
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
PPTX
ManageIQ - Sprint 268 Review - Slide Deck
ManageIQ
 
PDF
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
PDF
Nekopoi APK 2025 free lastest update
umarali35kp
 
PPTX
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
PDF
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
PPTX
Introduction to Artificial Intelligence
lohedi9363
 
PDF
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
PPTX
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 
Why TechBuilder is the Future of Pickup and Delivery App Development (1).pdf
TechBuilder
 
Design an Analysis of Algorithms II-SECS-1021-03
DilanEscobar1
 
Odoo POS Development Services by CandidRoot Solutions
CandidRoot Solutions Private Limited
 
Softaken Excel to vCard Converter Software.pdf
markwillsonmw004
 
Internet Downloader Manager (IDM) Crack 6.42 Build 41
ghrom2211g
 
L1 - Introduction to python Backend.pptx
MoizAhmed480282
 
Lecture 3: Operating Systems Introduction to Computer Hardware Systems
hci7se
 
Agentic AI : A Practical Guide. Undersating, Implementing and Scaling Autono...
IT IDOL Technologies
 
System and Network Administration Chapter 2
jaramharo
 
Design an Analysis of Algorithms I-SECS-1021-03
DilanEscobar1
 
ISO 45001 Occupational Health and Safety Management System
EHA Soft Solutions
 
Raksha Bandhan Grocery Pricing Trends in India 2025.pdf
Actowiz Solustions
 
ManageIQ - Sprint 268 Review - Slide Deck
ManageIQ
 
Flood Susceptibility Mapping Using Image-Based 2D-CNN Deep Learnin. Overview ...
lawrenceomai2
 
Nekopoi APK 2025 free lastest update
umarali35kp
 
Agentic AI Use Case- Contract Lifecycle Management (CLM).pptx
mhxyzzp
 
Addressing The Cult of Project Management Tools-Why Disconnected Work is Hold...
OnePlan Solutions
 
Introduction to Artificial Intelligence
lohedi9363
 
Adobe Illustrator 28.6 Crack My Vision of Vector Design
ghrom2211g
 
VVF-Customer-Presentation2025-Ver1.9.pptx
blackmambaettijean
 

Moving to G1GC

  • 1. Copyright 2015 Kirk Pepperdine Moving to G1GC
  • 2. Copyright 2015 Kirk Pepperdine About me - Offer performance tuning services and training - created jPDM, a performance tuning methodology - Write and speak about performance tuning - Co-founder of jClarity - engaged in building the first generation of performance diagnostic engines - Java Champion since 2006
  • 3. Copyright 2015 Kirk Pepperdine Questions To Be Answered - How does the G1GC algorithm work - What does Java heap look like - What are the tools we can use the to help us - want to engage in evidence based tuning - configure the collector to work better with our application - tune our application to work better with the collector - How does it compare to the other collectors
  • 4. Copyright 2015 Kirk Pepperdine Generational heap is - 1 large contigous reserved space - specified with -mx (not a type-o) - split into 5 (or 4) memory pools - Eden - Survivor (from, to) - Old - Perm (prior to Java 8)
  • 5. Copyright 2015 Kirk Pepperdine Generational Garbage Collection - Mark-Sweep Copy (evacuation) for Young - eden and survivor spaces - both serial and parallel implementations - Mark-Sweep (in-place) for Old space - Serial and Parallel with compaction - (mostly) Concurrent Mark-Sweep - incremential mode
  • 6. Copyright 2015 Kirk Pepperdine Why another collector - Scalability - pause time tends to be a function of heap size - Difficult to tune - dozens of parameters some of which are very difficult to understand how to use - -XX:PLABSize=???? - Completely unpredictable - well, maybe but that is a different talk
  • 7. Copyright 2015 Kirk Pepperdine G1GC - Designed to scale - break the pause time heap size dependency - Easier to tune - fewer configuration options - Predictable - offer pause time goals and have the collector tune it’s self - Does it work? - lets see!!!
  • 8. Copyright 2015 Kirk Pepperdine G1GC heap is - 1 large contigous reserved space - specified with -mx - split into ~2048 regions - size is 1, 2, 4, 8, 16, 32, or 64m For -mx10G, Region size = 10240M/2048 = 5m Number of regions = 10G/4m = 2560
  • 9. Copyright 2015 Kirk Pepperdine Regions - Placed in a free list - When used, tagged as - Eden, Survivor, Old, or Humongous - Retuned to free list after being swept
  • 10. Copyright 2015 Kirk Pepperdine Allocation - mutator threads get a region from region free list - tag region as Eden - allocate object into region - when region is full, get a new regions from free list Eden Eden Eden Eden
  • 11. Copyright 2015 Kirk Pepperdine Humongous Allocation - allocation is larger than 1/2 a regions size - size of a regions defines what is humongous - allocate into a humoungous region - created from a set of contigous regions Eden Eden Eden Eden Humongous
  • 12. Copyright 2015 Kirk Pepperdine Garbage Collection Triggers - Alloted number of Eden regions have been consumed - Unable to satisfy a Humongous allocation - Heap is full triggering a Full GC - Metaspace threshold is reached - full discussion beyond the scope of this talk
  • 13. Copyright 2015 Kirk Pepperdine Garbage Collection - Mark-Sweep/Mark GC combination - Mark-Sweep for young - (mostly) Concurrent-Mark for Old - Sweep evacuates live objects in a region to another region - automatic compaction - no need for fine grain free lists (expensive) - Marked Old regions are swept by the Young collector - Most phases require threads to be at a safe-point
  • 14. Copyright 2015 Kirk Pepperdine Safepoint - A point in a threads execution where it can safely stop for JVM maintenance - Cooperative effort - JVM signals it must perform some maintenance - mutator threads stop when they reach a safe point - JVM carries out maintenance - mutator threads are restarted - Measure TTSP vs time to perform maintenance
  • 15. Copyright 2015 Kirk Pepperdine Heap after a Mark/Sweep - all surviving objects are copied into (to) Survivor regions - Eden and (from) Survivor regions are returned to free regions list Humoungous Survivor
  • 16. Copyright 2015 Kirk Pepperdine Promotion to Old - Data is promoted to old - from survivor when it reaches tenuring threshold - to prevent survivor from being overrun - pre-emptive or reactive Humongous Survivor Old
  • 17. Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total 47.896: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 33336, predicted base time: 56.24 ms, remaining time: 143.76 ms, target pause time: 200.00 ms] 47.896: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 89 regions, survivors: 13 regions, predicted young region time: 174.09 ms] 47.896: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 89 regions, survivors: 13 regions, old: 0 regions, predicted pause time: 230.33 ms, target pause time: 200.00 ms] 47.943: [SoftReference, 0 refs, 0.0074530 secs]47.951: [WeakReference, 2 refs, 0.0038714 secs]47.955: [FinalReference, 1753 refs, 0.0058685 secs]47.961: [PhantomReference, 65 refs, 0.0025029 secs]47.963: [JNI Weak Reference, 0.0000256 secs], 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Yes, I know, the font is too small that will be fixed in a moment
  • 18. Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young), 0.0711540 secs] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log
  • 19. Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young), 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails
  • 20. Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total , 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails -XX:+PrintTenuringDistribution
  • 21. Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total 47.943: [SoftReference, 0 refs, 0.0074530 secs]47.951: [WeakReference, 2 refs, 0.0038714 secs]47.955: [FinalReference, 1753 refs, 0.0058685 secs]47.961: [PhantomReference, 65 refs, 0.0025029 secs]47.963: [JNI Weak Reference, 0.0000256 secs], 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC
  • 22. Copyright 2015 Kirk Pepperdine 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) Desired survivor size 109051904 bytes, new threshold 3 (max 15) - age 1: 54191968 bytes, 54191968 total - age 2: 40340312 bytes, 94532280 total - age 3: 41274112 bytes, 135806392 total 47.896: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 33336, predicted base time: 56.24 ms, remaining time: 143.76 ms, target pause time: 200.00 ms] 47.896: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 89 regions, survivors: 13 regions, predicted young region time: 174.09 ms] 47.896: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 89 regions, survivors: 13 regions, old: 0 regions, predicted pause time: 230.33 ms, target pause time: 200.00 ms] 47.943: [SoftReference, 0 refs, 0.0074530 secs]47.951: [WeakReference, 2 refs, 0.0038714 secs]47.955: [FinalReference, 1753 refs, 0.0058685 secs]47.961: [PhantomReference, 65 refs, 0.0025029 secs]47.963: [JNI Weak Reference, 0.0000256 secs], 0.0711540 secs] [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4] [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms] [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs] Captured using: -Xloggc:gc.log —XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC -XX:+PrintAdaptiveSizePolicy
  • 23. Copyright 2015 Kirk Pepperdine What is this 1 record telling us? - Initial state of the Java heap - what our application did to the heap - Final state of Java heap - what the GC algorithm did to the heap - how long did the whole process take - breakdown of phases - Need to use 100s of these records to make decisions
  • 24. Copyright 2015 Kirk Pepperdine Scan for Roots - Find all pointers external to the memory pool in question - Use RSet to track all incoming pointers from one region - prevent a heap scan for roots - uses cards, card marking - card is 1 word that tracks references into 512 bytes - mark bit dirty when reference is updated
  • 25. Copyright 2015 Kirk Pepperdine RSet - Collection card tables - one card table for pointers originating from each region - Dirty cards are placed in a concurrent refinement queue - refinement threads will build the RSet - aim is to reduce the cost of scan for roots
  • 26. Copyright 2015 Kirk Pepperdine RSet Refinement - Refinement queue is divided into 4 regions - White: no refinement threads are working - Green: number of cards that can be processed without exceeding 10% of pause time - Yellow: all refinement threads are working to keep up - Red: Application threads are involved in refinement
  • 27. Copyright 2015 Kirk Pepperdine CSets - Set of all regions to be swept - Goal is to keep pauses under MaxGCPauseMillis - controls the size of the CSet - CSet contain - all Young regions - selected Old regions during mixed collections - number / mixed GC ratio
  • 28. Copyright 2015 Kirk Pepperdine Reclaiming Memory (detailed) - Mark Sweep Copy (Evacuating) Garbage Collection - Capture all mutator threads at a safepoint - Scan for GC Roots - Trace all references from GC roots - mark all data reached during tracing - Copy all marked data into a “to space” - Reset supporting structures - Release all mutator threads
  • 29. Copyright 2015 Kirk Pepperdine Parallel Phases - external root scanning - updating remembered sets - scan remembered sets - code root scanning - object copy - string dedup
  • 30. Copyright 2015 Kirk Pepperdine Serial Phases - code root fixup - code root migration - clear CT - choose CSet - Reference processing - redirty cards - free CSet
  • 31. Copyright 2015 Kirk Pepperdine GC Log Entry Basics 2015-07-13T02:17:35.909-0700: 47.896: [GC pause (G1 Evacuation Pause) (young) …….. , 0.0711540 secs] …….. …….. [Eden: 1424.0M(1424.0M)->0.0B(1632.0M) Survivors: 208.0M->208.0M Heap: 3003.2M(32.0G)->1706.3M(32.0G)] [Times: user=1.02 sys=0.03, real=0.07 secs]
  • 32. Copyright 2015 Kirk Pepperdine Parallel Phases [Parallel Time: 43.5 ms, GC Workers: 23] [GC Worker Start (ms): Min: 47896.5, Avg: 47896.9, Max: 47897.3, Diff: 0.8] [Ext Root Scanning (ms): Min: 0.9, Avg: 1.5, Max: 3.3, Diff: 2.4, Sum: 35.4] [Update RS (ms): Min: 9.7, Avg: 10.9, Max: 11.8, Diff: 2.1, Sum: 251.0] [Processed Buffers: Min: 5, Avg: 11.3, Max: 26, Diff: 21, Sum: 260] [Scan RS (ms): Min: 2.2, Avg: 2.7, Max: 3.0, Diff: 0.8, Sum: 61.3] [Code Root Scanning (ms): Min: 0.0, Avg: 0.0, Max: 0.1, Diff: 0.1, Sum: 0.2] [Object Copy (ms): Min: 25.2, Avg: 26.3, Max: 27.5, Diff: 2.3, Sum: 604.1] [Termination (ms): Min: 0.0, Avg: 0.2, Max: 0.3, Diff: 0.3, Sum: 4.2] [GC Worker Other (ms): Min: 0.0, Avg: 0.2, Max: 0.4, Diff: 0.4, Sum: 3.6] [GC Worker Total (ms): Min: 41.2, Avg: 41.7, Max: 42.3, Diff: 1.0, Sum: 959.7] [GC Worker End (ms): Min: 47938.5, Avg: 47938.6, Max: 47938.9, Diff: 0.4]
  • 33. Copyright 2015 Kirk Pepperdine Serial Phases [Code Root Fixup: 0.5 ms] [Code Root Migration: 0.5 ms] [Code Root Purge: 0.0 ms] [Clear CT: 1.0 ms] [Other: 25.6 ms] [Choose CSet: 0.0 ms] [Ref Proc: 22.7 ms] [Ref Enq: 0.5 ms] [Redirty Cards: 1.0 ms] [Free CSet: 0.4 ms]
  • 34. Copyright 2015 Kirk Pepperdine Starting a (mostly) Concurrent Cycle - Scheduled when heap occupancy reaches 45% - initial-mark runs inside a Young collection - mark calculates livelyness - used for CSet inclusion decisions Eden Eden Eden Eden Humoungous Survivor Survivor Old OldOld Old Old Old Old Old OldOld Old Old Old Old Old
  • 35. Copyright 2015 Kirk Pepperdine Starting a (mostly) Concurrent Mark 4167.445: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested] 2015-10-17T21:13:17.981-0400: 4167.445: [GC pause (G1 Evacuation Pause) (young) (initial-mark) Desired survivor size 4194304 bytes, new threshold 1 (max 1) - age 1: 1585104 bytes, 1585104 total 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms] 4167.459: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 1 regions, survivors: 1 regions, predicted young region time: 0.90 ms] 4167.459: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 1 regions, survivors: 1 regions, old: 0 regions, predicted pause time: 4.88 ms, target pause time: 1.00 ms] , 0.0347452 secs] No need to squint, a bigger version is coming -XX:+PrintAdaptiveSizePolicy
  • 36. Copyright 2015 Kirk Pepperdine Initiating Heap Occupancy Percent reached 4167.445: [G1Ergonomics (Concurrent Cycles) initiate concurrent cycle, reason: concurrent cycle initiation requested]
  • 37. Copyright 2015 Kirk Pepperdine Piggyback an initial-mark onto the young collection 4167.445: [GC pause (G1 Evacuation Pause) (young) (initial-mark) …… ,0.0347452 secs]
  • 38. Copyright 2015 Kirk Pepperdine CSet construction 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms]
  • 39. Copyright 2015 Kirk Pepperdine CSet construction 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms] 4167.459: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 1 regions, survivors: 1 regions, predicted young region time: 0.90 ms]
  • 40. Copyright 2015 Kirk Pepperdine CSet construction 4167.459: [G1Ergonomics (CSet Construction) start choosing CSet, _pending_cards: 984, predicted base time: 3.98 ms, remaining time: 0.00 ms, target pause time: 1.00 ms] 4167.459: [G1Ergonomics (CSet Construction) add young regions to CSet, eden: 1 regions, survivors: 1 regions, predicted young region time: 0.90 ms] 4167.459: [G1Ergonomics (CSet Construction) finish choosing CSet, eden: 1 regions, survivors: 1 regions, old: 0 regions, predicted pause time: 4.88 ms, target pause time: 1.00 ms]
  • 41. Copyright 2015 Kirk Pepperdine 2.172: [GC pause (Metadata GC Threshold) (young) (initial-mark) [rest of young record removed] 2.177: [GC concurrent-root-region-scan-start] 2.179: [GC concurrent-root-region-scan-end, 0.0016985 secs] 2.179: [GC concurrent-mark-start] 2.184: [GC concurrent-mark-end, 0.0045815 secs] 2.184: [GC remark 2.184: [Finalize Marking, 0.0001992 secs] 2.184: [GC ref-proc2.184: [SoftReference, 0 refs, 0.0000104 secs]2.184: [WeakReference, 16 refs, 0.0000089 secs]2.184: [FinalReference, 14 refs, 0.0000082 secs]2.184: [PhantomReference, 0 refs, 1 refs, 0.0000116 secs]2.184: [JNI Weak Reference, 0.0000045 secs], 0.0000561 secs] 2.184: [Unloading, 0.0030695 secs], 0.0034609 secs] [Times: user=0.03 sys=0.00, real=0.01 secs] 2.187: [GC cleanup 35M->15M(256M), 0.0004233 secs] [Times: user=0.00 sys=0.00, real=0.00 secs] 2.188: [GC concurrent-cleanup-start] 2.188: [GC concurrent-cleanup-end, 0.0000148 secs]
  • 42. Copyright 2015 Kirk Pepperdine Common Failure Conditions [GC pause (young) (to-space exhausted), 0.1709670 secs] - Collection ran out of reserved space - protect against temporary overflows - -XX:G1ReservedPercent=10 - Heap is too small [GC concurrent-mark-reset-for-overflow] - Global marking stack filled - Collection started too late
  • 43. Copyright 2015 Kirk Pepperdine Tools for evidence based tuning - GC logs - Log file parser/visualization tool - HPJMeter - GCViewer - Censum - Flags - dozens of flags - most of them you don’t want to touch!!!!
  • 44. Copyright 2015 Kirk Pepperdine Flags (you want to use) -XX:+UseG1GC -mx4G -XX:MaxGCPauseMillis=200 -Xloggc:gc.log -XX:+PrintGCDetails -XX:+PrintTenuringDistribution -XX:+PrintReferenceGC" -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime"
  • 45. Copyright 2015 Kirk Pepperdine Flags (you might want to use) -XX:G1HeapRegionSize=1 -XX:InitiatingHeapOccupancyPercent=45 -XX:+UnlockExperimentalVMOptions" -XX:G1NewSizePercent=5 -XX:+UnlockDiagnosticVMOptions -XX:+G1PrintRegionLivenessInfo -XX:SurvivorRatio=6 -XX:MaxTenuringThreshold=15
  • 46. Copyright 2015 Kirk Pepperdine Flags (you should think twice about using) -XX:G1MixedGCCountTarget=8 -XX:+UnlockExperimentalVMOptions" -XX:G1MixedGCLiveThresholdPercent=85/65
  • 47. Copyright 2015 Kirk Pepperdine Flags (you should never use) -XX:+UnlockExperimentalVMOptions" -XX:G1OldCSetRegionThresholdPercent=10 -XX:G1MaxNewSizePercent=60 -XX:G1HeapWastePercent=10 -XX:G1RSetUpdatingPauseTimePercent=10
  • 48. Copyright 2015 Kirk Pepperdine Tuning Cassandra (benchmark) - Out of the box tuned for using CMS - exceptionally complex set of configurations - Reconfigured - to run G1 - given fixed unit of work which should ideally be cleared in 15 minutes Goal: Configure G1 to maximize MMU
  • 49. Copyright 2015 Kirk Pepperdine Some results 00:12:35 00:14:40 00:16:45 00:18:50 00:20:55 1 2 3 4 5 6 7 8 9 10 11
  • 50. Copyright 2015 Kirk Pepperdine More results 0 17500 35000 52500 70000 1 2 3 4 5 6 7 8 9 10 11 12
  • 51. Copyright 2015 Kirk Pepperdine Performance Seminar www.kodewerk.com Java Performance Tuning, May 26-29, Chania Greece