JVM memory management & Diagnostics

Editor's Notes

• #4: ========= Memory space is divided into generations • Separate pools holding objects of different age ranges • Based on hypothesis: – Most allocated objects die young – Few references from older to younger objects exist • To take advantage of this hypothesis, heap is divided into two generations – Young: small and collected frequently – Old : larger and occupancy grows slowly • Minor(young) and Major(Full) collections Permanent Generation • HotSpot JVM prior to JDK 8 had a third generation called Permanent Generation • Used for: – JVM internal representation of classes and their metadata – Class statics – Interned strings • Contiguous with the Java Heap Metaspace • JDK 8 does not have Permanent Generation • Class metadata is stored in a new space called Metaspace • Not contiguous with the Java Heap • Metaspace is allocated out of native memory • Maximum space available to the Metaspace is the available system memory • This can though be limited by MaxMetaspaceSize JVM option Compressed Class Space • If UseCompressedClassesPointers is enabled then two separate areas of memory are used for the classes and its metadata – Metaspace – Compressed class space • 64-bit class pointers are represented with 32-bit offsets • Class metadata referenced by the 32-bit offsets is stored in the Compressed Class Space • By default compressed class space is sized at 1GB • MaxMetaspaceSize sets an upper limit on the committed size of both of these spaces. Code Cache • Code Cache is used to store the compiled code generated by the Just-intime compilers • It is allocated out of native memory • Managed by the Code Cache Sweeper ===== From Monica beckworth video Most objects die young,  hence they can be aged inthe nursery generation.  The nursery generationis also commonly known  as the young generation.  The nursery itself is further divided  into Eden and survivor spaces.  Live objects within the nursery are aged  by moving them back andforth in the survivor spaces.
• #6: From gc plumbr pdf (pg 11 to 14)
• #8: 2. Content from my blog 2.1. http://dhaval-shah.com ======= > From Monica Beckwith Survivor objects are copied and aged in Survivor Space Overflow and aged objects are tenured to Old Generation Promotion of objects happens in the Promoting thread’s local area buffers (P LABs) ====== > From Monica Beckwith For content related to – Mark Sweep Compaction process understanding along with images for slides - Promotion failure and concurrent mode failure (CMF) : Understanding how (image for slides) and causes for the same
• #9: 1. Slide 36 and Slide 37 - 2. Content from my blog 2.1. https://www.cubrid.org/blog/understanding-java-garbage-collection ======= > From Monica Beckwith Survivor objects are copied and aged in Survivor Space Overflow and aged objects are tenured to Old Generation Promotion of objects happens in the Promoting thread’s local area buffers (P LABs) ====== > From Monica Beckwith For content related to – Mark Sweep Compaction process understanding along with images for slides - Promotion failure and concurrent mode failure (CMF) : Understanding how (image for slides) and causes for the same
• #10: 1. Slide 36 and Slide 37 - http://www.oracle.com/webfolder/technetwork/tutorials/mooc/JVM_Troubleshooting/week1/lesson1.pdf 2. Content from my blog 2.1. https://www.cubrid.org/blog/understanding-java-garbage-collection ======= > From Monica Beckwith Survivor objects are copied and aged in Survivor Space Overflow and aged objects are tenured to Old Generation Promotion of objects happens in the Promoting thread’s local area buffers (P LABs) ====== > From Monica Beckwith For content related to – Mark Sweep Compaction process understanding along with images for slides - Promotion failure and concurrent mode failure (CMF) : Understanding how (image for slides) and causes for the same
• #11: 1. Slide 36 and Slide 37 - http://www.oracle.com/webfolder/technetwork/tutorials/mooc/JVM_Troubleshooting/week1/lesson1.pdf 2. Content from my blog 2.1. https://www.cubrid.org/blog/understanding-java-garbage-collection ======= > From Monica Beckwith (https://www.safaribooksonline.com/library/view/advanced-java-performance/9780134653273/ajph_01_02.html) Survivor objects are copied and aged in Survivor Space Overflow and aged objects are tenured to Old Generation Promotion of objects happens in the Promoting thread’s local area buffers (P LABs) ====== > From Monica Beckwith (https://www.safaribooksonline.com/library/view/advanced-java-performance/9780134653273/ajph_01_03.html) For content related to – Mark Sweep Compaction process understanding along with images for slides - Promotion failure and concurrent mode failure (CMF) : Understanding how (image for slides) and causes for the same
• #12: Now that we have reviewed the core concepts behind GC algorithms, let us move to the specific implementations one can find inside the JVM. An important aspect to recognize first is the fact that, for most JVMs out there, two different GC algorithms are needed – one to clean the Young Generation and another to clean the Old Generation. You can choose from a variety of such algorithms bundled into the JVM. If you do not specify a garbage collection algorithm explicitly, a platform-specific default will be used. In this chapter, the working principles of each of those algorithms will be explained. For a quick cheat sheet, the following list is a fast way to get yourself up to speed with which algorithm combinations are possible. Note that this stands true for Java 8, for older Java versions the available combinations might differ a bit: If the above looks too complex, do not worry. In reality it all boils down to just four combinations highlighted in the table above. The rest are either deprecated, not supported or just impractical to apply in real world. So, in the following chapters we cover the working principles of the following combinations:   Serial GC for both the Young and Old generations   Parallel GC for both the Young and Old generations   Parallel New for Young + Concurrent Mark and Sweep (CMS) for the Old Generation   G1 in case of which the generations are not separated between the Young and Old ===== from deleted slide Slide 37, 41 to 44 - http://www.oracle.com/webfolder/technetwork/tutorials/mooc/JVM_Troubleshooting/week1/lesson1.pdf OR https://javapapers.com/java/types-of-java-garbage-collectors/ https://www.safaribooksonline.com/library/view/java-performance-companion/9780133796896/ch01.html 2. Content from my blog 2.1. https://www.cubrid.org/blog/understanding-java-garbage-collection ========================================= – Serial is a stop-the-world, copying collector that uses a single GC thread – ParNew is a stop-the-world, copying collector that uses multiple GC threads – Parallel Scavenge is a stop-the-world, copying collector that uses multiple GC threads
• #13: OR https://javapapers.com/java/types-of-java-garbage-collectors/ https://www.safaribooksonline.com/library/view/java-performance-companion/9780133796896/ch01.html ========================================= – Serial is a stop-the-world, copying collector that uses a single GC thread – ParNew is a stop-the-world, copying collector that uses multiple GC threads – Parallel Scavenge is a stop-the-world, copying collector that uses multiple GC threads Serial Old is a stop-the-world, mark-sweep-compact collector that uses a single GC thread – CMS is a mostly concurrent, low-pause collector – Parallel Old is a compacting collector that uses multiple GC threads Detailed explanation for content can be copied from javapapers url (as mentioned above) ============= from older parallel vs cms slide
• #14: 1. Slide 12 -
• #15: 1. Slide 12 - http://www.oracle.com/webfolder/technetwork/tutorials/mooc/JVM_Troubleshooting/week2/lesson2.pdf
• #16: Slide 12 - 0,50,2 – 50,50,0 – 100,50,-1
• #17: Slide 12 - http://www.oracle.com/webfolder/technetwork/tutorials/mooc/JVM_Troubleshooting/week2/lesson2.pdf 0,50,2 – 50,50,0 – 100,50,-1
• #18: Performance taxonomy from Ch1 of Optimizing Java ====== > From Monica Beckwith 2 main drivers towards optimization of GC – Throughput and latency Elaborate explanation about throughput and latency parameters w.r.t GC Venn Diagram about classifying Parallel GC and CMS GC , G1 GCw.r.t throughput / latency maximizer ======= > From Monica Beckwith Min GC overhead = Max application Throughput ------------------------------------------ CONTENT There are 2 key aspects of performance that need to be factored in whilst tuning GC – a- Throughput : Throughput is a metric that represents the rate of work a system or subsystem can perform. This is usually expressed as number of units of work in some time period. For example, we might be interested in how many transactions per second a system can execute. b- Latency : If a water pipe can produce 100 liters per second, then the volume produced in 1 second (100 liters) is the throughput. In this metaphor, the latency is effectively the length of the pipe. That is, it’s the time taken to process a single transaction and see a result at the other end of the pipe. It is normally quoted as an end-to-end time. It is dependent on workload, so a common approach is to produce a graph showing latency as a function of increasing workload.  Fundamental question that needs to be answered is how to maximize throughput? So far maximizing throughput, GC needs to be executed in parallel and by that principle, it will have minimal impact on the application Goal will be to do concurrent work and that too as quickly as possible. This implicitly means that, it, should increase concurrent threads for GC processing Parallel GC / Throughput GC - It has STW threads for Young and Old - parallel GC threads for yount and old - No GC work is carried out concurrently with your application 2nd key area that is also equally important whilst tuning GC is latency – How to reduce latency? This definitely means that it pauses / stalls only when it is absolutely needed. It should also work concurrently with application threads, which implicitly means that we have lesser STW GC pause time CMS GC / Latency GC - It is basically concurrent marking and sweeping and does it so smartly that compaction is none / minimal - STW are there but they are only used for Young collection and that too in some specific phases of marking cycle - No STW threads for old generations
• #19: Allocation rate is a term used when communicating the amount of memory allocated per time unit. Often it is expressed in MB/sec, but you can use PB/year if you feel like it. So that is all there is – no magic, just the amount of memory you allocate in your Java code measured over a period of time. An excessively high allocation rate can mean trouble for your application’s performance. When running on a JVM, the problem will be revealed by garbage collection posing a large overhead. From the GC log above, we can calculate the allocation rate as the difference between the sizes of the young generation after the completion of the last collection and before the start of the next one. Using the example above, we can extract the following information:  At 291 ms after the JVM was launched, 33,280 K of objects were created. The first minor GC event cleaned the young generation, after which there were 5,088 K of objects in the young generation left.  At 446 ms after launch, the young generation occupancy had grown to 38,368 K, triggering the next GC, which managed to reduce the young generation occupancy to 5,120 K.  At 829 ms after the launch, the size of the young generation was 71,680 K and the GC reduced it again to 5,120 K. This data can then be expressed in the following table calculating the allocation rate as deltas of the young occupancy: <> Why Should I Care? After measuring the allocation rate we can understand how the changes in allocation rate affect application throughput by increasing or reducing the frequency of GC pauses. First and foremost, you should notice that only minor GC pauses cleaning the young generation are affected. Neither the frequency nor duration of the GC pauses cleaning the old generation are directly impacted by the allocation rate, but instead by the promotion rate, a term that we will cover separately in the next section. Knowing that we can focus only on Minor GC pauses, we should next look into the different memory pools inside the young generation. As the allocation takes place in Eden, we can immediately look into how sizing Eden can impact the allocation rate. So we can hypothesize that increasing the size of Eden will reduce the frequency of minor GC pauses and thus allow the application to sustain faster allocation rates. Get an understanding of g -XX:NewSize -XX:MaxNewSize & -XX:SurvivorRatio parameters Now, before you jump to the conclusion that “bigger Eden is better”, you should notice that the allocation rate might and probably does not directly correlate with the actual throughput of your application. It is a technical measurement contributing to throughput. The allocation rate can and will have an impact on how frequently your minor GC pauses stop application threads, but to see the overall impact, you also need to take into account major GC pauses and measure throughput not in MB/sec but in the business operations your application provides. Another important aspect for analyzing GC logs will be to understand frequency of minor GC events. This indicates that there are lots and lots of objects being allocated. Additionally, the post-GC occupancy of the young generation remains low, and no full collections are happening. These symptoms indicate that the GC is having significant impact to the throughput of the application at hand. What is the Solution? In some cases, reducing the impact of high allocation rates can be as easy as increasing the size of the young generation. Doing so will not reduce the allocation rate itself, but will result in less frequent collections. The benefit of the approach kicks in when there will be only a few survivors every time. As the duration of a minor GC pause is impacted by the number of surviving objects, they will not noticeably increase here.
• #20: Before explaining the concept of premature promotion, we should familiarize ourselves with the concept it builds upon – the promotion rate. The promotion rate is measured in the amount of data propagated from the young generation to the old generation per time unit. It is often measured in MB/sec, similarly to the allocation rate. Promoting long-lived objects from the young generation to the old is how JVM is expected to behave. Recalling the generation hypothesis we can now construct a situation where not only long-lived objects end up in the old generation. Such a situation, where objects with a short life expectancy are not collected in the young generation and get promoted to the old generation, is called premature promotion. Cleaning these short-lived objects now becomes a job for major GC, which is not designed for frequent runs and results in longer GC pauses. This significantly affects the throughput of the application. From the above we can extract the size of the young Generation and the total heap both before and after the collection event. Knowing the consumption of the young generation and the total heap, it is easy to calculate the consumption of the old generation as just the delta between the two. Expressing the information in GC logs as: <Animation> will allow us to extract the promotion rate for the measured period. We can see that on average the promotion rate was 92 MB/sec, peaking at 140.95 MB/sec for a while. Notice that you can extract this information only from minor GC pauses. Full GC pauses do not expose the promotion rate as the change in the old generation usage in GC logs also includes objects cleaned by the major GC. Why Should I Care? Similarly to the allocation rate, the main impact of the promotion rate is the change of frequency in GC pauses. But as opposed to the allocation rate that affects the frequency of minor GC events, the promotion rate affects the frequency of major GC events. Let me explain – the more stuff you promote to the old generation the faster you will fill it up. Filling the old generation faster means that the frequency of the GC events cleaning the old generation will increase. As we have shown in earlier chapters, full garbage collections typically require much more time, as they have to interact with many more objects, and perform additional complex activities such as defragmentation. Could my JVMs be Affected? In general, the symptoms of premature promotion can take any of the following forms:  The application goes through frequent full GC runs over a short period of time.  The old generation consumption after each full GC is low, often under 10-20% of the total size of the old generation.  Facing the promotion rate approaching the allocation rate. In a nutshell, to fix this problem, we would need to make the buffered data fit into the young generation. There are two simple approaches for doing this. The first is to increase the young generation size by using -Xmx64m -XX:NewSize=32m parameters at JVM startup. Running the application with this change in configuration will make Full GC events much less frequent, while barely affecting the duration of minor collections: Another approach in this case would be to simply decrease the batch size, which would also give a similar result. Picking the right solution heavily depends on what is really happening in the application. In some cases, business logic does not permit decreasing batch size. In this case, increasing available memory or redistributing in favor of the young generation might be possible. If neither is a viable option, then perhaps data structures can be optimized to consume less memory. But the general goal in this case remains the same: make transient data fit into the young generation.
• #21: Total GC pause is 27 ms, out of which parallel time took 24.3 ms and from that object copy took max 23.6 mseconds. So object copy is one of the largest contributor to gC pause time. How to tune GC elapsed time- Size your generation and age your objects appropriately Higher the amount of data to be copied, the longer GC pause Premature promotions are big problems (as seen earlier) bcoz if ur old generation gets filled, it triggerts full GC and they are very time consuming.
• #25: Slide 12 MP