Jvm performance tuning

Editor's Notes

• #6: There is no point in proceeding further and trying to understand anything on performance front till the time we do not understand what is GC and how it works fundamentally. Here we will discuss everything that we need to, with regards to GC. What is GC, how it happens, when it happens and why is it important for performance tuning. We will also discuss “root objects” with respect to GC. Here we will not discuss how to tune GC as this is covered in another session. We will also discuss how Threads (once put in service) work, its execution, the different states of Thread, Object Monitor etc. This is extremely important to understand the Thread Dumps.
• #10: Why Garbage Collection ? The name "garbage collection" implies that objects no longer needed by the program are "garbage" and can be thrown away. A more accurate and up-to-date metaphor might be "memory recycling." When an object is no longer referenced by the program, the heap space it occupies can be recycled so that the space is made available for subsequent new objects. The garbage collector must somehow determine which objects are no longer referenced by the program and make available the heap space occupied by such unreferenced objects. In the process of freeing unreferenced objects, the garbage collector must run any finalizers of objects being freed. In addition to freeing unreferenced objects, a garbage collector may also combat heap fragmentation. Heap fragmentation occurs through the course of normal program execution. New objects are allocated, and unreferenced objects are freed such that free portions of heap memory are left in between portions occupied by live objects. Requests to allocate new objects may have to be filled by extending the size of the heap even though there is enough total unused space in the existing heap. This will happen if there is not enough contiguous free heap space available into which the new object will fit. On a virtual memory system, the extra paging (or swapping) required to service an ever growing heap can degrade the performance of the executing program. On an embedded system with low memory, fragmentation could cause the virtual machine to "run out of memory" unnecessarily. Garbage collection relieves you from the burden of freeing allocated memory. Knowing when to explicitly free allocated memory can be very tricky. Giving this job to the Java virtual machine has several advantages. First, it can make you more productive. When programming in non-garbage-collected languages you can spend many late hours (or days or weeks) chasing down an elusive memory problem. When programming in Java you can use that time more advantageously by getting ahead of schedule or simply going home to have a life. A second advantage of garbage collection is that it helps ensure program integrity. Garbage collection is an important part of Java's security strategy. Java programmers are unable to accidentally (or purposely) crash the Java virtual machine by incorrectly freeing memory. A potential disadvantage of a garbage-collected heap is that it adds an overhead that can affect program performance. The Java virtual machine has to keep track of which objects are being referenced by the executing program, and finalize and free unreferenced objects on the fly. This activity will likely require more CPU time than would have been required if the program explicitly freed unnecessary memory. In addition, programmers in a garbage-collected environment have less control over the scheduling of CPU time devoted to freeing objects that are no longer needed. When does GC happen ? Garbage Collection occurs when you execute System.gc() (A hint to the JVM that it is the right time to run GC) - this is known as explicit synchronous call to GC. GC mostly occurs asynchronously when free memory available to VM gets low. -noasyncgc option was available prior to java 1.2 to force JVM to not execute asynchronous GC (it still used to execute when VM ran out of memory as opposed to when it got low on memory) On the other side - There is no guarantee as to when or if the JVM will invoke the garbage collector -- even if a program explicitly calls System.gc(). Typically, the garbage collector won't be automatically run until a program needs more memory than is currently available. At this point, the JVM will first attempt to make more memory available by invoking the garbage collector. If this attempt still doesn't free enough resources, then the JVM will obtain more memory from the operating system until it finally reaches the maximum allowed.
• #12: When evaluating a garbage collection algorithm, we might consider any or all of the following criteria: Pause time. Does the collector stop the world to perform collection? For how long? Can pauses be bounded in time? Pause predictability. Can garbage collection pauses be scheduled at times that are convenient for the user program, rather than for the garbage collector? CPU usage. What percentage of the total available CPU time is spent in garbage collection? Memory footprint. Many garbage collection algorithms require dividing the heap into separate memory spaces, some of which may be inaccessible to the user program at certain times. This means that the actual size of the heap may be several times bigger than the maximum heap residency of the user program. Virtual memory interaction. On systems with limited physical memory, a full garbage collection may fault nonresident pages into memory to examine them during the collection process. Because the cost of a page fault is high, it is desirable that a garbage collector properly manage locality of reference. Cache interaction. Even on systems where the entire heap can fit into main memory, which is true of virtually all Java applications, garbage collection will often have the effect of flushing data used by the user program out of the cache, imposing a performance cost on the user program. Effects on program locality. While some believe that the job of the garbage collector is simply to reclaim unreachable memory, others believe that the garbage collector should also attempt to improve the reference locality of the user program. Compacting and copying collectors relocate objects during collection, which has the potential to improve locality. Compiler and runtime impact. Some garbage collection algorithms require significant cooperation from the compiler or runtime environment, such as updating reference counts whenever a pointer assignment is performed. This creates both work for the compiler, which must generate these bookkeeping instructions, and overhead for the runtime environment, which must execute these additional instructions. What is the performance impact of these requirements? Does it interfere with compile-time optimizations? Very important Users have different requirements of garbage collection. For example, some consider the right metric for a web server to be throughput, since pauses during garbage collection may be tolerable, or simply obscured by network latencies. However, in an interactive graphics program even short pauses may negatively affect the user experience. Glossary Cache Line: Data is transferred between memory and cache in blocks of fixed size, called cache lines. Virtual Memory Page: Nearly all implementations of virtual memory divide a virtual address space into pages, blocks of contiguous virtual memory addresses. Pages are usually at least 4 kilobytes in size; systems with large virtual address ranges or amounts of real memory generally use larger page sizes.
• #14: None of the standard garbage collectors in the JDK uses reference counting; instead, they all use some form of tracing collector. A tracing collector stops the world (although not necessarily for the entire duration of the collection) and starts tracing objects, starting at the root set and following references until all reachable objects have been examined. Roots can be found in program registers, in local (stack-based) variables in each thread's stack, and in static variables.
• #15: Mark-sweep is simple to implement, can reclaim cyclic structures easily, and doesn't place any burden on the compiler or mutator like reference counting does. But it has deficiencies -- collection pauses can be long, and the entire heap is visited in the sweep phase, which can have very negative performance consequences on virtual memory systems where the heap may be paged. The big problem with mark-sweep is that every active (that is, allocated) object, whether reachable or not, is visited during the sweep phase. Because a significant percentage of objects are likely to be garbage, this means that the collector is spending considerable effort examining and handling garbage. Mark-sweep collectors also tend to leave the heap fragmented, which can cause locality issues and can also cause allocation failures even when sufficient free memory appears to be available.
• #16: In a copying collector, another form of tracing collector, the heap is divided into two equally sized semi-spaces, one of which contains active data and the other is unused. When the active space fills up, the world is stopped and live objects are copied from the active space into the inactive space. The roles of the spaces are then flipped, with the old inactive space becoming the new active space. Copying collection has the advantage of only visiting live objects, which means garbage objects will not be examined, nor will they need to be paged into memory or brought into the cache. The duration of collection cycles in a copying collector is driven by the number of live objects. However, copying collectors have the added cost of copying the data from one space to another, adjusting all references to point to the new copy. In particular, long-lived objects will be copied back and forth on every collection.
• #17: Note: There is something called incremental collector as well that divides the monolithic gc work into smaller discrete operations with potentially long gaps in between. It is associated with STW such that it slices up STW collected into smaller pieces. Copying collectors have another benefit, which is that the set of live objects are compacted into the bottom of the heap. This not only improves locality of reference of the user program and eliminates heap fragmentation, but also greatly reduces the cost of object allocation -- object allocation becomes a simple pointer addition on the top-of-heap pointer. There is no need to maintain free lists or look-aside lists, or perform best-fit or first-fit algorithms -- allocating N bytes is as simple as adding N to the top-of-heap pointer and returning its previous value, as suggested in Listing 1: Listing 1. Inexpensive memory allocation in a copying collector void *malloc(int n) { if (heapTop - heapStart < n) doGarbageCollection(); void *wasStart = heapStart; heapStart += n; return wasStart; } Developers who have implemented sophisticated memory management schemes for non-garbage-collected languages may be surprised at how inexpensive allocation is -- a simple pointer addition -- in a copying collector. This may be one of the reasons for the pervasive belief that object allocation is expensive -- earlier JVM implementations did not use copying collectors, and developers are still implicitly assuming allocation cost is similar to other languages, like C, when in fact it may be significantly cheaper in the Java runtime. Not only is the cost of allocation smaller, but for objects that become garbage before the next collection cycle, the deallocation cost is zero, as the garbage object will be neither visited nor copied.
• #18: Note: There is something called incremental collector as well that divides the monolithic gc work into smaller discrete operations with potentially long gaps in between. It is associated with STW such that it slices up STW collected into smaller pieces. Copying collectors have another benefit, which is that the set of live objects are compacted into the bottom of the heap. This not only improves locality of reference of the user program and eliminates heap fragmentation, but also greatly reduces the cost of object allocation -- object allocation becomes a simple pointer addition on the top-of-heap pointer. There is no need to maintain free lists or look-aside lists, or perform best-fit or first-fit algorithms -- allocating N bytes is as simple as adding N to the top-of-heap pointer and returning its previous value, as suggested in Listing 1: Listing 1. Inexpensive memory allocation in a copying collector void *malloc(int n) { if (heapTop - heapStart < n) doGarbageCollection(); void *wasStart = heapStart; heapStart += n; return wasStart; } Developers who have implemented sophisticated memory management schemes for non-garbage-collected languages may be surprised at how inexpensive allocation is -- a simple pointer addition -- in a copying collector. This may be one of the reasons for the pervasive belief that object allocation is expensive -- earlier JVM implementations did not use copying collectors, and developers are still implicitly assuming allocation cost is similar to other languages, like C, when in fact it may be significantly cheaper in the Java runtime. Not only is the cost of allocation smaller, but for objects that become garbage before the next collection cycle, the deallocation cost is zero, as the garbage object will be neither visited nor copied.
• #19: Note: There is something called incremental collector as well that divides the monolithic gc work into smaller discrete operations with potentially long gaps in between. It is associated with STW such that it slices up STW collected into smaller pieces. Copying collectors have another benefit, which is that the set of live objects are compacted into the bottom of the heap. This not only improves locality of reference of the user program and eliminates heap fragmentation, but also greatly reduces the cost of object allocation -- object allocation becomes a simple pointer addition on the top-of-heap pointer. There is no need to maintain free lists or look-aside lists, or perform best-fit or first-fit algorithms -- allocating N bytes is as simple as adding N to the top-of-heap pointer and returning its previous value, as suggested in Listing 1: Listing 1. Inexpensive memory allocation in a copying collector void *malloc(int n) { if (heapTop - heapStart < n) doGarbageCollection(); void *wasStart = heapStart; heapStart += n; return wasStart; } Developers who have implemented sophisticated memory management schemes for non-garbage-collected languages may be surprised at how inexpensive allocation is -- a simple pointer addition -- in a copying collector. This may be one of the reasons for the pervasive belief that object allocation is expensive -- earlier JVM implementations did not use copying collectors, and developers are still implicitly assuming allocation cost is similar to other languages, like C, when in fact it may be significantly cheaper in the Java runtime. Not only is the cost of allocation smaller, but for objects that become garbage before the next collection cycle, the deallocation cost is zero, as the garbage object will be neither visited nor copied.
• #23: Synchronization When a thread exists a synchronized block, it performs a write barrier - it must flush out any variables modified in the block to main memory before releasing the lock. Similarly when entering the synchronized block it performs a read barrier - it is like invalidating the local memory and fetching any variables that are referenced in the block from main memory. Atomicity is guaranteed with synchronization Source of following : Wikipedia - http://en.wikipedia.org/wiki/Race_condition Software flaws in life-critical systems can be disastrous. Race conditions were among the flaws in the Therac-25 radiation therapy machine, which led to the death of at least three patients and injuries to several more.[2] Another example is the Energy Management System provided by GE Energy and used by Ohio-based FirstEnergy Corp. (among other power facilities). A race condition existed in the alarm subsystem; when three sagging power lines were tripped simultaneously, the condition prevented alerts from being raised to the monitoring technicians, delaying their awareness of the problem. This software flaw eventually led to the North American Blackout of 2003.[3] GE Energy later developed a software patch to correct the previously undiscovered error.
• #24: Synchronization When a thread exits a synchronized block, it performs a write barrier - it must flush out any variables modified in the block to main memory before releasing the lock. Similarly when entering the synchronized block it performs a read barrier - it is like invalidating the local memory and fetching any variables that are referenced in the block from main memory. Atomicity is guaranteed with synchronization Source of following : Wikipedia - http://en.wikipedia.org/wiki/Race_condition Software flaws in life-critical systems can be disastrous. Race conditions were among the flaws in the Therac-25 radiation therapy machine, which led to the death of at least three patients and injuries to several more.[2] Another example is the Energy Management System provided by GE Energy and used by Ohio-based FirstEnergy Corp. (among other power facilities). A race condition existed in the alarm subsystem; when three sagging power lines were tripped simultaneously, the condition prevented alerts from being raised to the monitoring technicians, delaying their awareness of the problem. This software flaw eventually led to the North American Blackout of 2003.[3] GE Energy later developed a software patch to correct the previously undiscovered error.
• #25: public static void main(String[] args) throws InterruptedException { final Object o1 = new Object(); final Object o2 = new Object(); Thread t1 = new Thread() { public void run() { synchronized (o1) { { setName("t1"); } System.out .println("Thread t1 has obtained the monitor of o1"); System.out .println("Some work is being done by t1 on o1 while occupying the monitor of o1......"); try { Thread.sleep(100); } catch (InterruptedException e) { // TODO Auto-generated catch block e.printStackTrace(); } synchronized (o2) { System.out .println("Thread t1 has now taken the monitor of o2"); System.out .println("Some work is being done by t1 on o2 while occupying the monitor of o2"); } } } }; Thread t2 = new Thread() { { setName("t2"); } public void run() { synchronized (o2) { System.out .println("Thread t2 has obtained the monitor of o2"); System.out .println("Some work is being done by t2 while occupying the monitor of o2"); synchronized (o1) { System.out .println("Thread t2 has now taken the monitor of o1"); System.out .println("Some work is being done by t2 on o1 while occupying the monitor of o1"); } } } }; t1.start(); t2.start(); }
• #28: A less common situation as compared to DeadLock. Starvation happens when one thread is deprived of the resource (a shared object for instance) because other thread has occupied it for a very long time and not releasing it. LiveLock: Again a less common a situation where - Two threads are responding to each other’s action and unable to proceed.
• #29: In this unit we will discuss how to profile the java application in order to get hot spots (the methods taking more time). We will learn the techniques of identifying such hotspots using Visual VM as the tool. We will write our own code and run Visual VM method profiler as an agent over it to do the profiling.
• #30: If visualvm has problems try doing the following: Kill all java processes On command prompt echo %TMP% What ever the tmp folder is, go to that folder and search for hsperf.... Rename the folder to match the hsperf_<username> - User name is case sensitive
• #31: An inferred call from the StringBuffer to append a double, which calls String.valueOf(), which calles Double.toString(), which in turn creates a FloatingDecimal object <init> is standard way to write a constructor call; <clinit> is standard way to show a class initializer being executed. FloatingDecimal is a class that handles most of the logic involved in converting floating-point numbers. Note: The classes mentioned above describing the method call trace may be different with different vendor of JVM (Java for Mac, Sun Java, IBM Java, jrockit from BEA all may have different stack trace. However, almost all of them will show FloatDecimal as a class taking max time
• #32: Method profiling is not enough. We need to probe into the number of objects that are being churned into memory. We also need to find objects that are leaking (loitering actually) and are long lived Does java leak memory ? No and Yes. No because GC ensures that there are no objects left that are un-referenced. Yes because the application design may have a flaw which can leave (unintentionally) a reference to an object which will prevent it to be collected - Loitering Object is what it is appropriately known as. Memory leakages, especially ClassLoader Leaks can create havoc and can bring your JVM down. We will learn how to find such Memory Leaks and causes behind such leaks. We will use appropriate tools in order to do the Object Profiling.
• #36: You can prevent memory leaks by watching for some common problems. Collection classes, such as hashtables and vectors, are common places to find the cause of a memory leak. This is particularly true if the class has been declared static and exists for the life of the application. Another common problem occurs when you register a class as an event listener without bothering to unregister when the class is no longer needed. Also, many times member variables of a class that point to other classes simply need to be set to null at the appropriate time.
• #37: Even if the JDBC driver were to implement finalize, it is possible for exceptions to be thrown during finalization. The resulting behavior is that any memory associated with the now "dormant" object will not be reclaimed, as finalize is guaranteed to be invoked only once. (Though usually you may reach max connections before you hit memory limits)
• #40: Source code at : /Users/MuhammedShakir/working/CourseCaseStudies/allJavaTechnologies/performanceProject/com/mslc/training/leaks/classloaders
• #44: There is one special case that should be noted here: a program that needs to be restarted periodically in order to prevent it from crashing with an OutOfMemoryError alert. Imagine that on the previous graph the max memory size was 1100MB. If the program started with about 900MB of memory used, it would take about 48 hours to crash because it leaks about 100MB of memory per day. Similarly, if the max memory size was set to 1000MB, the program would crash every 24 hours. However, if the program was regularly restarted more often than this interval, it would appear that all is fine. Regularly scheduled restarts may appear to help, but also might make "upward sloping memory use" (as shown in the previous graph) more difficult to notice because the graph is cut short before the pattern emerges. In a case like this, you'll need to look more carefully at the memory usage, or try to increase the available memory so that it's easier to see the pattern.
• #45: Gross memory usage monitoring is important - you would not want to see your application crashing because of OOME. So, is it that max amount of memory given to JVM will serve the purpose and give good performance ? or min amount of memory will be better for your app ? These are some of the questions that we will address. In short “How much memory you really need for your app ?” is what we will discuss here. We will also take a short discussion on whether 32 bit or 64 bit - which one will give better performance - very interesting stuff
• #46: JVM allocates memory for the heap based on -Xms and -Xmx. If not specified default sizes are used (may vary based on GC algo and JVM implementation). I am using OpenJDK for Mac OS and by default it takes 75M as initial size and 1GB as max memory. JVM claims memory from the OS equal to size specified in -Xms this is total memory claimed to start with. Runtime.getRuntime().totalMemory() will return this memory as soon as you start your JVM As memory needs keeps growing, JVM claims more memory from OS - At any given point in time, Runtime.getRuntime().totalMemory() returns the currently claimed memory from OS. What ever may be the memory needs, JVM will never be able to claim memory more than what is specified by -Xmx. Runtime.getRuntime().maxMemory() returns the max memory that can be claimed from OS. It is important to note that memory needs can keep varying and may be low in non-peak hours and high in peak-hours of usage. JVM implementations allow expansion of totalMemory and contraction. When does your application encounter OOME ? When totalMemory is less than maxMemory and JVM encounters a need to claim more memory from the OS. But OS itself is unable to provide the requested memory - OOME error occurs. This happens very rarely. totalMemory reached approximately equals maxMemory and JVM needs more memory, JVM will throw OOME - thumb rule : JVM never claims memory from OS more than what is specified by -Xmx The most important point of all from performance perspective is that the max and min size must be carefully and thoughtfully set. The reason is that if there if the number of times the expansion / contraction happens is very high, then your JVM will end up getting fragmented memory. Referencing objects on fragmented memory and garbage collection will be more costly in terms of performance as compared to contiguous area of memory. Many a times you will see that for enterprise apps architects would prefer setting min size as same as max so that there is no expansion at all. Needless to say that this must be thoughtfully done. Note that Runtime.getRuntime().totalMemory does not include one of the survivor spaces, since only one can be used at any given time, and also does not include the permanent generation, which holds metadata used by the virtual machine.
• #47: In this unit we will cover how to profile threads, get the Thread Dumps and analyze the same.
• #48: There are many tools that are bundled with Sun JDK and they are as follows: jconsole: This tools provides good amount of information on the status of Heap, Perm Gen, Shared Object Memory and Garbage Collection. It also provides information on active threads and now in Java 1.6 - the threads that are in deadlock. This tool also provides interface to interact with JMX jmap (use sudo jmap on mac) : prints shared object memory maps or heap memory details of a given process. You will not need this tool separately as jconsole offers the GUI display of the same info. jstack: prints Java stack traces of Java threads for a given Java process jinfo (use sudo jinfo on mac): prints Java configuration information for a given Java process. Again you will not need this as jconsole offers this information Some important points jmap Heap Dump generation will cause your JVM to become unresponsive so please ensure that no more traffic is sent to your affected / leaking JVM before running the jmap utility Sun HotSpot 1.5/1.6/1.7 Heap Dump file will also be generated automatically as a result of a OutOfMemoryError and by adding -XX:+HeapDumpOnOutOfMemoryError in your JVM start-up arguments Note: If the given process is running on a 64-bit VM, you may need to specify the -J-d64 option, e.g.: jmap -J-d64 -heap pid
• #49: In this short session we will discuss some important factors that can lead to performance related problems in client server communication.
• #50: Once you have understood what it means by profiling the application, you need to know what to profile. In this session we will learn what to profile and why ?
• #51: In order to probe the status of various aspects of the Runtime Environment, Java 1.2 introduced JVM PI (Profiler Interface) and then JVM TI (Tooling Interface) in 1.5. We will get an overview of these APIs and learn how agents use these APIs in order to get the status of memory, cpu usage, threads and method stacks runtime. By the end of this short session you will know what it means by profiling the application. What is Observability ? After you have measured your performance you may realize that it is not as per expectations, or you may want to find which method is taking maximum time or how much time is being spent on GC, what is the total memory available and how much is used. Where would you look for all this information ? Yes - the JVM. Why ? Because JVM is the machine - (virtual of course) that manages memory, does garbage collection and compiles and gets your code executed on the physical machine. This further makes it easy to understand that JVM must be observable. If JVM is closed and not observable then how will we get the information needed in order to probe the runtime execution of our application. Here comes the point then - JVM as an API called Observability API. It is this API that exposes services which when invoked, provides information on memory, cpu usage, threads, method stacks, number of classes loaded and lot more. The vendors write their performance testing tools that make these API calls in order to get runtime information. The tool then provides the information in more easy-to-understand graphical interface for analysis. Note that profilers can also give heap dumps which can be used to further analyze the object allocation in the heap area. JVM PI The Observability API was first introduced in Java 1.2 and was known as JVM PI (Java Virtual Machine - Profiling Interface). It provided features like profiling, debugging, monitoring, thread analysis and coverage analysis. JVM TI With Java 1.5 it was replaced with JVM TI (Tooling Interface) - not just change in name but with many more advanced features that gave a very picture of the runtime behavior of the JVM. Very importantly JVM TI had very minimal load on the JVM and can be easily used in production.
• #53: Visual VM Sampler The VisualVM-Sampler plugin which is available from the Plugins Center. The plugin provides the tool with a powerful performance and memory profiler which uses sampling, a technique that allows performance and memory data to be gathered with zero setup and virtually no overhead. By periodically polling the monitored application for thread dumps or memory histograms, the profiler helps in locating potential bottlenecks or memory leaks while still allowing the application to run at full speed. It has a timer. When the timer fires, it copies the current contents of every thread's stack. Then it translates the stack frames into method and object names, and records a count against the relative methods. Because of this, it doesn't need to instrument the code, and is therefore very lightweight. However, because it doesn't instrument the code is can miss short-running stuff. So it's mostly useful either for tracking long-running performance problems, or to quickly identify a serious hot-spot in your code. During CPU profiling, the tool gets thread dumps from the monitored application by a customizable sampling rate and displays the results the same way as the built-in instrumenting Profiler. Live tabular data showing Hot Spot methods enable you to immediately detect a bottleneck in the application. You can then save the collected data into a standard .nps snapshot that provides additional Call Tree and Combined views for detailed inspection. The profiler obtains the thread dumps using a JMX connection which means that any JMX-enabled application (JDK 5+ from various vendors) can be profiled both locally and remotely. You can even profile several applications at once. From now you can use VisualVM to profile almost any Java application no matter if it's a locally running Sun JDK application or a remote IBM JDK-powered application server! Tools are an important part of java profiling. In order to be able find the bottlenecks, memory leaks, deadlocks, thread starvation etc. we need a good tool. There are many paid and free tools in the marketplace; we will discuss some of them in this unit. This is not a tool specific training program and hence only the overview of tools will be discussed. AppDynamics : Very importantly it tracks everything write from the request to the method. You need to plugin the agent using -javaagent for e.g. -javaagent:<agent_install_dir>/javaagent.jar. The most important point about this tool is that it has very low overhead and can be used in production to identify bottlenecks. Also note that you AppDynamics provides agent for IBM Java as well.
• #55: Here we will cover “What & How to Monitor GC”. We will talk about JVM Flags used for monitoring, GC log record format, Secondary Info hidden in GC logs, GC sequential overhead. In earlier session we have discussed all that we must be knowing about fundamentals of GC. Now is the time to understand the various tuning options that are available which effects GC. We will discuss many GC Tuning techniques including using parallel GC (UseParallelGC) , using concurrent collector (UseConcMarkSweepGC), XX:ParallelGCThreads etc. We will briefly revisit the discussion we did on types of collectors. Having understood about GC mechanism and the different collectors available, it is time to understand how to tune the Heap Memory Area. It is this area where all the application objects are created. Tuning the heap and GC goes hand in hand and impacts one another. In this unit we will understand how to size the different generations of Heap so that GC is efficient to its core. We will throughly look at the vm arguments used to do both - Gross Tuning (min, max sizes) and Fine Tuning (ratios between the spaces like eden, survivor etc.) of the heap.You will learn about JVM arguments like MinHeapFreeRatio, MaxHeapFreeRatio, Xmx, Xms, NewRatio, SurvivorRatio YoungGenerationSizeIncrement, TenuredGenerationSizeIncrement AdaptiveSizeDecrementScaleFactor, DefaultInitialRAMFraction, DefaultMaxRAM
• #67: If you start ConcurrentMarkSweep then you would use only ParNewGC in Young Generation. The difference between UseParallelGC and UseParNewGC is given by sun as follows: The parallel copying collector (Enabled using -XX:+UseParNewGC). Like the original copying collector, this is a stop-the-world collector. However this collector parallelizes the copying collection over multiple threads, which is more efficient than the original single-thread copying collector for multi-CPU machines (though not for single-CPU machines). This algorithm potentially speeds up young generation collection by a factor equal to the number of CPUs available, when compared to the original singly-threaded copying collector. The parallel scavenge collector (Enabled using -XX:UseParallelGC). This is like the previous parallel copying collector, but the algorithm is tuned for gigabyte heaps (over 10GB) on multi-CPU machines. This collection algorithm is designed to maximize throughput while minimizing pauses. It has an optional adaptive tuning policy which will automatically resize heap spaces. If you use this collector, you can only use the the original mark-sweep collector in the old generation (i.e. the newer old generation concurrent collector cannot work with this young generation collector).
• #70: There are two JVM flags/options that control how many GC threads are used in the JVM: ParallelGCThreads and ParallelCMSThreads. ParallelGCThreads This flag controls the number of threads used in the parallel garbage collector. This includes the young generation collector used by default. If the parallel GC is used (-XX:+UseParallelGC) or turned on by default on a 'server-class' machine, this is what you care with regard to the number of GC threads. Here's the formula that decides how many GC threads are used in the JVM on Linux/x86: ParallelGCThreads = (ncpus <= 8) ? ncpus : 3 + ((ncpus * 5) / 8) Some examples are: When ncpus=4, ParallelGCThreads=4 When ncpus=8, ParallelGCThreads=8 When ncpus=16, ParallelGCThreads=13 A rationale for the number of GC threads lower than the core count in higher core count machines, that I can think of, is that parallel GC does not scale perfectly and the extra core count didn't help or even degraded the performance. ParallelCMSThreads This flag controls the number of threads used for the CMS (concurrent mark and sweep) garbage collector (-XX:+UseConcMarkSweepGC). CMS is often used to minimize the server latency by running the old generation GC with the application threads mostly concurrently. Even when CMS is used (for the old gen heap), a parallel GC is used for the young gen heap. So, the value of ParallelGCThreads still matters. Here's how the default value of ParallelCMSThreads is computed on Linux/x86: ParallelCMSThreads = (ParallelGCThreads + 3) / 4 Some examples are: When ncpus=4, ParallelCMSThreads =1 When ncpus=8, ParallelCMSThreads =2 When ncpus=16, ParallelCMSThreads =4 Typically, when the CMS GC is active, the CMS threads occupy the cores. The rest of the cores are available for application threads. For example, on a 8 core machine, since ParallelCMSThreads is 2, the remaining 6 cores are available for application threads. (As a side note, because all the threads have the same scheduling priority at the POSIX thread level in the JVM under Linux/x86, the CMS threads may not necessarily be on cores all of the time.) Takeaways Here are the takeaways for GC tuners out there: Since ParallelCMSThreads is computed based on the value of ParallelGCThreads, overriding ParallelGCThreads when using CMS affects ParallelCMSThreads and the CMS performance. Knowing how the default values of the flags helps better tune both the parallel GC and the CMS GC. Since the Sun JVM engineers probably empirically determined the default values in certain environment, it may not necessarily be the best for your environment. If you have worked around some multithreaded CMS crash bug in older Sun JDKs by running it single-threaded (for example this one), the workaround would have caused a tremendous performance degradation on many-core machines. So, if you run newer JDK and still uses the workaround, it's time to get rid of the workaround and allow CMS to take advantage of multicores.
• #74: As said by Performance Expert Kirk Perpendine : Currently if we want a low latency collector, we must resort to using the Concurrent Mark and Sweep (CMS) collector. While this does a great job of increasing liveliness of our applications by reducing that stop-the-world nature of GC that we've all grown to know and love, CMS can still create some devastating pause times. It does this because CMS does not compact. A non-compacting collector will eventually leave your heap looking like swiss cheese and when it does, it needs to compact. If you need to compact you need to stop all application threads and perform memory to memory copying to eliminate these holes as well as free list management, which ain't cheap.
• #79: It is important to note that G1 is not a real-time collector. It meets the set pause time target with high probability but not absolute certainty. Based on data from previous collections, G1 does an estimate of how many regions can be collected within the user specified target time. Thus, the collector has a reasonably accurate model of the cost of collecting the regions, and it uses this model to determine which and how many regions to collect while staying within the pause time target
• #80: It is important to note that G1 is not a real-time collector. It meets the set pause time target with high probability but not absolute certainty. Based on data from previous collections, G1 does an estimate of how many regions can be collected within the user specified target time. Thus, the collector has a reasonably accurate model of the cost of collecting the regions, and it uses this model to determine which and how many regions to collect while staying within the pause time target
• #81: The first focus of G1 is to provide a solution for users running applications that require large heaps with limited GC latency. This means heap sizes of around 6GB or larger, and stable and predictable pause time below 0.5 seconds. Applications running today with either the CMS or the ParallelOld garbage collector would benefit switching to G1 if the application has one or more of the following traits. More than 50% of the Java heap is occupied with live data. The rate of object allocation rate or promotion varies significantly. Undesired long garbage collection or compaction pauses (longer than 0.5 to 1 second)