ClassLoader Leaks

Mattias Jiderhamn – java.jiderhamn.se
Join the war on
Mattias Jiderhamn
ClassLoader leaks

java.lang.OutOfMemoryError:
PermGen space
:-(
Metaspace

Local dev env

Continuous Deploy
.war
Crashing

Agenda
What does it mean?
Why does it happen?
Where does it leak?
How to avoid?
OSS examples
Training!

What?

JVM memory
Stack
Heap
PermGen /
Metaspace

JVM memory
Per thread
Local variables and
method parameters
Stack

JVM memory
Heap
Young generation
Old generation /
tenured space
Eden
space
Survivor
space
Object instances

JVM memory
Permanent Generation
java.lang.Class instances
etc
Named before JEE and
class unloading
Renamed Metaspace in Java 8
Aka Method area
PermGen /
Metaspace

PermGen space / Metaspace
= Too many classes are loaded
What?

Why?

Reason for OOME
1. Your application is too large
-XX:MaxPermSize=256M
Java 8: Metaspace auto increase by default
2. java.lang.Class instances could not
be garbage collected after redeploy
PermGen space / Metaspace

Reference types
• Strong (i.e. normal) reference
–Never GC:ed if reachable
• Soft reference
–GC:ed before OutOfMemoryError
• Weak reference (WeakHashMap)
–GC:ed when no Strong or Soft refs
• Phantom reference
–… won’t prevent GC

Example
Foo
WeakHashMap
Map m = new WeakHashMap();
Foo myFoo = new Foo();
m.put(myFoo, ”bar”);
myFoo = null;

GC reachability
GC roots

Redeploy
Application Server
ClassLoader
app.war
ClassLoader
app.war’
ClassLoader
app.war’’

ClassLoader refs
ClassLoader
Class ClassClass
Instance

How leaks happen
Application Server
ClassLoader
Class ClassClass
Instance
GC root

Where?

Application Server
• Application Server bugs
• Logging frameworks
– Apache Commons Logging
Unless LogFactory.release()
– Log4j - some configs
Unless LogManager.shutdown()
– java.util.logging custom Level
• Bean Validation API (JSR 303)
• Unified Expression Language (javax.el)
?

GC roots
• Class loaded by system ClassLoader
– static field in JDK classes (java.* etc)
• Live thread
– Stack – local vars, method params
–java.lang.Thread instance
• Object held as synchronization monitor
• JNI references
• JVM specials…

System classes
• java.sql.DriverManager
• Bean introspection cache, shutdown hooks,
custom default Authenticator, custom
security Provider, custom MBeans, custom
ThreadGroup, custom property editor, …
• Reference to contextClassLoader of first caller

DriverManager

DriverManager
Application Server
app.war
mysql-jdbc.jar
JRE
DriverManager
ClassLoader
com.mysql.jdbc.Driver
1) …
… or 2)
DriverManager.deregisterDriver(…)

Context shutdown
public class MyCleanupListener implements
javax.servlet.ServletContextListener {
...
/** Called when application is undeployed */
public void contextDestroyed(
ServletContextEvent servletContextEvent) {
DriverManager.deregisterDriver(…);
}
}

Threads
• Thread stack
– Local variables
– Method parameters
• MyThread extends Thread
MyRunnable implements Runnable
• contextClassLoader +
AccessControlContext inherited
– Example HTTP 1.1 Keep-Alive-Timer

Context shutdown
public class MyCleanupListener implements
javax.servlet.ServletContextListener {
...
/** Called when application is undeployed */
public void contextDestroyed(
ServletContextEvent servletContextEvent) {
DriverManager.deregisterDriver(…);
// Stop threads here!
}
}

Stopping threads
public class MyThread extends Thread {
public void run() {
while(true) { // Bad idea!
// Do something
}
}
}

Stopping threads
public class MyThread extends Thread {
private boolean running = true;
public void run() {
while(running) { // Until stopped
// Do something
}
}
public void shutdown() {
running = false;
}
}
private volatile boolean running = true;
Heinz Kabutz / Java Specialists’ - The Law of the Blind Spot

Threads
• Thread stack
– Local variables
– Method parameters
• MyThread extends Thread
MyRunnable implements Runnable
• contextClassLoader +
AccessControlContext inherited
– Example HTTP 1.1 Keep-Alive-Timer
• ThreadLocal

ThreadLocal
WeakHashMap<Thread, ?>
ThreadLocal
Thread Foo

ThreadLocal
ThreadLocal JavaDoc:
”Each thread holds an implicit
reference to its copy of a thread-
local variable as long as the thread
is alive and the ThreadLocal
instance is accessible; …”

ThreadLocal
ThreadLocalMap
Entry
Thread
ThreadLocal Foo
put()
set()

ThreadLocal
Pooled threads:
•Threads may outlive ClassLoader
•ThreadLocal → ThreadGlobal!
(?)

ThreadLocal
ThreadLocalMap
Entry
Thread
ThreadLocal Foo
ClassClassLoader
static
Class

ThreadLocal
ThreadLocalMap
Entry
Thread
ThreadLocal Foo
ClassClassLoader
Stale entry

ThreadLocal
ThreadLocalMap JavaDoc:
”However, since reference queues
are not used, stale entries are
guaranteed to be removed only
when the table starts running out
of space”

ThreadLocal
•Custom value (incl references)
–static ThreadLocal = leak
–otherwise = unpredicted GC
•Custom ThreadLocal = no leak

ThreadLocal
try {
myThreadLocal.set(foo);
…
}
finally {
myThreadLocal.remove();
}
Always clear your ThreadLocals!

Known offenders
Apache ActiveMQ
Apache Axis
Apache Batik
Apache Commons Pool / DBCP
Apache CXF
AWT Toolkit
Bean Validation API / JSR 303
CGLIB (Hibernate / Spring / JBoss /
Apache Geronimo)
dom4j
EclipseLink
GeoTools
Google Guice
Groovy
GWT / javax.imageio
Hessian
iCal4J
Infinispan
IntrospectionUtils
JarURLConnection
Java Advanced Imaging (JAI)
Javassist
Java Cryptography Architecture (JCA)
Java Server Faces 2
javax.security.auth.Policy/login.Configuration
JGroups
Logback
JAXB
Mozilla Rhino
MVEL
OpenOffice Java Uno RunTime (JURT)
Oracle JDBC
RMI
Serialization (<= JDK 1.4.2)
Spring framework
Unified Expression Language / javax.el
URLConnection + HTTP 1.1
XML parsing (DocumentBuilderFactory)

How?

Leak Prevention Lib
• Application server independent
– Version 2 core module is pure Java SE!
• Covers a lot more than Tomcat
– System class references avoided/cleared
– Threads are stopped
– ThreadLocals are cleared
– Known offenders handled
• Logs warnings
• Apache 2 licensed
– You may modify and redistribute

Leak Prevention Lib
Zero runtime dependencies
No required config
Maven pom.xml (Servlet 3.0+):
<dependency>
<groupId>se.jiderhamn.classloader-leak-prevention</groupId>
<artifactId>classloader-leak-prevention-servlet3</artifactId>
<version>2.0.2</version>
</dependency>

Tomcat
Bugzilla #48895
ThreadLocalMap
Thread
Cleanup
Thread
remove()
Unsafe!
get()
set()
remove()
Removed in 6.0.27
Tomcat 7.0.6+ renews the thread pool
whenever an application is redeployed.
Disposable threads for lifecycle events.
Bugzilla #49159

Leak Prevention Lib
set(null) ThreadLocalMap
Entry
Thread
ThreadLocal Foo
Stale
Cleanup
Thread

Open Source
examples

Bean Validation API
Version 1.0.0.GA
javax.validation.Validation
Application Server
validation-api-1.0.0.GA.jar
app.war
hibernate-validator.jar

javax.el API
javax.el.BeanELResolver

Apache CXF
org.apache.cxf.transport.http.CXFAuthenticator
CXF-5442

OpenOffice JURT
com.sun.star.lib.util.AsynchronousFinalizer

Training!
Gear up!

Free tool
Eclipse Memory Analyzer
aka MAT
eclipse.org/mat

Heap dump
Run application server with
-XX:+HeapDumpOnOutOfMemoryError
-XX:HeapDumpPath=/temp

Links
Join the fight!
java.jiderhamn.se @mjiderhamn
github.com/mjiderhamn
• ClassLoader Leak Prevention Library
• Step by step guide: acquire heap
dump and analyze for leaks using
Eclipse Memory Analyzer (MAT)
• List of known offenders and links to bug reports
• Submit reports and pull requests!
???

ClassLoader Leaks

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to ClassLoader Leaks (20)

Recently uploaded (20)

ClassLoader Leaks

Editor's Notes