Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory

Dr. Low Latency or: How I Learned to Stop
Worrying about Pauses and Love the
Memory Heap

@author Jaromir Hamala, jhamala@c2b2.co.uk
@author(type=Twitter) @jerrinot

© C2B2 Consulting Limited 2013
All Rights Reserved

Who Am I?
■ Jaromir Hamala
■ A curious developer
■ Who likes to break play with stuff

All Rights Reserved

Safe Harbor
This talk may contain non-sense!

All Rights Reserved

Facts
• Memory is Cheap
• We have a lot of data
• Data in Memory == Fastest Possible Data (apart from CPU
caches)

http://www.npr.org/blogs/thetwo-way/2012/06/06/154416480/ho-hum-dull-and-boring-are-now-a-pair

All Rights Reserved

On-Heap Allocation
• CoolObject cool = new CoolObject()
• Great:
– It’s easy and usually *very* fast!
• Allocation can be actually faster than in C

– GC goodies!

• Not so great:
– GC leads to Stop-the-World Pauses
– Bigger Heap -> Longer Pauses
All Rights Reserved

Why to go Off-Heap?
• Latency caused by GC
• Locality
• Sharing with non-Java

http://digboston.com/boston-lulz/2013/08/trolley-trollop-first-comes-love/attachment/why-god-why-kitten1/

All Rights Reserved

GC Implementations
•

(Old Gen)

GC in HotSpot (Oracle/Sun JDK, OpenJDK)

– ParOld – STW pauses by design!
– CMS – fragmentation -> STW pause
– G1 – Immature (yet?), not fulfilling the expectations ->
occasional STW pauses

All Rights Reserved

Latency can kill you!

All Rights Reserved

What is Off-Heap?
• Area of memory not maintained by GC
– Manual memory control

• No Notion of Java object
– serialization

http://nickshell1983.wordpress.com/2010/06/29/taking-a-god-nudged-leap-of-faith/

All Rights Reserved

Off-Heap Options?
•
•
•
•
•

Direct Memory Buffer
sun.misc.Unsafe
JNI
Memory Mapped File
….

http://en.wikipedia.org/wiki/File:Road_to_Hell_One_Sheet.jpg

All Rights Reserved

Direct Memory Buffer
• ByteBuffer directBuf =
ByteBuffer.allocateDirect(int noBytes);

•
•
•
•

Part of Java NIO
Max. 2GB / buffer
Pure Java = Portable
getX()/putX() methods

All Rights Reserved

sun.misc.Unsafe
– long addr = unsafe.allocateMemory(noBytes)

– Not a standardized part of JDK!
– Implemented by most JVM vendors
• Sun / Oracle / IBM

– It can disappear from JDK without warning!

All Rights Reserved

Cassandra I.
public class NativeAllocator implements IAllocator
{
static final Unsafe unsafe;
static {
try {
Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
field.setAccessible(true);
unsafe = (sun.misc.Unsafe) field.get(null);
}
catch (Exception e) {
throw new AssertionError(e);
}
}
public long allocate(long size) {
return unsafe.allocateMemory(size);
}
public void free(long peer) {
unsafe.freeMemory(peer);
}
}

All Rights Reserved

JNI
• Java Native Interface
• Native library means worst portability
between platforms
• JNI Overhead
• Can re-use already existing (native) allocator

All Rights Reserved

Cassandra II.
• Java Native Interface via JNA
public class JEMallocAllocator implements IAllocator {
public interface JEMLibrary extends Library {
long malloc(long size);
void free(long pointer);
}
private final JEMLibrary library;
public JEMallocAllocator() {
library = (JEMLibrary) Native.loadLibrary("jemalloc", JEMLibrary.class);
}
public long allocate(long size) {
return library.malloc(size);
}
public void free(long peer) {
library.free(peer);
}
}
All Rights Reserved

Memory Mapped File
•
•
•
•

Part of NIO
Pure Java -> Portable
Allows sharing data between processes
Persisted by OS

All Rights Reserved

Javin Paul,
http://javarevisited.blogspot.com/2012/01/memorymapped-file-and-io-in-java.html
private static int count = 1010241024; //10 MB

public static void main(String[] args) throws Exception {
RandomAccessFile memoryMappedFile = new RandomAccessFile("largeFile.txt", "rw");
//Mapping a file into memory
MappedByteBuffer out = memoryMappedFile.getChannel().map(FileChannel.MapMode.READ_WRITE, 0,
count);
//Writing into Memory Mapped File
for (int i = 0; i < count; i++) {
out.put((byte) 'A');
}
System.out.println("Writing to Memory Mapped File is completed");

//reading from memory file in Java
for (int i = 0; i < 10 ; i++) {
System.out.print((char) out.get(i));
}
System.out.println("Reading from Memory Mapped File is completed");
}

All Rights Reserved

Use cases for Off-Heap?
• Caches / Data Grids
– A lot of data
– Objects with well defined lifecycle

• (Extremely) Latency-Sensitive tasks
– High-Frequency trading…

• Sharing Data with non-Java
All Rights Reserved

DirectMemory Usage
CacheService<Object, Object> cacheService =
new DirectMemory<Object, Object>()
.setNumberOfBuffers( 10 )
.setSize( 1000 )
.setInitialCapacity( 100000 )
.setConcurrencyLevel( 4 )
.newCacheService();

put( K key, V value )
put( K key, V value, int expiresIn )

All Rights Reserved

Final Warning(s)
• Stay On-Heap unless you have
no other choice!
• If you think you have no other
choice, think twice
• Do NOT re-invent the wheel!
(Memory Management)
• Measure, measure, measure!

All Rights Reserved

Further Sources
• http://mechanical-sympathy.blogspot.com/
– Martin Thompson. There is a brilliant mailing list as well!

• http://psy-lob-saw.blogspot.com
– Nitsan Wakart

• http://vanillajava.blogspot.com
– Peter Lawrey

• http://insightfullogic.com/blog/
– Richard Warburton

• http://ashkrit.blogspot.com
– Ashkrit

All Rights Reserved

Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory

More Related Content

Viewers also liked (20)

Similar to Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory (20)

More from C2B2 Consulting (20)

Recently uploaded (20)

Dr. Low Latency or: How I Learned to Stop Worrying about Pauses and Love the Memory