Scale Up with Lock-Free Algorithms @ JavaOne

Scale Up with
Lock-Free Algorithms
Non-blocking concurrency on JVM
Presented at JavaOne 2017
/Roman Elizarov @ JetBrains

Speaker: Roman Elizarov
• 16+ years experience
• Previously developed high-perf trading software
@ Devexperts
• Teach concurrent & distributed programming
@ St. Petersburg ITMO University
• Chief judge
@ Northern Eurasia Contest / ACM ICPC
• Now work on Kotlin
@ JetBrains

Data 1 Data 2 Data N
map map map

map map map
reduce
answer

Embarrassingly parallel
map map map
reduce
answer

Big Big Data
Real-time
Concurrent requests / processing

Big Big Data
Real-time
Concurrent requests / processing
Performance Scalability

A toy problem – stack
class Node<T>(val next: Node<T>?, val value: T)

public final class Node<T> {
private final Node<T> next;
private final T value;
public Node(Node<T> next, T value) {
this.next = next;
this.value = value;
}
public Node<T> getNext() {
return next;
}
public T getValue() {
return value;
}
}

A toy problem – empty stack
top

A toy problem – stack push
next = null
value = 1
top А

next = null
value = 1
top А
next = A
value = 2
B

next = A
value = 2
top
next = null
value = 1
B A

class LinkedStack<T> {
private var top: Node<T>? = null
fun push(value: T) {
top = Node(top, value)
}
}

next = A
value = 2
top
next = null
value = 1
B A

A toy problem – stack pop
next = A
value = 2
top
next = null
value = 1
B A
cur

next = A
value = 2
top
next = null
value = 1
B A
cur
result = 2

}
fun pop(): T? {
val cur = top ?: return null
top = cur.next
return cur.value
}
}

}
fun pop(): T? {
top = cur.next
return cur.value
}
}

A toy problem – concurrent push
next = null
value = 1
top А
next = A
value = 2
B

A toy problem – concurrent push
next = null
value = 1
top А
next = A
value = 2
B
next = A
value = 3
C

A toy problem – synchronized stack
@Synchronized
}
@Synchronized
fun pop(): T? {
top = cur.next
return cur.value
}
}

Benchmark
@State(Scope.Benchmark)
open class LinkedStackBenchmark {
private val stack = LinkedStack<Int>()
@Benchmark
fun benchmark() {
stack.push(1)
check(stack.pop() == 1)
}
}

Benchmark results
0
5
10
15
20
25
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LinkedStack
Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz; 32 HW threads; Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)

Contention
P
Q work
pop1
pop2
wait

Lock-free push
next = null
value = 1
top А
next = A
value = 2
B
expect

Lock-free push
next = null
value = 1
top А
next = A
value = 2
B
update

Lock-free push
next = null
value = 1
top А
next = A
value = 2
B
update
expect

AtomicReference
package java.util.concurrent.atomic;
/** @since 1.5 */
public class AtomicReference<V> {
private volatile V value;
public V get() {
return value;
}
public boolean compareAndSet(V expect, V update) {
// …
}
}

Using AtomicReference
class LockFree<T> {
private val top = AtomicReference<Node<T>?>(null)
while (true) {
val cur = top.get()
val upd = Node(cur, value)
if (top.compareAndSet(cur, upd)) return
}
}
}

Using AtomicReference - push
class LockFree<T> {
while (true) {
val cur = top.get()
}
}
}
1

class LockFree<T> {
while (true) {
val cur = top.get()
}
}
}
1
2

class LockFree<T> {
while (true) {
val cur = top.get()
}
}
}
1
2
3

1
2
3
class LockFree<T> {
while (true) {
val cur = top.get()
}
}
}

Using AtomicReference - pop
class LockFree<T> {
fun push(value: T) { … }
fun pop(): T? {
while (true) {
val cur = top.get() ?: return null
if (top.compareAndSet(cur, cur.next)) return cur.value
}
}
}

class LockFree<T> {
fun pop(): T? { … }
}

Using volatile variable
class LinkedStackLF<T> {
@Volatile
// ...
}
}

Using AtomicReferenceFieldUpdater
package java.util.concurrent.atomic;
/** @since 1.5 */
public abstract class AtomicReferenceFieldUpdater<T,V> {
public static <U,W> AtomicReferenceFieldUpdater<U,W> newUpdater(
Class<U> tclass, Class<W> vclass, String fieldName;
public abstract boolean compareAndSet(T obj, V expect, V update;
}

private volatile Node<T> top;

private static final AtomicReferenceFieldUpdater<LockFree, Node>
TOP = AtomicReferenceFieldUpdater
.newUpdater(LockFree.class, Node.class, "top");

private static final AtomicReferenceFieldUpdater<LockFree, Node>
TOP = AtomicReferenceFieldUpdater
.newUpdater(LockFree.class, Node.class, "top");
if (TOP.compareAndSet(this, cur, upd)) return;

Using VarHandle
package java.lang.invoke;
/** @since 9 */
public abstract class VarHandle {
@MethodHandle.PolymorphicSignature
public native boolean compareAndSet(Object... args);
}

Using VarHandle
private static final VarHandle TOP;
static {
try {
TOP = MethodHandles.lookup()
.findVarHandle(LockFree.class, "top", Node.class);
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new InternalError(e);
}
}

Using VarHandle
private static final VarHandle TOP;
static {
try {
TOP = MethodHandles.lookup()
.findVarHandle(LockFree.class, "top", Node.class);
} catch (NoSuchFieldException | IllegalAccessException e) {
throw new InternalError(e);
}
}
if (TOP.compareAndSet(this, cur, upd) return;

Using AtomicFU J
private val top = atomic<Node<T>?>(null)

Using AtomicFU J

Using AtomicFU J
Code like
AtomicReference

Using AtomicFU J
Bytecode
Code like
AtomicReference
compile

Using AtomicFU J
Bytecode
Code like
AtomicReference
AtomicReferenceFUcompile atomicFU

Using AtomicFU J
Bytecode
Code like
AtomicReference
VarHandlecompile atomicFU

Benchmark results
0
5
10
15
20
25
30
35
40
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LockFree
LinkedStack

0
5
10
15
20
25
30
35
40
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LockFree
LinkedStack
Benchmark results
Yeh!
Nay…

Contention
P
Q retry
pop1
pop2
try update

Too toy of a problem?
@Synchronized
@Synchronized
fun pop(): T? {
top = cur.next
return cur.value
}
}

Too toy of a problem – make it more real?
@Synchronized
@Synchronized
fun pop(): T? {
top = cur.next
Blackhole.consumeCPU(100L)
return cur.value
}
}

Too toy of a problem – make it more real?
class LockFree<T> {
fun pop(): T? {
while (true) {
val cur = top.value ?: return null
if (top.compareAndSet(cur, cur.next)) return cur.value
}
}
}

Benchmark results
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LockFree
LinkedStack

Workload
@Benchmark
fun benchmark() {
stack.push(1)
}
}

Read-dominated workload
@Benchmark
fun benchmarkReadDominated() {
stack.push(1)
repeat(10) { check(stack.peek() == 1) }
}
}

@Benchmark
stack.push(1)
}
}
@Synchronized
fun peek() = top?.value
}

open class LockFreeBenchmark {
private val stack = LockFree<Int>()
@Benchmark
stack.push(1)
}
}
class LockFree<T> {
fun peek() = top.value?.value
}

Benchmark results – x10 reads
0
5
10
15
20
25
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LockFree
LinkedStack

Benchmark results – x100 reads
0
1
2
3
4
5
6
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LockFree
LinkedStack

Real-world workload
@Benchmark
fun benchmarkReadWorld() {
stack.push(1)
repeat(10) {
check(stack.peek() == 1)
}
}

Benchmark results – real world
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
1.8
1 2 4 8 16 32 64 128
Millions
Number of threads
Throughput (ops/s)
LockFree
LinkedStack

Learn to ask the right questions
You shall, young Padawan.

Links
• JMH http://openjdk.java.net/projects/code-tools/jmh/
• Kotlin https://kotlinlang.org
• AtomicFU https://github.com/Kotlin/kotlinx.atomicfu

Thank you
Any questions?
Slides are available at www.slideshare.net/elizarov
email me to elizarov at gmail
relizarov

A toy problem – concurrent pop
next = A
value = 2
top
next = null
value = 1
B A
cur1 cur2

A toy problem – concurrent pop
next = A
value = 2
top
next = null
value = 1
B A
result1 = 2
cur1 cur2
result2 = 2

Scale Up with Lock-Free Algorithms @ JavaOne

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to Scale Up with Lock-Free Algorithms @ JavaOne (20)

More from Roman Elizarov (20)

Recently uploaded (20)

Scale Up with Lock-Free Algorithms @ JavaOne