Kotlin Coroutines Reloaded

Kotlin Coroutines Reloaded
Presented at JVM Language Summit, 2017
/Roman Elizarov @ JetBrains

Speaker: Roman Elizarov
• 16+ years experience
• Previously developed high-perf trading software
@ Devexperts
• Teach concurrent & distributed programming
@ St. Petersburg ITMO University
• Chief judge
@ Northeastern European Region of ACM ICPC
• Now work on Kotlin
@ JetBrains

Agenda
• Recap of Kotlin coroutines prototype
• Presented @ JVMLS, 2016 by Andrey Breslav
• The issues in the prototype design
• and the solutions that were found
• Design released to public in Kotlin 1.1
• Evolved coroutines support libraries
• Challenges & future directions

Recap of Kotlin coroutines
prototype
Presented @ JVMLS, 2016 by Andrey Breslav

Async the C# (JS, TS, Dart, etc) way
async Task<String> Work() { … }
async Task MoreWork()
{
Console.WriteLine("Work started");
var str = await Work();
Console.WriteLine($"Work completed {str}");
}

Async the C# (JS, TS, Dart, etc) way
async Task<String> work() { … }
async Task MoreWork()
{
Console.WriteLine("Work started");
var str = await Work();
Console.WriteLine($"Work completed {str}");
}

Async the Kotlin way (prototype)
fun work(): CompletableFuture<String> { … }
fun moreWork() = async {
println("Work started")
val str = await(work())
println("Work completed: $str")
}

Async the Kotlin way (prototype)
fun work(): CompletableFuture<String> { … }
}
Functions vs Keywords
Extensibility
Runs on stock JVM 1.6+
Purely local -- no global
bytecode transforms

Suspending functions
A grand unification between async/await and generate/yield

Suspending functions: use

Suspending functions: use
CompletableFuture<String>
// String result

Suspending functions: declare (prototype)
suspend fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Unit
// String result

callback void
// String result
Magic signature transformation

callback void
// String result
Continuation is a generic callback interface
interface Continuation<in T> {
fun resume(value: T)
fun resumeWithException(exception: Throwable)
}

Suspending functions: implement (prototype)
suspend fun <T> await(f: CompletableFuture<T>, c: Continuation<T>) {
f.whenComplete { value, exception ->
if (exception != null) c.resumeWithException(exception)
else c.resume(value!!)
}
}
// String result

Suspending functions: implement (prototype)
}
}
// String result
Simple, but wrong!

A problematic example
Where grand vision meets reality

}
}
fun problem() = async {
repeat(10_000) {
await(work())
}
}

}
}
repeat(10_000) {
await(work())
}
}
What if work() always returns a
future that is already complete?

}
}
repeat(10_000) {
await(work())
}
}
stack
problem$stateMachine

}
}
repeat(10_000) {
await(work())
}
}
stack
await

}
}
repeat(10_000) {
await(work())
}
}
stack
await
CompletableFuture.whenComplete

}
}
repeat(10_000) {
await(work())
}
}
stack
await
await$lambda

}
}
repeat(10_000) {
await(work())
}
}
stack
await
await$lambda
ContinuationImpl.resume

}
}
repeat(10_000) {
await(work())
}
}
stack
await
await$lambda

}
}
repeat(10_000) {
await(work())
}
}
stack
await
await$lambda
await

}
}
repeat(10_000) {
await(work())
}
}
stack
await
await$lambda
await
…
StackOverflowError

A solution
A difference between knowing the path & walking the path.

A solution
// String result

A solution (0): stack unwind convention
suspend fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Any?
// String result

A solution (0)
// String result
T | COROUTINE_SUSPENDED

A solution (0)
// String result
Did not suspend -> Returns result

A solution (0)
// String result
Did not suspend -> Returns result
Did suspend -> WILL invoke continuation

A solution (0)
suspend fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Any? {
…
}

A solution?
val consensus = AtomicReference<Any?>(UNDECIDED)
if (exception != null) {
if (!consensus.compareAndSet(UNDECIDED, Fail(exception)))
c.resumeWithException(exception)
} else {
if (!consensus.compareAndSet(UNDECIDED, value))
c.resume(value!!)
}
}
consensus.compareAndSet(UNDECIDED, COROUTINE_SUSPENDED)
val result = consensus.get()
if (result is Fail) throw result.exception
return result
}

A solution?
val consensus = AtomicReference<Any?>(UNDECIDED)
if (exception != null) {
if (!consensus.compareAndSet(UNDECIDED, Fail(exception)))
c.resumeWithException(exception)
} else {
if (!consensus.compareAndSet(UNDECIDED, value))
c.resume(value!!)
}
}
consensus.compareAndSet(UNDECIDED, COROUTINE_SUSPENDED)
val result = consensus.get()
if (result is Fail) throw result.exception
return result
}
Wat?

A solution (1): Call/declaration fidelity

A solution (1)

A solution (1)
suspend fun <T> await(f: CompletableFuture<T>): T
natural signature

A solution (1)
fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Any?
Compiles as (on JVM)

A solution (1)
CPS Transformation

A solution (1)
suspend fun <T> await(f: CompletableFuture<T>)
suspend fun <T> await(f: CompletableFuture<T>): T =
suspendCoroutineOrReturn { c ->
…
}
Recover continuation

A solution (1)
…
}
Inspired by call/cc from Scheme
Recover continuation

A solution (1)
…
}
Works as
Inspired by call/cc from Scheme

A solution (1)
…
}

A solution (1)
…
}
inline suspend fun <T> suspendCoroutineOrReturn(
crossinline block: (Continuation<T>) -> Any?): T

A solution (1)
…
}

A solution (1)
…
}
Intrinsic

A solution (1)
…
}
fun <T> suspendCoroutineOrReturn(
crossinline block: (Continuation<T>) -> Any?,
c: Continuation<T>): Any? =
block(c)
Intrinsic
CPS Transformation

A solution (2): Tail suspension

A solution (2)
…
}
Tail suspend invocation:
Continuation pass-through

A solution (2)
…
}
fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Any? =
suspendCoroutineOrReturn(c) { c ->
…
}
CPS Transformation

A solution (2)
…
}
fun <T> await(f: CompletableFuture<T>, c: Continuation<T>): Any? {
…
}
CPS Transformation

A solution (3)
public inline suspend fun <T> suspendCoroutine(
crossinline block: (Continuation<T>) -> Unit): T =
suspendCoroutineOrReturn { c: Continuation<T> ->
val safe = SafeContinuation(c)
block(safe)
safe.getResult()
}

A solution (3)
block(safe)
safe.getResult()
}
Any? Is gone

A solution (3)
block(safe)
safe.getResult()
}
Encapsulates result
consensus algorithm

A solution (4): Putting it all together

A solution (4)
suspendCoroutine { c ->
}
}
Looks simple, works correctly!

Recap steps to solution
• T | COROUTINE_SUSPENDED (Any?) to allow invocations that
do not suspend and thus avoid StackOverflowError
• Call/declaration fidelity via CPS transformation
• Introduce call/cc (suspendCoroutineOrReturn) to recover
hidden continuation
• Tail call invocation support to recover prototype semantics
• Use abstraction (suspendCoroutine) to hide implementation
complexities from end-users

The final touch: await extension
}
}

suspend fun <T> CompletableFuture<T>.await(): T =
whenComplete { value, exception ->
}
}

}
}
}

}
}
val str = work().await()
}
Reads left-to-right just like it executes

Coroutine builders
The mystery of inception

Coroutine builders (prototype)
fun <T> async(
coroutine c: FutureController<T>.() -> Continuation<Unit>
): CompletableFuture<T> {
val controller = FutureController<T>()
c(controller).resume(Unit)
return controller.future
}

fun <T> async(
}
A special modifier

fun <T> async(
}
Magic signature transformation

fun <T> async(
}
A boilerplate to start coroutine

fun <T> async(
): CompletableFuture<T> { … }
class FutureController<T> {
val future = CompletableFuture<T>()
operator fun handleResult(value: T, c: Continuation<Nothing>) {
future.complete(value)
}
operator fun handleException(exception: Throwable,
c: Continuation<Nothing>) {
future.completeExceptionally(exception)
}
}

fun <T> async(
operator fun handleResult(value: T, c: Continuation<Nothing>) {
}
operator fun handleException(exception: Throwable,
c: Continuation<Nothing>) {
}
}
was never used

fun <T> async(
operator fun handleResult(value: T) {
}
operator fun handleException(exception: Throwable) {
}
}

fun <T> async(
}
}
}
Looks like Continuation<T>

fun <T> async(
}
}
}
Something that takes Continuation<T>

Coroutine builders
fun <T> async(
c: suspend () -> T
c.startCoroutine(completion = controller)
}

Coroutine builders
fun <T> async(
c: suspend () -> T
}
natural signature
No special coroutine keyword

Coroutine builders
fun <T> async(
c: suspend () -> T
}
Provided by standard library

Coroutine builders
fun <T> async(
c: suspend () -> T
class FutureController<T> : Continuation<T> {
override fun resume(value: T) {
}
override fun resumeWithException(exception: Throwable) {
}
}

Coroutine builders
fun <T> async(
c: suspend () -> T
class FutureController<T> : Continuation<T> {
override fun resume(value: T) {
}
override fun resumeWithException(exception: Throwable) {
}
}
Serves as completion
continuation for coroutine

Bonus features
Free as in cheese

Arbitrary suspending calls
suspend fun <T> suspending(block: suspend () -> T): T =
suspendCoroutine { continuation ->
block.startCoroutine(completion = continuation)
}

}
}
Returns CompletableFuture

}
suspend fun moreWork(): T = suspending {
}

}
suspend fun moreWork(): T = suspending {
}
Tail suspend invocation
Non-tail (arbitrary) suspend invocation

suspend fun moreWork() {
}
Non-tail (arbitrary) suspend invocation

Stackless vs Stackful coroutines
The crucial distinction

Stackless Stackful
Restrictions Use in special ctx Use anywhere
Implemented in C#, Scala, Kotlin, … Quasar, Javaflow, …

Stackless Stackful
Implemented in C#, Scala, Kotlin, … Quasar, Javaflow, …
Can suspend in? suspend
functions
throws SuspendExecution /
@Suspendable functions

Stackless Stackful
Implemented in C#, Scala, … Kotlin, Quasar, Javaflow, …

Stackless Stackful
Implemented in C#, Scala, Kotlin,
Quasar, JavaFlow, …
LISP, Go, …

Stackless Stackful
Implemented in C#, Scala, … Kotlin, Quasar, Javaflow, …
False
dichotomy

Async vs Suspending functions
The actual difference

fun work() = async { … }
suspend fun work() { … }

fun work(): CompletableFuture<String> = async { … }
suspend fun work(): String { … }
In Kotlin you have a choice

fun workAsync(): CompletableFuture<String> = async { … }
suspend fun work(): String { … }

workAsync() VALID –> produces CompletableFuture<String>
workAsync().await() VALID –> produces String
concurrent & async behavior
sequential behavior
The most needed one, yet the most syntactically
clumsy, esp. for suspend-heavy (CSP) code style
The problem with async

Kotlin suspending functions
imitate sequential behavior
by default
Concurrency is hard
Concurrency has to be explicit

Composability
Making those legos click

Builders
fun <T> async(
c: suspend () -> T

Builders
fun <T> async(
c: suspend () -> T
): ListenableFuture<T> { … }

Builders
fun <T> async(
c: suspend () -> T
): MyOwnFuture<T> { … }

fun <T> async(
c: suspend () -> T
suspend fun <T> CompletableFuture<T>.await(): T { … }

fun <T> async(
c: suspend () -> T
suspend fun <T> ListenableFuture<T>.await(): T { … }

fun <T> async(
c: suspend () -> T
suspend fun <T> MyOwnFuture<T>.await(): T { … }

fun <T> async(
c: suspend () -> T
suspend fun <T> MyOwnFuture<T>.await(): T { … }
fun moreWorkAsync() = async {
val str = workAsync().await()
}
All combinations need to compose

Composability: evolved
• Kotlin suspending functions are composable by default
• Asynchronous use-case
• Define asynchronous suspending functions anywhere
• Use them inside any asynchronous coroutine builder
• Or inside other suspending functions
• generate/yield coroutines are synchronous
• Restricted via a special @RestrictsSuspension annotation
• Opt-in to define synchronous coroutines

Coroutine context
The last piece of composability puzzle

asyncUI (prototype)
asyncUI {
val image = await(loadImage(url))
myUI.updateImage(image)
}
Supposed to work in UI thread

asyncUI (prototype)
asyncUI {
}
A special coroutine builder

asyncUI (prototype)
asyncUI {
}
A special suspending function in its scope
Composability problem
again!

asyncUI: evolved
async(UI) {
val image = loadImageAsync(url).await()
}

asyncUI: evolved
async(UI) {
}
Explicit context convention for all builders
Can intercept continuation to
resume in the appropriate thread

The actual Continuation interface
async(UI) {
}
val context: CoroutineContext
}
Is used to transparently lookup an
interceptor for resumes

The actual Continuation interface
async(UI) {
}
val context: CoroutineContext
}
Does not have to be aware – receives
intercepted continuation to work with

Thread-safety
A million-dollar quest for correctly-synchronized (data-race free) code

A need for happens-before relation
val list = ArrayList<String>()
list.add(str)
}

list.add(str)
}
One thread

list.add(str)
}
One thread
Suspends / resumes

list.add(str)
}
One thread
Suspends / resumes
Another thread

list.add(str)
}
One thread
Suspends / resumes
Another thread
Is there a data race?
Do we need volatile when spilling
locals to a state machine?

list.add(str)
}
There is no data race here!
await establishes happens-before relation
happens-before

Challenges
If it only was all that simple…

Thread confinement vs coroutines
synchronized(monitor) {
}
}

synchronized(monitor) {
}
}
MONITORENTER in one thread
Suspends / resumes
MONITOREXIT in another thread
IllegalMonitorStateException

• Monitors
• Locks (j.u.c.l.ReentrantLock)
• Thread-locals
• …
• Thread.currentThread()
A CoroutineContext is a map of
coroutine-local elements as replacement

Stack traces in exceptions
work()
}
suspend fun work() {
someAsyncOp().await()
throw Exception()
}
stack
moreWork

work()
}
throw Exception()
}
stack
moreWork
work

work()
}
throw Exception()
}
stack
moreWork
work
await

work()
}
throw Exception()
}
stack
moreWork
work
Work$StateMachine :
Continuation
----------
fun resume(v)

work()
}
throw Exception()
}
stack
Work$StateMachine.resume
work

work()
}
throw Exception()
}
JVM stack
work

work()
}
throw Exception()
}
JVM stack
work
Coroutine stack
moreWork
work

work()
}
throw Exception()
}
JVM stack
work
Coroutine stack
moreWork
work
Gets thrown

work()
}
throw Exception()
}
JVM stack
work
Coroutine stack
moreWork
work
Gets thrown User wants

Library evolution
Going beyond the language & stdlib

Library evolution: already there
• Communication & synchronization primitives
• Job: A completable & cancellable entity to represent a coroutine
• Deferred: A future with suspend fun await (and no thread-blocking methods)
• Mutex: with suspend fun lock
• Channel: SendChannel & ReceiveChannel
• RendezvousChannel – synchronous rendezvous
• ArrayChannel – fixed size buffer
• LinkedListChannel – unlimited buffer
• ConflatedChannel – only the most recent sent element is kept
• BroadcastChannel: multiple subscribes, all receive

• Coroutine builders
• launch (fire & forget, returns a Job)
• async (returns Deferred)
• future (integration with CompletableFuture & ListenableFuture)
• runBlocking (to explicitly delimit code that blocks a thread)
• actor / produce (consume & produce messages over channels)
• publish (integration with reactive streams, returns Publisher)
• rxCompletable, rxSingle, rxObservable, rxFlowable (RxJava 1/2)

• Top-level functions
• select expression to await multiple events for full CSP-style programming
• delay for time-based logic in coroutines
• Extensions
• await for all kinds of futures (JDK, Guava, RxJava)
• aRead, aWrite, etc for AsynchronousXxxChannel in NIO
• Cancellation & job (coroutine/actor) hierarchies
• withTimeout for a composable way for any suspend function run w/timeout

Library evolution: WIP
• Serialization / migration of coroutines
• Migrating libraries to Kotlin/JS & Kotlin/Native (lang support – done)
• Full scope of pipelining operators on channels (filter, map, etc)
• Optimizations for single-producer and/or single-consumer cases
• Allow 3rd party primitives to participate in select (alternatives)
• ByteChannel for suspendable IO (http client/server, websocket, etc)
• See also ktor.io

A closing note on terminology
• We don’t use the term fiber/strand/green threads/…
• The term coroutine works just as well
• ”Fibers describe essentially the same concept as coroutines” © Wikipedia
Kotlin Coroutines are very light-weight threads

Experimental status of coroutines
• This design is new and unlike mainstream
• For some very good reasons
• We want community to try it for real
• So we released it as an experimental feature
• We guarantee backwards compatibility
• Old code compiled with coroutines continues to work
• We reserve the right to break forward compatibility
• We may add things so new code may not run w/old RT
• Design will be finalized at a later point
• Old code will continue to work via support library
• Migration aids to the final design will be provided
opt-in flag

Thank you
Any questions?
Slides are available at www.slideshare.net/elizarov
email elizarov at gmail
relizarov

Kotlin Coroutines Reloaded

More Related Content

What's hot (20)

Similar to Kotlin Coroutines Reloaded (20)

More from Roman Elizarov (19)

Recently uploaded (20)

Kotlin Coroutines Reloaded