Lambda: A Peek Under The Hood - Brian Goetz

Lambda: A peek under the
hood Insert Picture Here

Brian Goetz
Java Language Architect, Oracle


The following is intended to outline our general product
direction. It is intended for information purposes only,
and may not be incorporated into any contract.
It is not a commitment to deliver any material, code, or
functionality, and should not be relied upon in making
purchasing decisions. The development, release, and
timing of any features or functionality described for
Oracle’s products remains at the sole discretion of
Oracle.


About this talk

 This talk outlines the technical details of how lambda expressions are
implemented in Java SE 8 at the bytecode level
– This will be a highly technical talk!
– We assume some familiarity with JVM and bytecode
– Including JSR-292 facilities


Lambda expressions for Java

 A lambda expression is like an “anonymous method”
– Has an argument list, a return type, and a body
– people.forEach((Person e) ->
names.add(e.getName());
– Many of these can be inferred by the compiler
– people.forEach(e -> names.add(e.getName());

 Lambda expressions can capture values from the enclosing context
– people.removeAll(e -> e.getAge() < minAge);


Why lambdas for Java?

 Provide libraries a path to multicore
– Parallel-friendly APIs need internal iteration
– Internal iteration needs a concise code-as-data mechanism
 Empower library developers
– Enable a higher degree of cooperation between libraries and client code
 It’s about time!
– Java is the lone holdout among mainstream OO languages at this point to not have closures
– Adding closures to Java is no longer a radical idea
 Inner classes give us some of these benefits, but are too clunky
– Clunkiness is not entirely syntactic!
 How to represent lambda expressions at runtime is not a trivial question
– That’s what this talk is about


Lambda expressions for Java

 Big question #1: what is the type of a lambda expression?
– Most languages with lambdas have some notion of a function type
– Java has no concept of function type

– JVM has no native (unerased) representation of function type in VM type signatures

 Adding function types would create many questions
– How do we represent functions in VM type signatures?
– How do we create instances of function-typed variables?
– How do we deal with variance?
 Want to avoid significant VM changes


Functional interfaces

 We have historically modeled functions using single-method interfaces, such
as Runnable or Comparator
 Rather than complicate the type system, let’s just formalize that
– Give them a name: “functional interfaces”
– Always convert lambda expressions to instance of a functional interface

interface Predicate<T> { boolean test(T x); }
people.removeAll(p -> p.getAge() >= 18);

– Compiler figures out the types – lambda is converted to Predicate<Person>
 How does the lambda instance get created?


Why not “just” use inner classes?

 We could say that a lambda is “just” an inner class instance
 Then for the lambda p -> p.getAge() >= minAge we generate
class Foo$1 implements Predicate<Person> {
private final int $v0;
Foo$1(int v0) { this.$v0 = v0; }
public boolean test(Person p) {
return p.age < $v0;
}
}

 This means (among other things) one class per lambda expression



 If we translate lambdas to inner classes, then capturing a lambda is
invoking a constructor
list.removeAll(p -> p.getAge() >= minAge);

list.removeAll(new Foo$1(minAge));



 Inner class constructor invocation translates as

aload_1 // list
new #2 // class Foo$1
dup
iload_2 // minAge
invokespecial #3 // Method Foo$1."<init>":(I)V
invokeinterface #4 // java/util/List.removeAll:(Ljava/util/functions/Predicate;)Z



 Translating to inner classes means we inherit most of their problems
– Performance issues
– One class per lambda expression

– Type profile pollution

– Complicated “comb” lookup
 Whatever we do becomes a binary representation for lambdas in Java
– Would be stuck with it forever
– Would rather not conflate implementation with binary representation


Bytecode invocation modes

 Prior to Java SE 7, the JVM had four bytecodes for method
invocation
– invokestatic: for static methods
– invokevirtual: for class methods
– invokeinterface: for interface methods
– invokespecial: for constructors, private methods, and super-calls
 aload_1 specifies a// list name, method name, and method signature
Each class
new #2 // class Foo$1
dup
iload_2 // minAge
invokespecial #3 // Method Foo$1."<init>":(I)V
invokeinterface #4 // java/util/List.removeAll:(Ljava/util/functions/Predicate;)Z

New bytecode tool: MethodHandle

 Java SE 7 adds VM-level method handles
– Can store references to methods in the constant pool and load with LDC
– Can obtain a method handle for any method (or field access)
– VM will happily inline through MH calls
– MH API contains combinators for manipulating method handles
– Add, remove, reorder arguments

– Adapt (box, unbox, cast) arguments and return types

– Compose methods

– Compiler writers swiss-army knife!


Why not “just” use MethodHandle?

 At first, translating lambdas to MethodHandle seems obvious
– Lambda is language-level method object
– MethodHandle is VM-level method object
 We could
– Desugar lambdas expressions to methods
– Represent lambdas using MethodHandle in bytecode signatures



 If we represented lambdas as MethodHandle, we’d translate:

MethodHandle mh = LDC[lambda$1];
mh = MethodHandles.insertArguments(mh, 0, minAge);
list.removeAll(mh);

private static boolean lambda$1(int capturedK, Person p) {
return p.getAge() >= capturedK;
}



 If we did this, the signature of List.removeAll would be:
void removeAll(MethodHandle predicate)
 This is erasure on steroids!
– Can’t overload two methods that take differently “shaped” lambdas
– Still would need to encode the erased type information somewhere
 Also: is MH invocation performance yet competitive with bytecode
invocation?
 Again: conflates binary representation with implementation


Stepping back…

 We would like a binary interface not tied to a specific implementation
– Inner classes have too much baggage
– MethodHandle is too low-level, is erased
– Can’t force users to recompile, ever, so have to pick now
 What we need is … another level of indirection
– Let the static compiler emit a declarative recipe, rather than imperative code, for
creating a lambda
– Let the runtime execute that recipe however it deems best
– And make it darned fast
– This sounds like a job for invokedynamic!


New bytecode tool: invokedynamic

 Java SE 7 adds a fifth invocation mode: invokedynamic (indy)
 Behavior of invoke{virtual,static,interface} are fixed and Java-like
– Other languages need custom method linkage
 Basic idea: let some “language logic” determine call target
– And then get out of the way
– Language and VM become partners
in flexible and efficient method
dispatch



 The canonical use case for invokedynamic is something like
def add(a, b) { a+b }
 Here, the types of a and b are not known at compile time
– And can change from call to call … but probably don’t so often
– Good chance that if called with two ints, next call will be with two ints
– We win by not having to re-link the call site for every invocation



 The first time the JVM executes an invokedynamic
– Consults the bootstrap method for the call site (the “language logic”)
– Bootstrap returns a linked call site
– Call site can embed conditions under which it needs relinking
– Such as the argument types changing

– Otherwise, JVM does not have to consult bootstrap again

 After linkage, JVM can inline through linked indy callsites



 An indy callsite has three groups of operands
– A bootstrap method (the “language logic”)
– Called by the VM for linking the callsite on first invocation

– Not called again after that

– A static argument list, embedded in the constant pool
– Available to the bootstrap method

– A dynamic argument list, like any other method invocation
– Not available to the bootstrap, but their static types and arity are

– Passed to whatever method the callsite is linked to


Its not just for dynamic languages anymore

 So, if indy is for dynamic languages, why is the Java compiler using it?
– All the types involved are static
– What is dynamic here is the code generation strategy
– Generate inner classes?

– Use method handles?

– Use dynamic proxies?

– Use VM-private APIs for constructing objects?

 Indy lets us turn this choice into a pure implementation detail
– Separate from the binary representation


Its not just for dynamic languages anymore

 We use indy to embed a recipe for constructing a lambda at the capture
site, including
– The desugared implementation method
– The functional interface we are converting to
– Values captured from the lexical scope
– Additional metadata, such as serialization information
 The capture site is called the lambda factory
– Invoked with indy, returns an instance of the desired functional interface
– Subsequent captures bypass the (slow) linkage path


Desugaring lambdas to methods

 First, we desugar the lambda to a method
– Signature matches functional interface method
– Plus captured arguments prepended (must be effectively final)
– Simplest lambdas desugar to static methods
– But some need access to receiver, and so are instance methods
Predicate<Person> olderThanK = p -> p.getAge() >= k;

private static boolean lambda$1(int capturedK, Person p) {
}

Factories and metafactories

 We generate an indy call site which, when called, returns the lambda
– This is the lambda factory
– Bootstrap for the lambda factory list.removeAll(p -> p.getAge() >= minAge);
selects the translation strategy
– Bootstrap is called the lambda metafactory
Predicate $p = indy[bootstrap=LambdaMetafactory,
– Part of Java runtime staticargs=[Predicate, lambda$1],
dynargs=[minAge])
– Captured args passed list.removeAll($p);

to lambda factory private static boolean lambda$1(int capturedK, Person p) {
}


Translation strategies

 The metafactory could spin inner classes dynamically
– Generate the same class the compiler would, just at runtime
– Link factory call site to constructor of generated class
– Conveniently, dynamic args and ctor args will line up

– Our initial strategy until we can prove that there’s a better one
 Alternately could spin one wrapper class per interface
– Constructor would take a method handle
– Methods would invoke that method handle
 Could also use dynamic proxies or MethodHandleProxy
 Or VM-private APIs to build object from scratch, or…


Indy: the ultimate lazy initialization

 For stateless (non-capturing) lambdas, we can create one single instance of
the lambda object and always return that
– Very common case – many lambdas capture nothing
– People sometimes do this by hand in source code – e.g., pulling a Comparator into
a static final variable
 Indy functions as a lazily initialized cache
– Defers initialization cost to first use
– No overhead if lambda is never used
– No extra field or static initializer
– All stateless lambdas get lazy init and caching for free


Indy: the ultimate procrastination aid

 By deferring the code generation choice to runtime, it becomes a
pure implementation detail
– Can be changed dynamically
– We can settle on a binary protocol now (metafactory API) while delaying
the choice of code generation strategy
– Moving more work from static compiler to runtime

– Can change code generation strategy across VM versions, or even days
of the week


Indy: the ultimate indirection aid

 Just because we defer code generation strategy to runtime, we don’t
have to pay the price on every call
– Metafactory only invoked once per call site
– For non-capturing case, subsequent captures are FREE
– VM optimizes to constant load

– For capturing case, subsequent capture cost on order of a constructor
call / method handle manipulation
– MF links to constructor for generated class


Performance costs

 Any translation scheme imposes costs at several levels:
– Linkage cost – one-time cost of setting up capture
– Capture cost – cost of creating a lambda
– Invocation cost – cost of invoking the lambda method
 For inner class instances, these correspond to:
– Linkage: loading the class
– Capture: invoking the constructor
– Invocation: invokeinterface


Performance example – capture cost

 Oracle Performance Team measured capture costs
– 4 socket x 10 core x 2 thread Nehalem EX server
– All numbers in ops/uSec
 Worst-case lambda numbers equal to inner classes
Single-threaded Saturated
– Best-case numbers much better Scalability

–
Inner class And this is 160 our “fallback” strategy
just 1407 8.8x
Non-capturing lambda 636 23201 36.4x

Capturing lambda 160 1400 8.8x


Not just for the Java Language!

 The lambda conversion metafactories will be part of java.lang.invoke
– Semantics tailored to Java language needs
– But, other languages may find it useful too!
 Java APIs will be full of functional interfaces
– Collection.forEach(Block)
 Other languages probably will want to call these APIs
– Maybe using their own closures
– Will want a similar conversion
 Since metafactories are likely to receive future VM optimization attention, using
platform runtime is likely to be faster than spinning your own inner classes


Possible VM support

 VM could intrinsify lambda capture sites
– Capture semantics are straightforward properties of method handles
– Capture operation is pure, therefore freely reorderable
– Can use code motion to delay/eliminate captures
 Lambda capture is like a “boxing” operation
– Essentially boxing a method handle into lambda object
– Invocation is the corresponding “unbox”
– Can use box elimination techniques to eliminate capture overhead
– Intrinsification of capture + inline + escape analysis


Serialization

 No language feature is complete without some interaction with serialization 
– Users will expect this code to work

interface Foo extends Serializable {
public boolean eval();
}
Foo f = () -> false;
// now serialize f

 We can’t just serialize the lambda object
– Implementing class won’t exist at deserialization time
– Deserializing VM may use a different translation strategy
– Need a dynamic serialization strategy too!
– Without exposing security holes…


Serialization

 Just as our classfile representation for a lambda is a recipe, our serialized
representation needs to be to
– We can use readResolve / writeReplace
– Instead of serializing lambda directly, serialize the recipe (say, to some well defined
interface SerializedLambda)
– This means that for serializable lambdas, MF must provide a way of getting at the recipe
– We provide an alternate MF bootstrap for that
 On deserialization, reconstitute from recipe
– Using then-current translation strategy, which might be different from the one that originally
created the lambda
– Without opening new security holes


Summary

 We use invokedynamic to capture lambda expressions
– Gives us flexibility and performance
– Free to change translation strategy at runtime
 Even using the “dumb” translation strategy…
– No worse than inner classes in the worst case
– 5-20x better than inner classes in a common case


For more information

 Project Lambda: http://openjdk.java.net/projects/lambda/
– Lambda spec EDR #2: http://
jcp.org/aboutJava/communityprocess/edr/jsr335/index2.html
– Lambda Overview: http://cr.openjdk.java.net/~
briangoetz/lambda/lambda-state-4.html
– Binary builds: http://jdk8.java.net/lambda/


Graphic Section Divider


Lambda: A Peek Under The Hood - Brian Goetz

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Lambda: A Peek Under The Hood - Brian Goetz (20)

More from JAX London (20)

Recently uploaded (20)

Lambda: A Peek Under The Hood - Brian Goetz