![Capabilities to the Rescue
Effects can be modeled with capabilities
For instance, the following are equivalent:
def f(): T throws E
def f()(using CanThrow[E]): T
Effect
Capability](https://image.slidesharecdn.com/odersky-adg-v5-220711161332-10853f8f/85/Capabilities-for-Resources-and-Effects-9-320.jpg)
![Capabilities Solve Effect Polymorphism
For instance, consider map on List[A]:
def map[B](f: A => B): List[B]](https://image.slidesharecdn.com/odersky-adg-v5-220711161332-10853f8f/85/Capabilities-for-Resources-and-Effects-10-320.jpg)
![Capabilities Solve Effect Polymorphism
For instance, consider map on List[A]:
def map[B](f: A => B): List[B]
With traditional effect systems:
def map[B, E](f: A -> B eff E): List[B] eff E](https://image.slidesharecdn.com/odersky-adg-v5-220711161332-10853f8f/85/Capabilities-for-Resources-and-Effects-11-320.jpg)
![Capabilities Solve Effect Polymorphism
For instance, consider map on List[A]:
def map[B](f: A => B): List[B]
With traditional effect systems:
def map[B, E](f: A -> B eff E): List[B] eff E
With capabilities:
def map[B](f: A => B): List[B]
Here A => B is the type of impure functions that can capture
any capability as a free variable. Compare with A -> B for pure functions.](https://image.slidesharecdn.com/odersky-adg-v5-220711161332-10853f8f/85/Capabilities-for-Resources-and-Effects-12-320.jpg)
![Capabilities are Resources
A CanThrow[E] capability is generated by a try that catches E.
Example
class TooLarge extends Exception
def f(x: Int): Int throws TooLarge =
if x < limit then x * x else throw TooLarge()
val xs: List[Int] = …
try xs.map(f) // generates ct: CanThrow[TooLarge]
catch case ex: TooLarge => Nil
The capability has limited lifetime: When the try exits, the capability can no
longer be used.](https://image.slidesharecdn.com/odersky-adg-v5-220711161332-10853f8f/85/Capabilities-for-Resources-and-Effects-13-320.jpg)
![Capabilities are Resources
A CanThrow[E] capability is generated by a try that catches E.
Example
class TooLarge extends Exception
def f(x: Int)(using CanThrow[TooLarge]): Int
if x < limit then x * x else throw TooLarge()
val xs: List[Int] = …
try xs.map(f(_)(using CanThrow[TooLarge]))
catch case ex: TooLarge => Nil
The capability has limited lifetime: When the try exits, the capability can no
longer be used.](https://image.slidesharecdn.com/odersky-adg-v5-220711161332-10853f8f/85/Capabilities-for-Resources-and-Effects-14-320.jpg)
Capabilities for Resources and Effects
- 1. Caprese AdG project presentation Martin Odersky EPFL
- 2. Capabilities for Resources and Effects Two areas where static typing has lagged behind • Resources • Effects 100’s of papers over 40 years, no large scale adoption yet.
- 3. Capabilities for Resources and Effects Two areas where static typing has lagged behind • Resources • Effects 100’s of papers over 40 years, no large scale adoption yet. Core Insights • Resources and effects can be expressed as capabilities. • Retained capabilities should be be tracked in types.
- 4. Resources Values that are available only with certain restrictions, such as • lifetime • sharing • quantity Examples • regions of memory • file handles • channels • database and network connections ... Aspects of the computation beyond shapes of inputs and outputs that we want to track in types. Examples • updating variables • throwing exceptions • I/O • suspending a computation ... Effects
- 5. Resources and Effects in Programming abstraction compile-time tracking ? 20+M investment Mozillla, ERC, … ?
- 6. Resources and Effects in Programming abstraction compile-time tracking Linear Haskell Singularity Mezzo Alias types Algebraic effects 20+M investment Mozillla, ERC, … ?
- 7. Resources and Effects in Programming abstraction compile-time tracking Caprese 20+M investment Mozillla, ERC, … ?
- 8. The Effect Polymorphism Problem Unlike other types, effects are transitive: The effects of f1 include the effects of the functions it calls, transitively. How to describe the effects of f1 the call graph is dynamic? Effect polymorphism problem I/O f1 f2 … fn throw Exc x := y await f()
- 9. Capabilities to the Rescue Effects can be modeled with capabilities For instance, the following are equivalent: def f(): T throws E def f()(using CanThrow[E]): T Effect Capability
- 10. Capabilities Solve Effect Polymorphism For instance, consider map on List[A]: def map[B](f: A => B): List[B]
- 11. Capabilities Solve Effect Polymorphism For instance, consider map on List[A]: def map[B](f: A => B): List[B] With traditional effect systems: def map[B, E](f: A -> B eff E): List[B] eff E
- 12. Capabilities Solve Effect Polymorphism For instance, consider map on List[A]: def map[B](f: A => B): List[B] With traditional effect systems: def map[B, E](f: A -> B eff E): List[B] eff E With capabilities: def map[B](f: A => B): List[B] Here A => B is the type of impure functions that can capture any capability as a free variable. Compare with A -> B for pure functions.
- 13. Capabilities are Resources A CanThrow[E] capability is generated by a try that catches E. Example class TooLarge extends Exception def f(x: Int): Int throws TooLarge = if x < limit then x * x else throw TooLarge() val xs: List[Int] = … try xs.map(f) // generates ct: CanThrow[TooLarge] catch case ex: TooLarge => Nil The capability has limited lifetime: When the try exits, the capability can no longer be used.
- 14. Capabilities are Resources A CanThrow[E] capability is generated by a try that catches E. Example class TooLarge extends Exception def f(x: Int)(using CanThrow[TooLarge]): Int if x < limit then x * x else throw TooLarge() val xs: List[Int] = … try xs.map(f(_)(using CanThrow[TooLarge])) catch case ex: TooLarge => Nil The capability has limited lifetime: When the try exits, the capability can no longer be used.
- 15. Scoped Capabilities The following program still throws an unhandled exception: val it = try xs.iterator.map(f) catch case TooLarge => Iterator.empty it.next() To rule out this program statically, we need ways to enforce resource restrictions in the type system. Core idea: Track in a type which capabilities can be captured by its instances.
- 16. Capabilities and Capturing Types Definition: A capturing type {c1,...,cn}T consists of a type T and a capture set of capabilities {c1,...,cn}that tracks references retained by values of type T. Definition: A capability is a reference of a capturing type with a non-empty capture set. • Every capability gets its authority from some other, more sweeping capability which it captures. • There is a root capability from which ultimately all others are derived. Type-systematic description of the object-capability model. Details in: Odersky, Boruch Gruszecki, Lee, Brachthäuser, Lhotak: Scoped Capabilities for Polymorphic Effects, Arxiv 2207.03402
- 17. Capture Calculus: A formalization of a minimal core language with statically tracked capabilities Point of Departure
- 18. Capture Calculus: A formalization of a minimal core language with statically tracked capabilities Capture checker prototype for a slightly larger language subset Point of Departure
- 19. Capture Calculus: A formalization of a minimal core language with statically tracked capabilities Capture checker prototype for a slightly larger language subset Point of Departure Very promising in the small, but can we scale it up?
- 20. Project Structure Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency
- 21. Project Structure Challenges: Core Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency • Can we extend the theory to more language constructs that are important in practice? • Can we extend the theory to resource restrictions other than lifetime? • Can we find sound and effective inference algorithms?
- 22. Project Structure Challenges: Extensions • Is capture checking expressive enough to model a large range of effect and resource usage patterns? • Can static checking help in designing efficient and safe memory systems and concurrent runtimes? Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency
- 23. Project Structure Challenges: Infrastructure Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency • Are capture annotations lightweight enough to be used widely in practice? • Can capture checking be made efficient enough to be always on? • Can error diagnostics be made clear enough to be intelligible for non-experts? • How to migrate and interoperate with existing code?
- 24. Project Structure Challenges: Applications Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency Can we validate the applicability of the research in selected application domains? Language-based security
- 25. Project Structure Challenges: Applications Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency Can we validate the applicability of the research in selected application domains? Efficient computing
- 26. Project Structure Challenges: Applications Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency Can we validate the applicability of the research in selected application domains? Distributed systems
- 27. Project Structure Challenges: Applications Caprese Core • Foundations • Capture checking Infrastructure • Language integration • Libraries Applications • Formal methods • Efficient computing • Distributed • Security Extensions • Effect Domains • Memory Safety • Concurrency Can we validate the applicability of the research in selected application domains? Formal methods
- 28. Why Us? To succeed, the project needs • strong theoretical foundations, • solid language implementation and tooling, • a large educated user base for empirical evaluations. Scala is unique in that it combines all three aspects.
- 29. Outlook If successful, this work will solve several long-standing problems in programming: • Effect polymorphism - getting flexibility without the overhead • Mixing synchronous and asynchronous code • Combining manual allocation and automatic GC • Fearless concurrency, excluding data races and other hazards It will lead to exciting new ways to model functional and imperative programming as two ends of a spectrum. It will be the key to combining without compromises productivity, safety and performance.
Editor's Notes
- #2: Hello