Drilling the Async Library
0 likes1,160 views
The document discusses the async library in Scala, a tool for asynchronous programming that utilizes futures and macros to simplify concurrency. It covers the definitions and uses of futures, the role of macros for compile-time reflection, and the functionality of the async and await methods for handling asynchronous operations. Additionally, it highlights the library's benefits, limitations, and code transformation processes during compilation and execution.
Drilling the Async Library
- 1. Drilling the async Library By :- SAHIL SAWHNEY Software Consultant KNOLDUS SOFTWARE LLP By :- SAHIL SAWHNEY Software Consultant KNOLDUS SOFTWARE LLP
- 2. Agenda Understanding ● Futures ● Case Study ● Macros ● Demonstration 1 ● The async library ● Demonstration 2
- 3. Prologue async library is an asynchronous programming facility for Scala that offers a direct API for working with Futures. It was added in Scala version 2.10 and is implemented using macros. Its main constructs, async and await, are inspired by similar constructs introduced in C# 5.0. async library is an asynchronous programming facility for Scala that offers a direct API for working with Futures. It was added in Scala version 2.10 and is implemented using macros. Its main constructs, async and await, are inspired by similar constructs introduced in C# 5.0.
- 4. CHAPTER - 1 Future ( The What, Why, When, … ) Future ( The What, Why, When, … )
- 5. What is a Future? A Future is a place holder (an imaginary waiting box) which holds a value that may become available at some point. Example :- Consider a web service that fetch data from a specified URL and returns a String (time consuming operation) How to deal with such a scenario? A Future is a place holder (an imaginary waiting box) which holds a value that may become available at some point. Example :- Consider a web service that fetch data from a specified URL and returns a String (time consuming operation) How to deal with such a scenario?
- 7. Why Future? ➔One of the most important trait of a programming language is to be patient for the code segments to complete their execution asynchronously with maximum possible parallelism. ➔And we use futures because they facilitates Scala with ‘the art of being patient and non blocking’. ➔One of the most important trait of a programming language is to be patient for the code segments to complete their execution asynchronously with maximum possible parallelism. ➔And we use futures because they facilitates Scala with ‘the art of being patient and non blocking’.
- 8. Asynchronous and Parallelism Asynchronous means :- “Not occurring at the same time.” - Dictionary.com "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once." - Rob Pike ➔In true parallelism you need at least two cores and the threads need to both be executing at the same time. Asynchronous means :- “Not occurring at the same time.” - Dictionary.com "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once." - Rob Pike ➔In true parallelism you need at least two cores and the threads need to both be executing at the same time.
- 9. When do we use Future? The operations which may take time (like I/O) to execute are enclosed under the Future scope which starts asynchronous computation and returns a Future value holding the result of the computation. We said that Future is an imaginary waiting box, so how to access the resultant value from this box? The operations which may take time (like I/O) to execute are enclosed under the Future scope which starts asynchronous computation and returns a Future value holding the result of the computation. We said that Future is an imaginary waiting box, so how to access the resultant value from this box?
- 10. Access result of Future
- 11. Access result of Future cont... The result (value or exception) from a Future can be accessed by following ways :- 1). Callbacks 2). Combinators 3). For Comprehensive 4). Await.result 5). The async library (async,await)
- 13. Callbacks cont.. b). onSuccess -> c). onFailure ->
- 14. Combinators Combinators like map, flatMap are used to transform a future value to a resultant future value. Combinators like map, flatMap are used to transform a future value to a resultant future value. Any problem in the above code ?
- 15. Combinators cont.. Combinators like recover, recoverWith are used to handle the exception which may occur while accessing the value of a future. Combinators like recover, recoverWith are used to handle the exception which may occur while accessing the value of a future. Applying recover combinator
- 16. For Comprehensive It is most commonly used in scenarios where we need multiple Future values to compute a result which is further a Future value. It is most commonly used in scenarios where we need multiple Future values to compute a result which is further a Future value.
- 17. Await.result Internally Await use blocking and yields the result of associated Future by blocking the current thread, thus killing the asynchronous approach to code. Hence, It Await should only be used in test cases. Internally Await use blocking and yields the result of associated Future by blocking the current thread, thus killing the asynchronous approach to code. Hence, It Await should only be used in test cases.
- 18. CHAPTER - 2 Macro ( What kind of sorcery is this? ) Macro ( What kind of sorcery is this? )
- 19. What is macro? ➔In Scala macros are a way to implement compile time reflection. ➔These are ‘special functions’ as using macros we can access the compiler API’s (using the context) which provide us with the privilege to manage the AST(Abstract Syntax Tree) generated by the compiler. ➔AST is a data structure used by the Scala compiler to store the information about the compiled code ➔In Scala macros are a way to implement compile time reflection. ➔These are ‘special functions’ as using macros we can access the compiler API’s (using the context) which provide us with the privilege to manage the AST(Abstract Syntax Tree) generated by the compiler. ➔AST is a data structure used by the Scala compiler to store the information about the compiled code
- 20. Why macro? Macros give programmer the power to :- ➔Inspecting type of an object, including generic type (Traversing, inspecting the AST) ➔Creating new objects.(Appending the AST with new child) ➔Access the member function of the object(Accessing the child nodes in AST) Macros give programmer the power to :- ➔Inspecting type of an object, including generic type (Traversing, inspecting the AST) ➔Creating new objects.(Appending the AST with new child) ➔Access the member function of the object(Accessing the child nodes in AST)
- 21. When to use macro? Macro(The reflection cousin) is used when we have to performs the following operations: ➔Introspection: Program can examine itself. ➔Intercession: Program can modify its state/meaning. Macro(The reflection cousin) is used when we have to performs the following operations: ➔Introspection: Program can examine itself. ➔Intercession: Program can modify its state/meaning.
- 22. Declaring a macro definition def add(num1:Int, num2:Int):Int = macro addImpl Here, ‘add’ is the name of the method ‘num1, num2’ is the parameter of type ’Int’ ‘Int’ is the return type of ‘add’ ‘macro’ is the keyword ‘addImpl’ is another method which provide the implementation for macro
- 23. Implementing a macro Macro implementation is a method which defines the functionality of macro. It is a bit different from the normal methods in a way that the macro implementation work on AST and is called at the compile time (by compiler) with AST of the parameters rather than the parameter itself and also returns an AST of its return value. Macro implementation is a method which defines the functionality of macro. It is a bit different from the normal methods in a way that the macro implementation work on AST and is called at the compile time (by compiler) with AST of the parameters rather than the parameter itself and also returns an AST of its return value.
- 24. Reify, Splice!!! Whats that? Macros implementation method returns an AST and this is achieved using the ‘reify’ and ‘splice’ method. a). reify() – The ‘reify’ method is in itself a macro which turns the code enclosed in its scope into its corresponding AST and type. b). splice() – The ‘splice’ method is a programmatic antonym of ‘reify’ as it turns the AST into a value with corresponding type. It can only be used inside the scope of ‘reify’. Macros implementation method returns an AST and this is achieved using the ‘reify’ and ‘splice’ method. a). reify() – The ‘reify’ method is in itself a macro which turns the code enclosed in its scope into its corresponding AST and type. b). splice() – The ‘splice’ method is a programmatic antonym of ‘reify’ as it turns the AST into a value with corresponding type. It can only be used inside the scope of ‘reify’.
- 25. The compiler magic Let us now understand the sequence of events that occur when the compiler encounters a macro. Firstly, Call to macro implementation transforms in following way :- def add(num1:Int, num2:Int):Int = macro addImpl Bold part is converted to :- addImpl(c)(AST < 2 >, AST < 1 >) Here, 2 and 1 are the parameters to macro(operands to be added) and ‘AST<2>’ = ‘Literal(Constant(2))’ Let us now understand the sequence of events that occur when the compiler encounters a macro. Firstly, Call to macro implementation transforms in following way :- def add(num1:Int, num2:Int):Int = macro addImpl Bold part is converted to :- addImpl(c)(AST < 2 >, AST < 1 >) Here, 2 and 1 are the parameters to macro(operands to be added) and ‘AST<2>’ = ‘Literal(Constant(2))’
- 26. The compiler magic cont.. Secondly, the return value of macro implementation which in itself is an AST(with a type) gets inlined to the AST of the main program and is type checked in turn. In other words, the macro declaration acts as the part of main AST to which the AST of the returned value of macro implementation is attached. Secondly, the return value of macro implementation which in itself is an AST(with a type) gets inlined to the AST of the main program and is type checked in turn. In other words, the macro declaration acts as the part of main AST to which the AST of the returned value of macro implementation is attached.
- 27. A very important note Call to a macro can not be present in same ‘.scala file’ in which the macro and its implementation are present because the .scala file containing the code for macro implementation must be compiled first. Call to a macro can not be present in same ‘.scala file’ in which the macro and its implementation are present because the .scala file containing the code for macro implementation must be compiled first.
- 28. CHAPTER - 3 Async library ( Finally :D ) Async library ( Finally :D )
- 29. What is an async Library? ➔ async library can be considered as another alternative of low-level callbacks or high order functions like map and flat-map and is used to access the result of the future. ➔ Internally the async library is implemented using callbacks, our beloved macros and needs an ExecutionContext for code execution. ➔ It has Async object with only two methods namely ‘async()’ and ‘await()’
- 30. The async method ➔Marks the beginning of the asynchronous code block ➔The execution of the code enclosed in async scope is execute on a thread from the thread pool. ➔ Inside the async scope we generally call the time consuming operations like call to a web service, fetching content of the file, fetching data from the databases like elastic search etc. ➔Marks the beginning of the asynchronous code block ➔The execution of the code enclosed in async scope is execute on a thread from the thread pool. ➔ Inside the async scope we generally call the time consuming operations like call to a web service, fetching content of the file, fetching data from the databases like elastic search etc.
- 31. The await method ➔As the name suggests, this method waits for a future to complete its execution. ➔It can only be used inside a ‘async’ block but with some limitations ➔The call to ‘await’ method are translated into the call to ‘onComplete’ of the associated Future by the async macro ➔As the name suggests, this method waits for a future to complete its execution. ➔It can only be used inside a ‘async’ block but with some limitations ➔The call to ‘await’ method are translated into the call to ‘onComplete’ of the associated Future by the async macro
- 32. Why use async library? ➔Increase the readability and simplicity of code. ➔Reduces code complexity for the programmer. ➔Increase the readability and simplicity of code. ➔Reduces code complexity for the programmer.
- 33. When to use async library? When we want to access the value of a future in a non blocking way. Since the documentation of the async library is marked with improvement process, it has quite some limitations. When we want to access the value of a future in a non blocking way. Since the documentation of the async library is marked with improvement process, it has quite some limitations.
- 34. Limitations of async library? ➔‘await’ requires a directly enclosing ‘async’ block ➔‘await’ can not be used inside a nested object, trait, or class inside ‘async’ ➔ ‘await’ must not be used inside a Boolean short- circuit argument ➔Return expressions are illegal inside an ‘async’ block. ➔‘await’ requires a directly enclosing ‘async’ block ➔‘await’ can not be used inside a nested object, trait, or class inside ‘async’ ➔ ‘await’ must not be used inside a Boolean short- circuit argument ➔Return expressions are illegal inside an ‘async’ block.
- 35. The code transformations The code inside the async scope undergoes two levels of transformation In first phase, code is normalized to a form that can further be transformed into a state machine. In second phase, normalized code is transformed into state machine The code inside the async scope undergoes two levels of transformation In first phase, code is normalized to a form that can further be transformed into a state machine. In second phase, normalized code is transformed into state machine
- 36. ‘A-Normal Form'(ANF) transformation Following changes happen at phase one ➔All control flow constructs like ‘if’ and ‘match’ are transformed from expression to statements and there results are stored in compiler fabricated variable. ➔All the calls to the ‘await’ methods are removed from the code and instead compiler fabricated variables are used to store their result. Following changes happen at phase one ➔All control flow constructs like ‘if’ and ‘match’ are transformed from expression to statements and there results are stored in compiler fabricated variable. ➔All the calls to the ‘await’ methods are removed from the code and instead compiler fabricated variables are used to store their result.
- 37. Full Async Transformation (State machine code) The code associated with the full async transformation is compiler synthesised and is not discussed in details here(litle bit of abstarction is always welcomed :p) Finally, Corresponding bytecode for the state machine code is generated at the end of the compilation process. The code associated with the full async transformation is compiler synthesised and is not discussed in details here(litle bit of abstarction is always welcomed :p) Finally, Corresponding bytecode for the state machine code is generated at the end of the compilation process.
- 38. A very important note Call to a macro can not be present in same ‘.scala file’ in which the macro and its implementation are present because the .scala file containing the code for macro implementation must be compiled first. Call to a macro can not be present in same ‘.scala file’ in which the macro and its implementation are present because the .scala file containing the code for macro implementation must be compiled first.
- 39. Summing up At COMPILE time -> The code marked under the async scope is transformed into its corresponding compiler synthesized state machine code by the ‘asyncImpl’ macro implementation method of the ‘async’ macro from which the byte-code is generated at the end of the compilation process. At COMPILE time -> The code marked under the async scope is transformed into its corresponding compiler synthesized state machine code by the ‘asyncImpl’ macro implementation method of the ‘async’ macro from which the byte-code is generated at the end of the compilation process.
- 40. Summing up cont.. At Run time -> As the byte-code corresponding to the async block is approached for its execution, a thread from the thread-pool(remember the EcecutionContext) is assigned to it and the execution takes place asynchronously. At Run time -> As the byte-code corresponding to the async block is approached for its execution, a thread from the thread-pool(remember the EcecutionContext) is assigned to it and the execution takes place asynchronously.
- 41. Important note The async scope in itself is asynchronous but the execution inside this scope is synchronous. The async scope in itself is asynchronous but the execution inside this scope is synchronous.
- 44. Any Questions?