GildaVM: a Non-Blocking I/O Architecture for the Cog VM
- 1. GildaVM: a Non-Blocking I/O Architecture for the Cog VM Pablo Tesone Pharo Consortium 1 Guille Polito CNRS UMR9189 CRIStAL, Inria RMoD Eliot Miranda Stellect Systems Inc David Simmons The Light Phone, USA
- 2. Blocking I/O !2 • I/O execution blocks the interpreter • While in a I/O call the interpreter is blocked • E.g., System-calls, FFI
- 3. FFI? Foreign Function Interface Image VM External Libraries FFI We can communicate with anything that has a C API Operating System API !3
- 4. Unified FFI in a nutshell UFFI handles: - Look-up of functions - Marshalling of arguments - Execution - Marshalling of the return values !4
- 5. Concurrency in Pharo !5 P1 P2 P3 Interpreter Thread p1 p3 p1 p2 p2
- 6. Concurrency in Pharo !6 P1 P2 P3 Interpreter Thread p1 p3 p1 p2 p2 Interpreter handles process scheduling
- 7. Concurrency in Pharo !7 P1 P2 P3 Interpreter Thread p1 p3 p1 p2 p2 p1 int function(char* foo, int bar)
- 8. Concurrency in Pharo !8 P1 P2 P3 Interpreter Thread p1 p3 p1 p2 p2 p1 Out of Interpreter Interpreter loses control int function(char* foo, int bar)
- 9. What we want! int function(char* foo, int bar) !9 P1 P2 Interpreter #1 Interpreter #2
- 10. What we want! int function(char* foo, int bar) !10 P1 P2 Interpreter #1 Interpreter #2 • Requires extensive modification of VM, Plugins and Image core libraries • Applications should be written with threading in mind • Real multithreading not only for FFI
- 11. Proposal: Global Interpreter Lock VM int function(char* foo, int bar) P1 P2 Interpreter #1 Interpreter #2 !11
- 12. Research Questions • RQ1: How does scheduling work in presence of processes and native threads? • RQ2: What is the overhead of thread switching? !12
- 13. Process scheduling with many VM threads !13 P1 P2 P3 VM Thread #1 p1 p1 p2 VM Thread #2 p3 p2 One VM Thread owns the VM at each time
- 14. Process Affinity !14 p3 bindToThreadId: 2 • Explicit binding • When a process is activated, it is run in the affined thread • Or in the same thread if not affined VM Thread #1 p1 p1 p2 VM Thread #2 p3 p2 Disown VM Own VM
- 15. Non-blocking FFI !15 • Before FFI calls the current thread disowns the VM • Another thread owns the VM • Non-blocked processes are scheduled VM Thread #1 p1 p1 p2 VM Thread #2 p2 Disown VM Own VM
- 16. Short callouts? !16 • If naive, each disown creates a lot of overhead! VM Thread #1 p1 p1 p2 VM Thread #2 p2 Disown VM Own VM Own VM Disown VM p1 p2 Disown VM Own VM Own VM Disown VM
- 17. sleep verify VM Thread #1 p1 p1 p2 VM Thread #2 Disown VM verify Watchdog sleep Own VM verify sleep vm thread can continue! p1 Watchdog Native Thread !17 • A watchdog periodically verifies if the VM is busy • If idle, selects a thread with work to do and activate it
- 18. sleep verify VM Thread #1 p1 p1 p2 VM Thread #2 Disown VM verify Watchdog sleep Own VM verify sleep vm thread can continue! p1 Short calls !18 • The watchdog sleeping window defines the “length” of the short call
- 19. sleep verify VM Thread #1 p1 p1 p2 VM Thread #2 Disown VM verify Watchdog sleep verify sleep Own VM p2 Long call preemption !19 • The watchdog sleeping window also defines the max “length” of idle-ness
- 20. Process switch without affinity 50 iterations, mean showed !20
- 21. Long I/Os sequencial concurrent processes 2 one-sec callouts 50 iterations, mean showed !21
- 22. Short calls sequencial concurrent processes 100,000 short callouts 50 iterations, mean showed !22
- 23. • Callbacks • Reentrant callbacks • • More on preemption • Implementation details • Also in the paper… !23
- 24. sleep verify VM Thread #1 p1 p1 p2 VM Thread #2 Disown VM verify Watchdog sleep Own VM verify sleep vm thread can continue! p1 Future Work #1: watchdog impact • If the watchdog window is not aligned with the FFI calls, short callouts are recognised as long ones (false positives) • Long watchdog window will recognise long calls as short calls and be blocking (false negatives) !24
- 25. Future Work #2: thread management • Should VM threads be created implicitly or explicitly? • Should the thread pool be size-bound? Analyse strategies for particular applications. !25
- 26. Conclusion • A Global interpreter lock architecture for green-threaded smalltalk implementations • Good for parallelising long blocking I/O • Some strategies to reduce the overhead of thread switch Pablo Tesone Pharo Consortium Guille Polito CNRS UMR9189 CRIStAL, Inria RMoD Eliot Miranda Stellect Systems Inc David Simmons The Light Phone, USA