![Let’s Instrument the Compiler
prf := PrfMethodProfiler new.
prf addPackage: OpalCompiler package.
prf addPackage: RBParser package.
prf profile: [ Integer recompile ].
29](https://image.slidesharecdn.com/08-codeinstrumentationandthepitfallsofabstraction-230913135949-27b00a48/85/Instrumentation-the-Pitfalls-of-Abstraction-29-320.jpg)
![^ self < 2
ifTrue: [1]
ifFalse: [
(self-1) benchFib + (self-2) benchFib + 1]](https://image.slidesharecdn.com/08-codeinstrumentationandthepitfallsofabstraction-230913135949-27b00a48/85/Instrumentation-the-Pitfalls-of-Abstraction-32-320.jpg)
![Profiling the Compiler — again
• Partial Instrumentation
• Down from ~110x to ~12x
44
Baseline
Generic+Online
Customized+Delayed
0.0
20.0
40.0
60.0
80.0
100.0
120.0
140.0
Time
(relative
to
no
instrumentation)
prf := PrfMethodProfiler new.
prf addPackage: OpalCompiler package.
prf addPackage: RBParser package.
prf profile: [ Integer recompile ].](https://image.slidesharecdn.com/08-codeinstrumentationandthepitfallsofabstraction-230913135949-27b00a48/85/Instrumentation-the-Pitfalls-of-Abstraction-44-320.jpg)
Instrumentation & the Pitfalls of Abstraction
- 1. Instrumentation & the Pitfalls of Abstraction ESUG - 2023 - Lyon * Supported by AlaMVic Action Exploratoire INRIA guillermo.polito@inria.fr @guillep Evref fervE 1 Generated by DALL-E
- 2. First: About Me 2 Evref fervE • Keywords: compilers, testing, test generation • Ph.D.: Re fl ection, debloating, dynamic updates • Interests: tooling, benchmarking, 日本語, board games, concurrency Talk to me! Or: guillermo.polito@inria.fr guillermo.polito@inria.fr @guillep
- 3. Building Dynamic Analyses • Dynamic call graphs • Code coverage • Pro fi lers • Time • Number of calls 3
- 4. Method Wrappers, Objects as Methods 4
- 5. Remember Method Lookup sender receiver class s1 s2 … s4 … m1 m2 m3 s2 superclass s1 … s5 … inherits from instance of reference message 5
- 6. sender receiver class s1 s2 … s4 … m1 m3 s2 superclass s1 … s5 … Objects as methods o1 inherits from instance of reference message 6
- 7. sender receiver class s1 s2 … s4 … m1 m3 s2 superclass s1 … s5 … Objects as methods + run:with:in: o1 run:with:in: inherits from instance of reference message 7
- 8. How far can we get with run:with:in: ? 8
- 9. A First Method Proxy run: aSelector with: anArrayOfObjects in: aReceiver | result | self logBefore: aSelector. result := self forwardMethod: originalMethod withReceiver: aReceiver withArguments: anArrayOfObjects. self logAfter: aSelector. ^ result 9
- 10. Let’s instrument factorial doit 17 px log: fwd… factorial run:with:in: 10
- 11. doit 17 Let’s instrument factorial px log: fwd… run:with:in: factorial 11
- 12. Let’s instrument factorial doit 17 px log: fwd… run:with:in: factorial 16 factorial … 12
- 13. Let’s instrument factorial doit 17 px log: fwd… run:with:in: factorial 16 px log: fwd… run:with:in: factorial … 13
- 14. Let’s get a bit more hardcore 14
- 18. Meta-Recursions doit aSet add: px run:with:in: log: otherSet add: run:with:in: • The instrumentation gets instrumented! • And, with more complex instrumentation, more di ffi cult to debug • The burden: on the developer 18
- 20. Solving Meta-Recursions doit aSet add: px run:with:in: log: otherSet add: Instrumentation zone 20
- 21. Solving Meta-Recursions doit aSet add: px run:with:in: log: otherSet add: Instrumentation zone 21
- 22. And That’s not All • Stack unwind (non-local returns, exceptions) pass around the logAfter: • Concurrent access to our instrumentation zone? • lose logs • break the instrumentation • Maybe we can do some concessions: e.g., do not proxy the proxy… This burden, is on the developer 22
- 23. The Cost of Missing Abstraction • The language gives us only low-level instrumentation hooks • #run:with:in: • #doesNotUnderstand: • #cannotInterpret: • i.e., they are at the wrong level of abstraction for proper instrumentation Covering the GAP, is on the developer 23
- 24. The Proxy We Have doit aSet add: px run:with:in: log: otherSet add: Instrumentation zone 24
- 25. The Stratified Proxy We Want doit aSet add: px run:with:in: log: otherSet add: Instrumentation zone handler before: after: Infrastructure • Meta-recursion • Concurrency User concern • logging? • analysis? 25
- 27. Safe Method Proxies + Exact Method Profiler • Method Proxies: https://github.com/pharo-contributions/MethodProxies • Method Pro fi ler: https://github.com/pharo-contributions/MethodPro fi ler • + common instrumentation layer between proxies and meta-links ! 27
- 28. Let’s get a bit more hardcore again 28
- 29. Let’s Instrument the Compiler prf := PrfMethodProfiler new. prf addPackage: OpalCompiler package. prf addPackage: RBParser package. prf profile: [ Integer recompile ]. 29
- 30. prf := PrfMethodProfiler new.
- 32. ^ self < 2 ifTrue: [1] ifFalse: [ (self-1) benchFib + (self-2) benchFib + 1]
- 33. Let’s Benchmark with Fibonacci • Best case for proxy infrastructure • no exceptions • no non-local return • no meta-recursion • no concurrent usages by default fi b n - 1 fi b n - 2 fi b n - 3 fi b n - 4 fi b n - 4 fi b n - 5 fi b n - 3 fi b n - 4 fi b n - 5
- 34. Let’s Benchmark with Fibonacci (II) • Good case to measure pro fi ler/proxy overhead • Simulate a big call-tree • Leaves are fast paths (early exits) • => high overhead expected • fib(n) ~~ number of messages fi b n - 1 fi b n - 2 fi b n - 3 fi b n - 4 fi b n - 4 fi b n - 5 fi b n - 3 fi b n - 4 fi b n - 5
- 35. Our Lower Bound is run:with:in: 35 run: aSelector with: anArrayOfObjects in: aReceiver ^ self forwardMethod: originalMethod withReceiver: aReceiver withArguments: anArrayOfObjects
- 36. run:with:in: Performance vs fib(x) • ~25x slower ! • Seems faster for lower args • Noise due to µs measures? 36 0 10 20 30 0 5 10 15 20 25 Time (relative to no proxy) Fibonacci Argument * Averages of 100 runs in µs. X = 1 to: 28
- 37. run:with:in: Performance vs Messages • Consistent ~25x overhead • Cries for language implementation improvement (!!) 37 * Averages of 100 runs in µs. 0 300000 600000 900000 1200000 0 5 10 15 20 25 Time (relative to no proxy) Messages
- 38. The Cost of Safety • Safe method proxies are ~3000x worse • Non-clean closures • allocation • thisContext rei fi cation • More messages (!) • #ensure: • meta-recursion control • #before, #after hooks 38 0 300000 600000 900000 1200000 0 1000 2000 3000 4000 Time (relative to no proxy) Messages * Averages of 100 runs in µs.
- 39. Can we get better? • Down to ~400x just removing abstraction • Inlinings (!!) • to remove messages • to avoid blocks • di ff erentiate fast vs slow path • concurrent, meta-recursive 39 0 300000 600000 900000 1200000 0 500 1000 1500 2000 2500 3000 3500 Time (relative to no proxy) Messages * Averages of 100 runs in µs.
- 40. Overhead of Call-Tree Construction • 2 proxy variants • Generic: handler object • Customized: inlined handler • 2 instrumentation variants • Online: build the call tree while executing • Delayed: trace the minimum to build it in a post-process • Comparison Baseline: safe+opt proxy with no instrumentation 40 * Averages of 100 runs in µs.
- 41. Call-tree construction • Generic + Online was o ff the charts :) • => o ff the presentation too • Delaying the analysis is the best • Customization gets only slightly better • removes 4 messages per call 41 * Averages of 100 runs in µs. 0 300000 600000 900000 1200000 0 2 4 6 8 10 12 Time (relative to Generic Online Analysis) Messages Generic - No Instrumentation Customized Proxy - Online Analysis Generic Proxy - Delayed Analysis Customized Proxy - Delayed Analysis
- 42. Call-tree construction • Generic + Online was o ff the charts :) • => o ff the presentation too • Delaying the analysis is the best • Customization gets only slightly better • removes 4 messages per call 42 * Averages of 100 runs in µs. 0 300000 600000 900000 1200000 0 2 4 6 8 10 12 Time (relative to Generic Online Analysis) Messages Generic - No Instrumentation Customized Proxy - Online Analysis Generic Proxy - Delayed Analysis Customized Proxy - Delayed Analysis
- 43. Zooming in • Delayed is ~1.25x proxy alone ~1.25 * 400x (safety) ~= ~500x overhead (over no instrumentation) 43 * Averages of 100 runs in µs. 0 300000 600000 900000 1200000 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 Time (relative to Generic Online Analysis) Messages Generic - No Instrumentation Generic Proxy - Delayed Analysis Customized Proxy - Delayed Analysis
- 44. Profiling the Compiler — again • Partial Instrumentation • Down from ~110x to ~12x 44 Baseline Generic+Online Customized+Delayed 0.0 20.0 40.0 60.0 80.0 100.0 120.0 140.0 Time (relative to no instrumentation) prf := PrfMethodProfiler new. prf addPackage: OpalCompiler package. prf addPackage: RBParser package. prf profile: [ Integer recompile ].
- 45. Takeaways • Users need native language support for instrumentation • Safe, strati fi ed and **e ffi cient** • Low-level hooks are not enough: they miss abstractions • Think twice when writing your own proxy implementation! • Think concurrency, think stack unwind, thing meta-recursions 45 * Supported by AlaMVic Action Exploratoire INRIA Evref fervE https://github.com/pharo-contributions/MethodProxies https://github.com/pharo-contributions/MethodPro fi ler