Hijacking Ruby Syntax in Ruby (RubyConf 2018)
1 like11,092 views
The document outlines a presentation on advanced Ruby features and techniques by @joker1007 and @tagomoris during RubyConf 2018. Key topics discussed include Ruby binding, tracepoint, method hooks, and various hacks such as method modifiers and resource management enhancements. The presentation emphasizes the creative use of these features to extend Ruby's functionality while cautioning about the complexities and potential pitfalls of these 'magic' techniques.
![Tracepoint in overrider, abstriker
TracePoint Hook
Overrider
self.override_methods = [
:method_a,
:method_b,
]
▸ Use Method#super_method method to check method
existence (ex. method(:method_a).super_method)](https://image.slidesharecdn.com/rubyconf2018hijackingrubysyntax-181115220158/85/Hijacking-Ruby-Syntax-in-Ruby-RubyConf-2018-33-320.jpg)
Hijacking Ruby Syntax in Ruby (RubyConf 2018)
- 1. HIJACKING RUBY SYNTAX IN RUBY RubyConf 2018 LA 2018/11/15 (Thu) @joker1007 & @tagomoris
- 2. Self-Intro Joker ▸ id: joker1007 ▸ Repro inc. CTO ▸ I’m familiar with Ruby/Rails/Fluentd/ECS/Presto. ▸ This talk is my first speech abroad!!
- 3. Satoshi Tagomori (@tagomoris) Fluentd, MessagePack-Ruby, Norikra, Woothee, ... Arm Ltd.
- 5. Agenda ▸ Ruby Features ▸ binding ▸ Tracepoint ▸ method hook ▸ Hacks ▸ finalist, overrider, abstriker ▸ with_resources, deferral
- 6. Ruby Features: Binding Binding ▸ Context object, includes: ▸ Kernel#binding receiver ▸ local variables ▸ For template engines (?) ▸ Methods: ▸ #receiver, #eval, #local_variables, #local_variable_get, #local_variable_defined?, #local_variable_set
- 7. Binding creates new object per call
- 8. Setting variables are ignored on binding
- 9. Overwriting existing variable is effective!
- 10. Ruby Features: Binding Binding#local_variable_set ▸ Method to ▸ add a variable only in a binding instance ▸ overwrite values of existing variables in original context
- 11. Ruby Features: TracePoint TracePoint ▸ Tracing events in VM ▸ and call hook block ▸ For various events: ▸ :line , :raise ▸ :class , :end ▸ :call , :return , :c_call , :c_return , :b_call , :b_return ▸ :thread_begin , :thread_end , :fiber_switch ▸ "We can use TracePoint to gather information specifically for exceptions:" (from Ruby doc) ▸ This is COMPLETELY WRONG statement...
- 13. TracePoint interrupted the argument (stringified) and return value (upcased)
- 15. Ruby Features: TracePoint TracePoint Methods ▸ Methods: ▸ Control: disable, enable, enabled? ▸ Event what/where: event, defined_class, path, lineno ▸ Method names: method_id, callee_id ▸ Event special: raised_exception, return_value ▸ And: binding ▸ So what? ▸ We can use TracePoint ▸ to gather information ▸ to overwrite everything
- 17. Ruby Features: Refinements ▸ Refinements provide a way to extend a class locally ▸ Useful use case. (Safety monkey patching) Ruby Features: Refinements
- 18. Another use case ▸ Super private method Ruby Features: Refinements
- 19. Ruby Features: Method Hooks ▸ Ruby has six method hooks ▸ Module#method_added ▸ Module#method_removed ▸ Module#method_undefined ▸ BasicObject#singleton_method_added ▸ BasicObject#singleton_method_removed ▸ BasicObject#singleton_method_undefined Ruby Features: method hooks
- 20. Sample: #method_added Ruby Features: method hooks
- 21. Method hook provides a way to implement method modifier.
- 22. Hacks: Method modifiers ▸ final (https://github.com/joker1007/finalist) ▸ forbid method override ▸ override (https://github.com/joker1007/overrider) ▸ enforce method has super method ▸ abstract (https://github.com/joker1007/abstriker) ▸ enforce method override These method modifiers work when Ruby defines class. It is runtime, but in most case, before main logic. Hacks: method modifiers
- 23. Sample code: finalist Hacks: method modifiers
- 25. Sample code: abstriker Hacks: method modifiers
- 26. How to implement method modifiers I use so many hook methods. #included, #extended, #method_added, and TracePoint. And I use Ripper. In other words, I use the power of many black magics !! Hacks: method modifiers
- 27. Use case of method_added in finalist Ruby Features: method hooks MyObject Finalist def method_added def verify_final_method ▸ #method_added checks violations call
- 28. Limitation of Method Hooks ▸ But it is not enough to implement “finalist” actually ▸ Ruby has so many cases of method definition ▸ def or #define_method ▸ #include module ▸ #extend, #prepend ▸ Each case has dedicated hooks Ruby Features: method hooks
- 29. #include changes only chain of method discovering Foo Object Bar Insert module to hierarchy It is different from method adding Class#ancestors displays class-module hierarchy Ruby Features: method hooks
- 30. Hooks in finalist gem hook method override event by method_added subclass singleton_method_added subclass class methods included included modules extended extended moduled
- 31. And I talk about TracePoint too
- 32. overrider and abstriker use TracePoint ▸ #inherited and #included to start TracePoint Tracepoint in overrider, abstriker MyObject Overrider/Abstriker def included(or inherited) TracePoint.trace(:end, :c_return, :raise) call
- 33. Tracepoint in overrider, abstriker TracePoint Hook Overrider self.override_methods = [ :method_a, :method_b, ] ▸ Use Method#super_method method to check method existence (ex. method(:method_a).super_method)
- 34. Why use TracePoint? ▸ In order to verify method existence at the end of class definition. ▸ Ruby interpreter needs to wait until the end of class definition to know a method absence. ▸ override and abstract cannot detect violation just when they are called. ▸ In ruby, The only way to detect the violation is TracePoint.
- 35. Advanced TracePoint: Detect particular class end Advanced Tracepoint :end event cannot trace definition by `Class.new`. Use :c_return and return_value to detect particular class end
- 36. Advanced TracePoint: Ripper combination Advanced Tracepoint ▸ Ripper: ▸ a standard library, a parser for Ruby code ▸ outputs token list and S-expression ▸ S-expression is similar to AST ▸ has token string and token position ▸ Future option: RubyVM::AbstractSyntaxTree (2.6.0 or later)
- 38. Advanced TracePoint: Ripper combination Advanced Tracepoint Detect target Sexp node by TracePoint#lineno Sexp node type expresses the style of method cal
- 39. Ripper empowers TracePoint ▸ Conclusion: ▸ TracePoint detects events and where it occurs ▸ Ripper.sexp provides how methods were called ▸ Other use case ▸ power_assert gem also uses this combination Advanced Tracepoint
- 40. Black Magic is dangerous actually, but it is very fun, and it extends Ruby potential These gems are proof of concepts, But these are decent practical.
- 41. Ruby Quiz Break (25-30min) What the difference between: - #undef_method - #remove_method
- 42. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo def foo class Bar < Foo def foo Bar.new.foo()
- 43. RUBY QUIZ class Foo def foo class Foo def foo class Bar < Foo class Bar < Foo Bar.new.foo() remove_method(:foo) def foo NoMethodError undef_method(:foo)
- 44. Hack: with_resources Add "with" Statement to Ruby ▸ Safe resource allocate/release ▸ Ensure to release resources ▸ at the end of a lexical scope ▸ in reverse order of allocation ▸ Idioms used very frequently ▸ Other languages: ▸ Java: try-with-resources ▸ Python: with ▸ C#: using
- 45. Hack: with_resources Safe Resource Allocation/Release Statement in Ruby ▸ Open method with blocks ▸ File.open(path){|f| ... } ▸ Ruby way (?) ▸ More indentation ▸ Not implemented sometimes (e.g., TCPSocket)
- 46. Hack: with_resources with_resources.gem ▸ Safe resource allocation ▸ Top level #with ▸ by Kernel refinement ▸ Resource allocation as lambda ▸ Multi statements to allocate resources ▸ to release first resource if second resource allocation raises exception ▸ Block to define scope for resources https://github.com/tagomoris/with_resources
- 47. Hack: with_resources Implementing #with in Ruby ▸ TracePoint ▸ "b_return": pass allocated resources to block arguments ▸ "line": identify allocated resources in lambda ▸ Binding ▸ detect newly defined local variables in allocation lambda ▸ Refinements ▸ introduce #with in top-level without side effects https://github.com/tagomoris/with_resources
- 48. Hack: deferral Alternative: defer? ▸ Multi-step resource allocation in a method ▸ Nesting! w/ #with ▸ not so bad ▸ many nesting looks a bit messy :( ▸ Alternative? ▸ defer in golang
- 49. Hack: deferral Adding #defer to Ruby ▸ Block based #defer ▸ Should work ▸ Requires 1-level nesting *always* ▸ Defer.start, end (+ nesting) look too much (than golang) ▸ Abnormal case: reassigning variables
- 50. Hack: deferral deferral.gem ▸ Safe resource release ▸ Top level #defer ▸ by Kernel refinements ▸ Deferred processing to release resources ▸ at the end of scope (method, block) ▸ or exception raised
- 51. Hack: deferral Implementing "defer" in Ruby ▸ #defer ▸ Enable TracePoint if not yet ▸ Initialize internal stack frame ▸ TracePoint ▸ Monitor method call stack ▸ Get the snapshot of local variables in defer block ▸ Call release blocks at the end of scope ▸ Binding ▸ save/restore local variables of release block ▸ Refinements ▸ introduce #defer in top-level without side effects stack level 0 stack level 1
- 52. The Hard Thing in Magical World: Debugging!
- 53. "Give yourself to the dark side. It is the only way you can save your (Ruby)friends." - Darth vader Thank You! @joker1007 & @tagomoris