![MessageVerifier
>> verifier = ActiveSupport::MessageVerifier.new("my super secret")
=> #<ActiveSupport::MessageVerifier:0x2d559ec @secret="my super secret",
@digest="SHA1">
>> data = [1, 10.days.from_now.utc]
=> [1, Sat Oct 24 05:28:07 UTC 2009]
>> token = verifier.generate(data) # Generate a token that is safe to distribute
=>
"BAhbB2kGSXU6CVRpbWUNBWcbgO7HdXAGOh9AbWFyc2hhbF93aXRoX3V0Y19jb2Vy
Y2lvblQ=--ff41cf5575006a2797cad49e6738361346292bfa"
>> id, expiry_time = verifier.verify(token) # Get the data back
=> [1, Sat Oct 24 05:28:07 UTC 2009]](https://image.slidesharecdn.com/lessonslearnt-091018133732-phpapp02/85/Lessons-Learnt-in-2009-58-320.jpg)
![MessageVerifier
# User.rb
def remember_me_token
User.remember_me_verifier.generate([self.id, self.salt])
end
# Controller - when user checks the ‘remember me’
def send_remember_cookie!
cookies[:auth_token] = {
:value => @current_user.remember_me_token,
:expires => 20.years.from_now.utc }
end](https://image.slidesharecdn.com/lessonslearnt-091018133732-phpapp02/85/Lessons-Learnt-in-2009-61-320.jpg)
Lessons Learnt in 2009
- 1. Lessons learnt in 2009 Pratik Naik Jobless
- 2. Lessons learnt in 2009 Pratik Naik Freelancer/ ActionRails Blogger m.onkey.org Rails Core Team member
- 4. And some hobby apps http://planetrubyonrails.com http://tweetmuffler.com
- 5. • My first ever presentation. • Usually don’t like any conferences • But it’s Brazil!!
- 6. So if I screw up... DONT Tweet FFS
- 7. Overview • Using Ruby Enterprise Edition • Testing • Faster tests • Factory v/s Fixtures • Security • Auto escaping for XSS prevention • More MessageVerifier • Asynchronous job processing with DJ • How to use • Fork based batch processing • Scaling • Scaling file uploads with mod_porter • Better Pagination
- 8. This talk is targeted at Ruby Web Developers Not everything may apply to things outside the web Also if this doesn’t interest you, now is the time to go the next room :-)
- 9. Lesson 1 Always use REE
- 10. What is REE ?
- 11. Ruby + COW GC And a bunch of other cool patches
- 12. Maintained by (Photo acquired by the use of force) (That was just googling) The Phusion Guys
- 13. Who uses REE ? And many others
- 14. Who uses REE ? And many others In Production
- 15. Why should you use it for the development ?
- 16. Topmost Reason Super Fast Tests
- 17. MRI - Ruby 1.8.6 $ time rake real 1m45.293s user 0m54.341s sys 0m33.008s
- 18. REE - Ruby 1.8.6 $ time rake real 1m30.219s user 0m40.290s sys 0m25.433s
- 19. That’s15 seconds faster Completely Free YMMV
- 20. Only Catch Twitter’s GC Settings
- 21. .profile RUBY_HEAP_MIN_SLOTS=500000 RUBY_HEAP_SLOTS_INCREMENT=250000 RUBY_HEAP_SLOTS_GROWTH_FACTOR=1 RUBY_GC_MALLOC_LIMIT=50000000 http://blog.evanweaver.com/articles/2009/04/09/ruby-gc- tuning/
- 22. Second Reason ruby-prof & Rails Performance Tests
- 23. What are Rails Performance tests ? Integration Tests + ruby-prof
- 25. $ script/generate performance_test Home exists test/performance/ create test/performance/home_test.rb class HomeTest < ActionController::PerformanceTest def test_homepage get '/' end end
- 26. MRI $ rake test:profile HomeTest#test_homepage (29 ms warmup) wall_time: 25 ms memory: 0.00 KB objects: 0 Needs a custom installation with special Patches
- 27. REE $ rake test:profile HomeTest#test_homepage (48 ms warmup) wall_time: 16 ms memory: 698.18 KB objects: 21752 Just “works”
- 28. Lesson 2 Efficient Testing • Faster Tests • Factory • More Integration Tests & Less Unit Tests
- 29. Faster Tests Tickle http://github.com/lifo/tickle
- 30. $ script/plugin install git://github.com/lifo/tickle.git
- 31. Benchmarks ( they’re real ) $ time rake real 1m30.219s user 0m40.290s sys 0m25.433s
- 32. Benchmarks ( they’re real ) $ time rake tickle real 0m55.691s user 0m37.532s sys 0m22.563s
- 33. That’s another 35 seconds ( On top of the 15 seconds initially saved by REE )
- 34. To get the Best of Tickle • Experiment with the number of processes • Create more test databases It’s all in the README
- 36. Faster Tests fast_context http://github.com/lifo/fast_context
- 37. Problem That’s 5 DB Queries and 5 GET Requests
- 38. Solution That’s 1 DB Query and 1 GET Request 5x Faster w/ one word change
- 39. Catch ? • Tests no longer atomic • Developers should not need to care about atomicity of the tests • It’s an optimization
- 40. Writing more effective tests in less time
- 41. Factory v/s Fixtures ★ Slow ★ Fast ★ Very easy to manage ★ Hard to manage ★ Describes the data ★ Doesn’t describe the being tested data ★ Hardly Breaks ★ Very Brittle ★ Runs all the callbacks ★ Does not run callbacks
- 42. Example : Fixtures What can go wrong ? • Someone could change users(:lifo) to be no longer a ‘free’ account holder • Someone could add more items to ‘lifo’ • Someone could remove lifo’s item! • ‘create_more_items’ could fails because ‘lifo’ failed validations
- 43. Example : Factory What can go wrong ? Not much
- 44. Factory + Faker Awesome Development Data (Clients and Designers Love it)
- 45. Writing Good Factories • Should be able to loop 10.times { Factory(:user) } • No associations in the base Factory Factory(:user) and Factory(:user_with_items) • Should pass the validations
- 46. Integration Tests > Unit Tests
- 49. rails_xss http://github.com/NZKoz/rails_xss By Michael Kozkiarski of http://therailsway.com/
- 50. rails_xss Without rails_xss With rails_xss <%= “<script>alert(‘foo’)</script>” %> <%= “<script>alert(‘foo’)</script>” %> => => <script>alert(‘foo’)</script> <script>alert('foo')</script> Must use h() explicitly No need of h()
- 51. rails_xss • Built in Rails 3 • Enabled by the rails_xss plugin in Rails 2.3.next • Requires Erubis
- 52. rails_xss Introduces the concept of SafeBuffer ( * just the relevant bits )
- 53. rails_xss >> buffer = ActionView::SafeBuffer.new => "" >> buffer << "Hello ! " => "Hello ! " >> buffer << "<script>" => "Hello ! <script>"
- 54. rails_xss • Uses Erubis hooks to make <%= %> tags to always return a SafeBuffer • Modifies all the relevant Rails helpers and mark them as html_safe!
- 55. rails_xss When you don’t want to escape <%= "<a href='#{foo_path}'>foo</a>".html_safe! %>
- 56. MessageVerifier http://api.rubyonrails.org/classes/ActiveSupport/MessageVerifier.html
- 57. MessageVerifier Secret Ruby Data Signed Message (Derived from Cookie Session Store)
- 58. MessageVerifier >> verifier = ActiveSupport::MessageVerifier.new("my super secret") => #<ActiveSupport::MessageVerifier:0x2d559ec @secret="my super secret", @digest="SHA1"> >> data = [1, 10.days.from_now.utc] => [1, Sat Oct 24 05:28:07 UTC 2009] >> token = verifier.generate(data) # Generate a token that is safe to distribute => "BAhbB2kGSXU6CVRpbWUNBWcbgO7HdXAGOh9AbWFyc2hhbF93aXRoX3V0Y19jb2Vy Y2lvblQ=--ff41cf5575006a2797cad49e6738361346292bfa" >> id, expiry_time = verifier.verify(token) # Get the data back => [1, Sat Oct 24 05:28:07 UTC 2009]
- 59. MessageVerifier Example use case “Remember Me” Functionality
- 60. MessageVerifier When you store the ‘remember me’ tokens in the db • Extra column • More maintenance Expiring tokens after every use or after password reset • Doesn’t play well with multiple browsers
- 61. MessageVerifier # User.rb def remember_me_token User.remember_me_verifier.generate([self.id, self.salt]) end # Controller - when user checks the ‘remember me’ def send_remember_cookie! cookies[:auth_token] = { :value => @current_user.remember_me_token, :expires => 20.years.from_now.utc } end
- 62. MessageVerifier • Use a different secret for every use of MessageVerifier rake secret • Make sure to use the ‘salt’ for generating the token, making sure the token expires on the password change
- 64. bcrypt Bcrypt MD5/SHA1 ★ Designed for generating password ★ Designed for detecting data hash tampering ★ Meant to be “slow” ★ Meant to be super “fast”
- 65. bcrypt • bcrypt-ruby gem by Coda Hale works great • Reduces the need of ‘salt’ column by storing the salt in the encrypted password column • Allows you to increase the ‘cost factor’ as the computers get faster
- 66. Lesson 4 Background processing with the DJ http://github.com/tobi/delayed_job
- 67. What is DJ ? DJ Worker Webserver DJ Jobs Database Jobs Worker Webserver DJ Worker Database backed asynchronous priority queue
- 68. How to use DJ ? Minimal Example using Delayed::Job.enqueue
- 69. How to use DJ ? More practical example
- 70. How to use DJ ? Using send_later
- 71. How to use DJ ? Using handle_asynchronously
- 72. Batch Processing w/ DJ
- 73. Batch Processing w/ DJ Tweetmuffler Requirement That’s average 4-10 external calls per user. Every 2 minutes.
- 74. Batch Processing w/ DJ Initial Implementation 1 job/user
- 75. Batch Processing w/ DJ Problems with that ?
- 76. Batch Processing w/ DJ DID NOT SCALE • Too Slow • Way too much memory required • Too many workers required
- 77. Batch Processing w/ DJ Solution Fork based workers w/ REE
- 78. Batch Processing w/ DJ
- 79. Batch Processing w/ DJ Has scaled great so far • 10x faster • Uses 40% less memory • Just 1 worker needed
- 80. Batch Processing w/ DJ General things to remember when forking w/ Ruby • Always reset the database connections • Always reset any open file handlers • Make sure the child calls exit! from an ensure block • Make sure mysql allows sufficient number of connections
- 81. Batch Processing w/ DJ REE Specific things to remember when forking • Call GC.start before you fork • Call GC.copy_on_write_friendly = true as early as possible. Possibly from the top of the Rakefile and environment.rb
- 82. Lesson 5 Scaling
- 84. Mod Porter What’s the problem ? • Rails processes are resource intensive • Multipart parsing for large files can get slower • Keeping a Rails process occupied for multipart parsing of large files can have serious scaling issues
- 85. Mod Porter How does mod_porter work ? • mod_porter is an apache module built on top of libapreq • libapreq does the heavy job of multipart parsing in a cheap little apache process • mod_porter sends those multipart files as tmpfile urls to the Rails app • mod_porter Rails plugin makes the whole thing transparent to the application
- 86. Mod Porter Apache Config File <VirtualHost *:8080> ServerName actionrails.com DocumentRoot /Users/actionrails/application/current/public Porter On PorterSharedSecret secret </VirtualHost> Rails Configration class ApplicationController < ActionController::Base self.mod_porter_secret = "secret" end
- 87. will_paginate Does not scale
- 88. will_paginate The common pattern SELECT * FROM `posts` LIMIT 10,10
- 89. will_paginate Scaling Problems • Large OFFSET are harder to scale • Problems clear when you have more rows than the memory can hold • Very hard to cache • Extra COUNT queries
- 90. How to scale Pagination?
- 91. Scalable Pagination Github
- 92. Scalable Pagination Twitter
- 93. Scalable Pagination What’s common with Github and Twitter ? • Don’t show all the page links • Don’t show the total count • AJAX is much easier to scale when it comes to pagination • Pagination query does not use OFFSET, just LIMIT.
- 94. Scalable Pagination Page 1 page1 = SELECT * FROM `posts` LIMIT 10 WHERE id > 0 ASC id page2_min_id = page1.last.id Page 2 page2 = SELECT * FROM `posts` LIMIT 10 WHERE id > page2_min_id ASC id
- 95. Scalable Pagination Benefits ? • Using no OFFSET is much faster • Plays great with caching. No records ever get repeated • A little less user friendly as you cannot show all the page numbers
- 96. Source http://www.scribd.com/doc/14683263/Efficient-Pagination-Using-MySQL By the Yahoo folks
- 97. That’s all !