Rails scaling
6 likes948 views
This document provides tips on how to scale a Rails application, team, and yourself. It discusses strategies for scaling infrastructure like using Git and monitoring tools, optimizing deployments, server setup, and background processing. Specific techniques are suggested for handling large data sets, file uploads, and Facebook integration. The document also stresses the importance of communication within the team and using tools to stay productive.
![Big data sets
Only select what is really needed
Submission.accepted_submissions.select(latest_cols).where
(:is_photo_of_the_week => false).order(sort)
latest_cols =
[:id, :user_id, :fb_count, :fb_count_en, :shard, :picture, :picture_is_landscape, :cr
eated_at, "TO_CHAR(submissions.updated_at, 'YYYYMMDDHH24MISS.US')
as updated_at"]](https://image.slidesharecdn.com/railsscaling-111128024711-phpapp02/85/Rails-scaling-22-320.jpg)
Rails scaling
- 1. Real Life Rails Scaling
- 2. Agenda • Scale your App • Scale your Team • Scale yourself
- 4. Homepage
- 5. Features
- 6. Competition
- 7. Numbers irb(main):001:0> Submission.count => 179430 irb(main):002:0> User.count => 206546 # du -sh public/system/ 24GB public/system/
- 9. Infrastructure • Use GIT & get a tool your team can use • GIT commit eMails • Google Docs for shared documents • Try to ban eMail (impossible...)
- 10. Monitoring • Consider monitoring software • Find slow requests, slow queries etc • We used new relic RPM on a trial: ! •
- 11. Deployment • Staging system on the same server • Auto deploy to Staging on commit • Useful when in 2+-part team • One command to deploy (cap deploy) to production • Full Root access == full fun
- 12. Server setup • Database on a second server • Direct link via private network • Shared storage via NFS possible, better if hosted via secondary nodes (asset host) • More RAM == more happy
- 13. HTML best practices • Less code == better code • Compact HTML markup • Compact CSS & JS • Short class names • Think: Every 1KB matters • Include JS per page only, not global
- 14. HTML tuning • http://validator.w3.org/ • http://developer.yahoo.com/yslow/ • Rails 3.1: Asset pipeline for compression
- 15. Rails request tuning • Think: Return to user as soon as possible • Think: Every request can block another • Don’t: Process images immediately • Don’t: Recalculate lists / caches in a request • Do: Use background jobs • Do: Set flags for further actions
- 16. Background processing • Beanstalk is heavy for tons of requests, but great for real time • Consider cronjob for background tasks • whenever gem
- 17. (Image-) File uploads • Uploading images will take a while • Store image immediately & return • Create thumbnails later & in background • We used carrierwave & conditional versions
- 18. (Image-) File uploads • Conditional versions are kick ass!
- 19. (Image-) File uploads • One directory can only keep 32768 files... • Sharding: distribute files into subdirectories Carrierwave rocks! public/system/uploads/ 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg public/system/uploads/ 2011-11-23 2011-11-24 1/pic.jpg 2/pic.jpg 3/pic.jpg 4/pic.jpg 5/pic.jpg
- 20. Facebook integration • Before: fb_graph gem to read user photos • Problem: 1000 requests per minute & facebook blocks the access -> site down • Solved: fb connect JavaScript
- 21. Big data sets • Identify key data • Fragment cache each item • Use JS for post- processing
- 22. Big data sets Only select what is really needed Submission.accepted_submissions.select(latest_cols).where (:is_photo_of_the_week => false).order(sort) latest_cols = [:id, :user_id, :fb_count, :fb_count_en, :shard, :picture, :picture_is_landscape, :cr eated_at, "TO_CHAR(submissions.updated_at, 'YYYYMMDDHH24MISS.US') as updated_at"]
- 24. Tune your team
- 25. Communication • Project plan • Communicate clear about bottlenecks • Like eMail, server storage
- 26. GIT access to everyone • Deploy to staging on each commit
- 27. Tune yourself
- 28. Your tools+ • Many good Rails blogs are blocked by GFW • Get a non-free VPN • During an intense project, stuff comes up • Create a delta file • Do testing and automate using a build server
- 29. Delta File
- 30. Podcasts • Watch & apply: • railscasts.com • codeschool.com