Scaling Drupal on Amazon Web Services (DrupalCamp Brighton)
3 likes2,045 views
The presentation discusses scaling Drupal on Amazon Web Services (AWS) by addressing high throughput, resilience, and effective server provisioning. It emphasizes the use of a decentralized architecture with services like EC2, RDS, and load balancers, along with automation tools like Ansible for deployment. The final setup is noted to be complex but capable of handling significant user traffic efficiently.
![[auto]scaling
• AMIs = Amazon Machine Image
• Autoscaling policies
• “pets versus cattle” again
• Problem: servers must have latest Drupal code
• Conflict with Fabric 'push‘ technique
• Interim solution: create an AMI after
each deployment
35](https://image.slidesharecdn.com/scalingdrupalonamazonwebservicesdr-150128105032-conversion-gate01/85/Scaling-Drupal-on-Amazon-Web-Services-DrupalCamp-Brighton-35-320.jpg)
Scaling Drupal on Amazon Web Services (DrupalCamp Brighton)
- 1. Scaling Drupal on Amazon Web Services Tristan Roddis DrupalCamp Brighton, January 2015
- 2. What we created: www.qdl.qa 2
- 3. 3
- 4. 4
- 5. 5
- 6. • Several hundred thousand pages • Tens of terabytes of data • Requirement for high throughput • Requirement for high resilience 6
- 9. Let’s not scale it like this! 9
- 12. Avoid single point of failure 12
- 13. AWS TLAs • EC2 – Elastic Compute Cloud = virtual servers • RDS – Relational Database Service = database • ELB – Elastic Load Balancer = distribute requests 13
- 14. Therefore… 14
- 15. RDS has easy redundancy 15
- 16. So now we have 16
- 18. GlusterFS components • EBS – Elastic Block Storage = file storage up to 1Tb • LVM – Logical Volume Manager = grow and shrink volumes • Gluster has storage ‘bricks’ • Cluster appears as single mount point (NFS, Gluster protocol etc.) 18
- 19. Distribute servers • Amazon has different Regions and Availability Zones 19
- 20. Use different Availability Zones 20
- 21. I lied… 21
- 22. Adding Solr and IIPImage… 22
- 23. Additional security • VPC – Virtual Private Cloud = can create your own network architecture • Security Groups = firewall rules • IAM = Identity and Access Management = restrict AWS roles 23
- 24. Final setup 24
- 25. Woah – that’s complicated 25
- 26. Server provisioning • Automate everything! • Use Ansible (or Chef, Puppet, etc.) 26 “If you have to SSH into your servers, then your automation has failed.” — Rich Adams
- 27. 27
- 28. Pets vs cattle 28 “When one of them gets sick, you shoot 'em in the head and replace 'em with a new one.” —Randy Bias, CEO of Cloudscaling
- 29. Deployment I: Drupal tools • Drush • Drush master • Features • Strongarm 29
- 30. Deployment II • Use Fabric (or Ansible, Capistrano, etc.) • @runs_once decorator for e.g. drush commands. 30
- 31. Load Test • Jmeter • Blitz.io or similar 31
- 32. Tune performance • Tools: Apache Benchmark (ab), Yslow, Google PageSpeed • xhprof – remove bottlenecks • Fix warnings and notices • Disable database logging • OpCode cache (Zend Optimizer, APC) • Minify CSS/JS • Lazy Loading via AJAX 32
- 33. 33
- 34. Add caching • Varnish • Redis/memcached for cache tables (can use Amazon Elasticache) • CDN (e.g. Amazon Cloudfront) • Re-run load tests 34
- 35. [auto]scaling • AMIs = Amazon Machine Image • Autoscaling policies • “pets versus cattle” again • Problem: servers must have latest Drupal code • Conflict with Fabric 'push‘ technique • Interim solution: create an AMI after each deployment 35
- 36. Backup • AMIs - Amazon Machine Image = snapshot server images • ELB snapshots • S3 – Simple Storage Service = REST storage • Glacier = long-term storage • We used: – Snapshots with RDS – S3 with versioning, plus Glacier for Drupal assets and image assets 36
- 37. Monitoring I: Cloudwatch alerts 37
- 38. Monitoring II: log analysis • SumoLogic to aggregate logs (or Loggly, Papertrail, etc.) 38
- 39. Monitoring III: notifications • OpsGenie (or PagerDuty or similar) 39
- 40. Gotchas • GlusterFS for PHP files – don’t! • Custom Drupal cache – careful of uncontrolled growth! 40
- 41. Tips • Use “termination protection” • Enable ELB logging • “Maintenance 200” module • Create a “down for maintenance” server 41
- 42. Conclusion • 21 instances • 6 load balancers • 67 ELB volumes • Lot of work to set up, but quick to alter • Easily coped with launch traffic: – 21k sessions/day – peak of 350 concurrent users 42
- 43. Further Reading • Justin Slattery, "Multiple Region Autoscaling Drupal in Amazon Web Services" http://dev.mlsdigital.net/posts/Cloud-Native- Drupal/ • Rich Adams, "AWS Tips I Wish I'd Known Before I Started" https://wblinks.com/notes/aws-tips-i- wish-id-known-before-i-started/ • Laura Hamilton, "Are your servers pets or cattle?" http://www.lauradhamilton.com/servers- pets-versus-cattle 43