Leveraging a distributed architecture to your advantage
1 like1,521 views
The document discusses scaling challenges for web applications built on a LAMP stack (Linux, Apache, MySQL, PHP) and emphasizes the importance of proper strategies for scaling and deployment. It also outlines common scaling mistakes, such as not preparing for disaster recovery and failing to automate deployment processes. Key takeaways include preparing code for scalability, distributing hosting options, testing resilience, and automating workflows to improve efficiency.
Leveraging a distributed architecture to your advantage
- 1. TO YOUR ADVANTAGE LEVERAGING A DISTRIBUTED ARCHITECTURE
- 2. WHO AM I? MICHELANGELO VAN DAM ▸ Professional PHP architect ▸ President PHPBenelux ▸ Coach at CoderDojo ▸ Contributor to open source projects ▸ Zend Certified Engineer
- 3. WHO AM I? MICHELANGELO VAN DAM ▸ Professional PHP architect ▸ President PHPBenelux ▸ Coach at CoderDojo ▸ Contributor to open source projects ▸ Zend Certified Engineer Rafael Dohms @rdohms
- 4. WHO AM I? MICHELANGELO VAN DAM ▸ Professional PHP architect ▸ President PHPBenelux ▸ Coach at CoderDojo ▸ Contributor to open source projects ▸ Zend Certified Engineer Rafael Dohms @rdohms Me @DragonBe
- 6. WHAT IS A LAMP STACK? A COMMON ACRONYM ▸ Linux ▸ Apache ▸ MySQL ▸ PHP
- 8. WHY USED? SMALL WEB APPLICATIONS ▸ WordPress blogs ▸ Joomla marketing sites ▸ Drupal business sites ▸ TYPO3 publications ▸ A lot more …
- 9. SIMPLICITY IS ULTIMATELY A MATTER OF FOCUS. Ann Voskamp TWEETABLE QUOTE
- 10. BENEFITS GALORE WE LOVE EASY ▸ Single point of concern ▸ Easy to maintain ▸ Easy to distribute ▸ Easy to protect
- 11. BENEFITS GALORE WE LOVE EASY ▸ Single point of concern ▸ Easy to maintain ▸ Easy to distribute ▸ Easy to protect
- 12. NOT ALL SUNSHINE DOESN'T SCALE WELL ▸ Increase of users ▸ Huge amount of data ▸ Additional logic or features ▸ Compliance requirements
- 13. MORE WE LIKE
- 15. WE NEED TO SCALE!
- 19. SCALING MISTAKE #1: DON’T ADD MORE MACHINES AND THINK YOU’RE SCALED. Remember this!!! TWEETABLE QUOTE
- 20. SOME WISDOM SCALE WITH PURPOSE ▸ Separate based on responsibility ▸ Pinpoint your weakest link ▸ You need more of them ▸ Look for “alternate” (better) solutions ▸ e.g. Nginx vs. Apache ▸ Don’t over-scale
- 22. IS YOUR APP READY TO BE SCALED? Find the “hard” requirements!
- 23. ANALYSING YOUR CODE SOMETHING TO LOOK OUT FOR resources.db.params.host = "database.server.tld" resources.db.params.port = 3306 resources.db.params.username = "fda23a84" resources.db.params.password = "b10a8db164e0754105b7a99be72e3fe5" resources.db.params.dbname = "webapp" resources.db.isDefaultTableAdapter = true
- 24. REPLACE WITH SOMETHING THAT CHANGES OUTSIDE YOUR APP ENVIRONMENT VARIABLES CAN HELP resources.db.params.host = WEBAPP_DB_HOST resources.db.params.port = WEBAPP_DB_PORT resources.db.params.username = WEBAPP_DB_USERNAME resources.db.params.password = WEBAPP_DB_PASSWORD resources.db.params.dbname = WEBAPP_DB_DBNAME resources.db.isDefaultTableAdapter = WEBAPP_DB_DEFAULT
- 25. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED
- 26. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching
- 27. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine
- 28. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer
- 29. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication
- 30. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues
- 31. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues ▸ Workers
- 32. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues ▸ Workers ▸ API’s
- 33. HOW SOMETHING SIMPLE THINGS BECOMES VERY COMPLEX VERY QUICKLY MORE ITEMS ADDED ▸ Caching ▸ Search engine ▸ Load balancer ▸ Replication ▸ Queues ▸ Workers ▸ API’s ▸ and even more things …
- 34. PHP-FPM NGINX Shared FS MariaDB MariaDB MariaDB MongoDB MongoDB MongoDB Redis Cache Redis Cache Redis Cache ElasticSearch ElasticSearch Varnish RabbitMQ PHP (workers) PHP (workers) PHP-FPM NGINX
- 35. EVENT SOURCING: GBC (MARCO PIVETTA) MUST-SEE PRESENTATION
- 36. MAKING CHOICES
- 38. ON PREM
- 39. ON PREM CLOUD
- 41. MOVE TO THE “CLOUD”, TRUST ME I’M A PROFESSIONAL!!!
- 43. Source: The Register 2015-09-20 Source: CBS News 2017-02-28 Source: Forbes 2014-11-18 Source: ZDNet 2016-09-15 Source: C|Net 2009-06-29 Source: The Register 2016-04-20 Source: InfoWorld 2013-03-22
- 45. THIS IS BAD!!! NO NEED TO SAY…
- 46. SCALING MISTAKE #2: PUT ALL YOUR EGGS IN ONE BASKET. Remember this!!! TWEETABLE QUOTE
- 47. STRATEGY IS KEY!
- 49. ONLINE? WHAT DID WE DO TO STAY
- 50. CAN YOU LIVE WITH REDUCED CAPACITY BUT STAY ONLINE? First question to clients
- 51. Hosted / On Premise
- 52. DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 Elas-c Load Balancing CloudFront CloudSearch West Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch North Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch South America
- 53. West Europe North Europe South America
- 54. DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 Elas-c Load Balancing CloudFront CloudSearch West Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch North Europe South America
- 55. DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 Elas-c Load Balancing CloudFront CloudSearch West Europe DynamoDB EC2 EC2 Elas-csearchRDS - MySQL Instance SQS Queue S3 CloudFront CloudSearch North Europe South America WARNING: THIS MEANS YOU NEED TO REPLICATE ALL TRANSACTIONS OVER BOTH CLOUD PROVIDERS!!! (COST X 2)
- 56. SCALING MISTAKE #3: NOT READY FOR DISASTER Remember this!!! TWEETABLE QUOTE
- 57. RESILIENCE TESTING NETFLIX SIMIAN ARMY ▸ Chaos Monkey ▸ Chaos Gorilla ▸ Chaos Kong ▸ Janitor Monkey ▸ Doctor Monkey ▸ Compliance Monkey ▸ Latency Monkey ▸ Security Monkey
- 58. ALWAYS PREPARE FOR THE WORST… …so you’re ready when it happens.
- 59. SCALING MISTAKE #4: DON’T AUTOMATE YOUR DEPLOYMENT PROCESSES Remember this!!! TWEETABLE QUOTE
- 60. Build Feedback Execute Delivery SCM Commits SCM Change Polling test staging production
- 61. Unit tests Metrics Provisioning new target Provisioning databases Provisioning workers Integration tests Promote branch CI Unit tests Metrics Provisioning new target Provisioning databases Provisioning workers Integration tests Promote branch CI
- 64. AUTOMATE YOUR DEPLOYMENT PROCESS So you can deploy multiple times a day.
- 65. WHAT TO TAKE HOME FROM THIS TALK RECAP ▸ Prepare you code to scale and distribute ▸ When online is important, scale over multiple hosting options ▸ Test your resilience for failure ▸ Automate your processes ▸ Build pipelines for all tasks ▸ Release with confidence
- 66. in it2PROFESSIONAL PHP SERVICES Michelangelo van Dam Zend Certified Engineer contact@in2it.be - www.in2it.be - T in2itvof - F in2itvof Microsoft Azure Zend Framework Consulting Quality Assurance & Disaster Recovery