Mesos + Singularity: PaaS automation & Sustainable Development Velocity for mortals
4 likes736 views
The document discusses the integration of Mesos and Singularity in creating a Platform as a Service (PaaS) that enhances development efficiency and operational flexibility. It outlines the transition from monolithic to microservices architecture, emphasizing the benefits of autonomous teams and continuous deployment. Additional details include maintaining a centralized deployment system, the performance of the Mesos framework, and recommendations for optimal infrastructure investment.
Mesos + Singularity: PaaS automation & Sustainable Development Velocity for mortals
- 1. Mesos + Singularity:Mesos + Singularity: PaaS automation for mortalsPaaS automation for mortals Gregory Chomatas @gchomatas PaaS team
- 2. 120 meters: My shortest travel to a Conference120 meters: My shortest travel to a Conference
- 3. Thales of Miletus - 624 BC Those who can, do, the others philosophise... Really? Miletus
- 4. Optionality* with MesosOptionality* with Mesos Invest in a Mesos-powered PaaS and keep doing what you love most; building your product * Optionality is the property of asymmetric upside (preferably unlimited) w ith corresponding limited downside (preferably tiny)
- 5. Blogging SEO Social Media CMS Lead Management Landing Pages Calls-to-Action Marketing Automation Email Analytics CRM Sync
- 6. The underlying culture & structureThe underlying culture & structure 12-factor apps .net monolith to microservices Small, autonomous teams with end-to-end ownership - no ops
- 7. High productivityHigh productivity ~100 engineers 800+ components that can be updated/scaled independently QA: ~400 small to medium AWS machines PROD: ~750 medium to large AWS machines Source: Martin Fowler http://martinfowler.com/bliki/MicroservicePremium.html
- 8. 1. Develop locally 2. Provision QA aws instance 3. Deploy via local Python script 4. Provision PROD aws instance 5. Deploy via local Python script 6. Repeat 4 & 5 to scale … 10. Repeat 4&5 at 4am to replace hw But for how long...But for how long...
- 9. Statically partitioning the cluster is inefficientStatically partitioning the cluster is inefficient
- 10. The cost of flexibility & asynchronicityThe cost of flexibility & asynchronicity High operational overhead Poor utilisation & elasticity Higher rate of failures
- 11. Redress the balance with a Mesos-based PaaSRedress the balance with a Mesos-based PaaS Abstract away machines Homogenous environment Scale out in seconds Centralized deployables registry Sept 2013: Our First Mesos Cluster
- 12. To Boldly go...to Singularity
- 13. Singularity: do more with a single schedulerSingularity: do more with a single scheduler Great UI & HTTP API Native Docker Support Health Checks Load Balancing API Log Maintenance Oct 2013: Start building Singularity
- 14. Singularity: do even more...Singularity: do even more... Security / artifact signature verification Agent & Rack maintenance Webhooks Auto-rollback Email Notifications Executor cleanup Singularity Components
- 15. The PaaS StackThe PaaS Stack BUILD DEPLOY RUN Jenkins Orion Singularity
- 16. The Build / Deploy cycleThe Build / Deploy cycle buildpack runner S3
- 17. The Deployer - Dry runThe Deployer - Dry run
- 18. The Deployer - DeployingThe Deployer - Deploying
- 20. SingularitySingularity Task viewTask view
- 21. Singularity fSingularity file tailingile tailing
- 22. SingularitySingularity Health Checks & ResourcesHealth Checks & Resources
- 23. SingularitySingularity All Deployables viewAll Deployables view
- 25. SingularitySingularity Cluster Maintenance viewCluster Maintenance view
- 26. Migration to Mesos - TimelineMigration to Mesos - Timeline
- 27. Manual Server ProvisioningManual Server Provisioning
- 28. Server provisioning UI usageServer provisioning UI usage
- 29. 1800+ deployables1800+ deployables ~300 deploys / day~300 deploys / day 10 minutes from git10 minutes from git push to productionpush to production
- 30. I want persistent IMAP connections to 200k+I want persistent IMAP connections to 200k+ inboxes (Jul 2015)inboxes (Jul 2015)
- 32. Stateful Services Single Process services Hard coded stationary hosts Cgroups memory isolation User resistance Migration IssuesMigration Issues
- 33. All eggs in one basket Mesos / Framework issues (pingback port) Failures (Zookeeper, Mesos, Singularity) Cluster Maintenance Missing features Operational IssuesOperational Issues
- 34. Phased rollout of new Kernel, Instance types Rolling upgrade of instance basic SW with puppet vars MaintenanceMaintenance Rolling upgrade of master/agent process with ansible Local testing on docker cluster Roll out at infra-QA then product-QA and last to Production cluster Deploy tools deploy themselves but maintain command line alternative with fabric
- 35. Optionality with Mesos@HubSpotOptionality with Mesos@HubSpot Singularity Ghidorah - Load Balancers in Mesos Massive Builds in Mesos Baragon - Tasks Load Balancer Manager Mesos Spark Cluster
- 36. The sweet spotThe sweet spot Source: Mark Leslie (http://firstround.com/review/The-Arc-of-Company-Life-and-How-to-Prolong-It/)
- 37. Invest early in a deploy & build infrastructureInvest early in a deploy & build infrastructure Dedicate 1-2 engineers to experiment onDedicate 1-2 engineers to experiment on a Mesos powered PaaSa Mesos powered PaaS Try Singularity today!Try Singularity today! github.com/HubSpot/Singularity
- 38. HubSpot Blog: How We Built Our Stack For Shipping at Scale Blazar: An out-of-this world build system! Baragon: Load Balancer API Useful linksUseful links