Using puppet, foreman and git to develop and operate a large scale internet service (eBuddy).
- 1. Using Puppet, Foreman and Git to Develop and Operate a Large Scale Internet Service
- 2. Agenda • Introduction and background • Puppet • Foreman • Demo: Deploy change to Development Cluster • Future plans • Questions and Answers
- 4. Joost van de Wijgerd • 37 Chief Architect / VP Operations • Dutch • In IT Industry since 1999 • @ eBuddy since 2009 • Responsibilities – Overall Architecture – Datacenter Operations
- 5. About eBuddy • Founded in 2003 • eBuddy Chat Platform peak at 35M MAU, 9M DAU, 17B Messages per month • Started XMS Messaging Platform in 2010 • Currently managing 3 production sites and 2 development sites • About 500 instances (physical and virtual) total
- 6. DevOps!? – Some Quotes from the Web • “Developer and Operations collaboration” • “Infrastructure as code” • “Using Automation” • “Using Kanban”
- 7. DevOps at eBuddy • Operations – Provide Infrastructure and Monitoring – Provisioning with Foreman, automating with Puppet – Providing a Puppet framework for Developers – Organized through Kanban • Development (Backend Teams) – Deliver systems not compiled software (including puppet classes) – Manage Development environments – Deploy to Production Environments – Using Scrum
- 8. 9 Pull artifact (Pro) Process 3 upload 4 scp puppetmaster (Dev) 8 puppet run 2 checkout 1 Read properties 0 6 post_commit 7 Puppet kick 1 commit 5 push properties
- 9. Puppet
- 10. What is Puppet? • Centralized Configuration Management • Automate repetitive tasks • Deploy applications
- 11. How does it work?
- 12. Puppet Details • Puppet DSL built on top of Ruby • Puppet Classes defined in Puppet Modules • Use Modules to compose application stacks • Collection of Puppet Classes make an environment • Puppet environment(s) stored in Git and pulled by Puppet Masters
- 13. Module Example • a module is simply a directory tree with a specific, predictable structure: – MODULE NAME • manifests contains init.pp file with module class • files served as static files to agent nodes • templates ERB (Ruby Templating) files evaluated at runtime • lib plugins (Ruby Code)
- 15. Foreman
- 16. What is Foreman? • External Node Classifier for Puppet • Add Nodes to Host Groups • Assign Puppet Classes per Host or Host Group • Define parameters per Host Group or per Host • Report Collector for Puppet runs • Kick Puppet agents • Access control – Let Developers deploy on Production with needing platform access
- 17. EBUDDY THEME
- 18. EBUDDY THEME
- 19. EBUDDY THEME
- 20. EBUDDY THEME
- 21. EBUDDY THEME
- 22. Demo
- 23. Future Plans
- 24. Future Plans • Complete automatic provisioning with Foreman – Fill in MAC address and Host Group.. Et Voila! • Refactor for Puppet 3.x and Hiera – Completely separate Data and Code – To support multiple environments from a single codebase • Implement staged Git repo approach with Pull Requests and Code Reviews • Unsupervised application deployment
- 25. Q&A
- 26. We’re Hiring! • We’re a Super.Cool.Company© • Visit jobs.ebuddy.com
Editor's Notes
- #12: Facts Examples: installed application version, os version, memory size
- #15: service, package and file are built-in resource typesResource-type handled by pluggable providers -> example aptAny changes to the ntp.conf will cause the service to restart (if running)
- #20: Same parameters can be set on Host level (hierarchical)