Puppet at Spotify
1 like691 views
Spotify has over 20 million monthly active users and launched in 17 countries with over 20 million songs and more than 1 billion playlists. Spotify uses Puppet to manage roughly 3500 nodes across 3 Puppet installations, each with their own PuppetDB and certificate authority. Puppet has been used at Spotify for 2.5 years, with more than 300 changes per month contributed by 137 committers to their Puppet repository, which includes 240 Puppet modules. The infrastructure uses Git branches and Puppet environments to manage changes and will switch from various node classification methods to using Hiera.
Puppet at Spotify
- 1. Puppet at Spotify Erik Dalén February 1, 2013
- 2. Spotify • Over 20 million monthly active users • Launched in 17 countries • Over 20 million songs • More than 1 billion playlists Growing quickly
- 3. Puppet Infrastructure • Rougly 3500 nodes • 3 different Puppet installations, each with their own PuppetDB and CA • One puppetmaster per data centre
- 4. Puppet Infrastructure Puppet users since 2.5 years • More than 300 changes per month • 137 committers to our Puppet repository • 240 puppet modules Code review by operations using Gerrit
- 5. Puppet Infrastructure Git branch == Puppet environment
- 6. Node Classification At the moment using different ENCs in different Puppet installations Will be switching to using hiera to classify nodes
- 7. Node Classification Hostname standard: sto1-puppet-a1 <site>-<service>-<pool><poolid> Mainly service fact used to determine the classes a machine should have
- 8. Service discovery Using SRV records to discover services
- 9. Service discovery wasd: github.com/spotify/wasd Ingests Cassandra rings and DNS records across multiple sites at a regular interval, then exposes information about hosts and services. One can ask what a host does, optionally for which sites; which hosts in a site offer a service; which hosts offer a service for a site. Using a REST API.
- 10. Questions?