Docker at Spotify - Dockercon14
- 1. Docker at Spotify ! Rohan Singh @rohansingh
- 2. Spotify “Music for every moment” ! 40+ million active users ! 20+ million songs available (we’re in 55+ countries, differs by market) ! Live in the US for 2 years 2
- 3. What are we working with? ! 100+ distinct backend services ! 5000+ production servers in 4 sites worldwide ! Almost 300 servers per ops engineer 3
- 5. Build Debian packages ! Requisition new hardware ! Use combination of Puppet and internal tooling (deployify) 5
- 6. old way: SSH to a bunch of machines, apt-get install || upgrade ! deployify: Uses fabric, which just SSH’s, runs apt-get install || upgrade 6
- 7. What’s wrong with that? 7
- 8. Requisitioning new hardware is slow ! Hardware utilization is low ! We’ve automated and improved, but still too slow 8
- 9. Deployments often fail… ! …though usually only partially 9
- 10. ! Machines get denormalized easily 10
- 11. Debian packaging is its own special hell So many damn files! ! debhelper ! Distro & version specific ! apt-get update 11
- 12. What do we want? 12
- 14. Why Docker? ! It gives us “repeatable”, right out of the box ! ! And it’s fairly straightforward, too! 14
- 15. Repeatability? ! ! The code you test isn’t necessarily the code that runs in prod 15
- 16. Repeatability — Docker ! Build an image ! Run tests against the image ! Run the tested image in prod 16
- 17. Straightforward ! Just one Dockerfile ! Pretty simple syntax ! Immutable images — easy to reason about 17
- 18. Fault-tolerance? Even if your code is perfect, deployments might fail ! Puppet or Debian repos are down ! Network issues ! The machine dies 18
- 19. Fault-tolerance — Docker Just one thing has to work — pulling & running an image ! If it fails, just try again ! No machines left in weird states* ! ! *(in theory) 19
- 20. What more do we need? 20
- 21. Deploying Docker containers across our entire fleet of servers ! Some way to deal with Docker or container failures ! That’s pretty much it 21
- 22. Looking around for a Docker platform Several really feature-full offerings ! Pretty opinionated and all-encompassing ! All we need is reliably deploying containers to a fleet of Docker hosts 22
- 23. Helios 23
- 24. Makes sure your containers are deployed & running exactly where you want them… ! …and that’s pretty much it ! ! github.com/spotify/helios 24
- 25. What’s in the box? Helios master frontend that you talk to ! Helios agent runs alongside each Docker instance ! Zookeeper ! Straightforward CLI 25
- 26. How does it work? Create a job — tell Helios which Docker image to deploy ! Deploy the job — tell Helios where to run the Docker containers 26
- 27. See it in action 27
- 28. What’s next? 28
- 29. Helios is just the beginning ! Brought up plenty of Helios+Docker agents in our private cloud for our devs ! Integrated Helios with JUnit — write JUnit tests run against Docker containers 29
- 31. This week — first Helios+Docker-based service in production ! ! In the future — Containerizing more of our infrastructure 31
- 32. Give it a go Open source: github.com/spotify/helios ! Try it out: git clone https://github.com/spotify/helios.git && cd helios && vagrant up ! ! ! Come play with us (spotify.com/jobs) 32