Docker in the Cloud
Download as PPT, PDF2 likes876 views
The document discusses Sascha Möllering's presentation on deploying Docker applications in various cloud environments including AWS, Microsoft Azure, and private data centers. It provides real-world examples of implementations using JVM-based event-driven applications with technologies like Apache Kafka and Redis, and explains deployment strategies for different cloud infrastructures. The document also highlights challenges in detecting deployment targets and supports custom deployment configurations through various cloud services.
Docker in the Cloud
- 1. Sascha Möllering | zanox AG Docker in the Cloud
- 2. About me Sascha Möllering sascha@autoscaling.io sascha.moellering@zanox.com Lead Engineer / DevOps Hipster zanox AG http://autoscaling.io @sascha242
- 3. About me Java Magazin 11.14 Architektur im Kontext der Cloud: Patterns und Best Prac AWS Virtual Cloud Summit Continuous delivery in AWS using Docker http://aws-summit.de/ Entwickler Spezial: Docker Über den Wolken: Ein Erfahrungsbericht zum Docker-Ein
- 4. Agenda • Problem? • Real World example • Docker implementations • Deployment • Q&A
- 6. Problem? • Using different Cloud providers • Additionally: own DC • Different infrastructure • Different deployment mechanisms
- 7. Solution! • Different applications! • For each deployment target -> different package!
- 10. • One container to rule them all! • Application detects deployment target • Application uses specific infrastructure Real Solution!
- 11. Real World example • JVM based event-driven application • Ingests data into bus system • Uses Redis as a cache • Base framework is Vert.x • Pre-build Docker image
- 13. Real World example • Deployment targets: – Own data center – Amazon Web Services – Microsoft Azure
- 14. Real World example • Our own data center: – Linux servers with Docker 1.6 – Apache Kafka 0.8.1/0.8.2 • Topics and partitions • Easy to write, hard to read from – Redis 2.8
- 15. Real World example • Amazon Web Services: – EC2 instance with Docker 1.6 – Amazon Kinesis • Amazon’s implementation of Kafka (?) • Streams and shards • KCL does the heavy lifting – ElastiCache with Redis engine • Amazon’s implementation of Redis
- 16. Real World example • Microsoft Azure: – Azure Ubuntu VM with Docker 1.6 – Azure Service Bus • Bus system in Azure • AMQP and JMS • Problem: no partitioning for Java – Azure Cache • Redis implementation in Azure
- 17. Real World example • For Redis: Jedis client in all cases • Difference: bus system – mod-kafka – mod-kinesis – mod-service-bus
- 18. Real World example • mod-kafka: – Vert.x module for Apache Kafka – Recieves events by other verticles – Sends events to Kafka broker – Either Strings or binary data – Supports StatsD – Based on Apache Kafka Java API
- 19. Real World example • mod-kinesis: – Vert.x module for Amazon Kinesis – Recieves events by other verticles – Sends events to Kinesis stream – Either Strings or binary data – Support for partitioning in events – Based on AWS Java SDK
- 20. Real World example • mod-service-bus: – Vert.x module for Azure Service Bus – Recieves events by other verticles – Support for binary data only ATM – No partitioning (Service Bus problem) – Based on AMQP and JMS
- 21. Real World example But … how can we detect where the application is running?
- 23. Real World example boolean isAzure = true; try { RoleEnvironment.getRoles(); } catch (Exception exc) { isAzure = false;}
- 24. Real World example • Problem: not working in Docker container • Alternative: check for /etc/waagent.conf • File has to be mapped in Docker container
- 25. Real World example • StarterVerticle pattern: – Bootstrapping Vert.x application – Start all verticles in StarterVerticle – Deploy all modules in StartVerticle
- 28. Docker implementations • Basic implementation: – EC2 using User Data – Install Docker and start service – Either pull Container in User Data script – Or Remote API
- 29. Docker implementations • AWS OpsWorks: – No “official” support – Docker Chef Cookbook by AWS – Problem: Docker Images are build in the Cookbook
- 30. Docker implementations • Amazon Elastic Beanstalk: – Support for single/multi container environments – ELB support on container level – Multi container environments -> ECS resources – Running containers locally is possible
- 31. Docker implementations • Amazon EC2 Container Service: – Container management service for Docker – Description language for applications (“task definition”) – Supports batch mode (taks that only run once) – Supports long running taks – ELB support on container level
- 33. Docker implementations • xplat-cli: – Support for: • Ubuntu from Azure Image Gallery • Custom Linux image with Azure Linux VM Agent – Similar to plain EC2 approach
- 34. Docker implementations • xplat-cli: – azure vm docker create – Docker remote API
- 36. Deployment
- 37. Deployment Amazon Cloud Workstation GitHub Jenkins Python/Boto Docker Image Docker Registry S3 bucket with Docker Images
- 38. Deployment
- 39. Deployment
- 40. Deployment • AWS OpsWorks: – Docker Chef Cookbook by AWS – Add a new stack containing the Cookbook – Add a new layer and add custom recipes – Docker Images are build in the Cookbook – Use custom JSON to modify stack configuration
- 41. Deployment • Amazon Elastic Beanstalk: – Create an application using console or CLI – Create Dockerrun.aws.json to deploy a Docker container – Create a ZIP-file containing application files (optional)
- 42. Deployment 1. Create a cluster 2. Create a task definition 3. Create a service 4. Configure cluster 5. Run task
- 43. Deployment
- 44. Deployment • Remote API is exposed docker --tls -H tcp://<vm-name- you-used>.cloudapp.net:4243 info
- 45. Deployment • Useful Docker clients: – Spotify Docker Client – docker-py
- 46. Deployment
- 47. Deployment
- 48. Q&A