Docker Clustering - Batteries Included
6 likes1,035 views
The document discusses Docker clustering and its components, particularly focusing on Swarm, which is a native clustering tool for Docker. It provides an overview of Docker, including installation and use of Swarm, managing resources, and handling containers with various constraints and affinities. Additionally, it highlights storage drivers and future features of Swarm such as rescheduling policies and leader election.
![Minimal Image (small tangent)
A Whopping 7.19 MB
The Dockerfile -->
FROM scratch
COPY ./swarm /swarm
COPY ./certs/ca-certificates.crt
/etc/ssl/certs/ca-certificates.crt
ENV SWARM_HOST :2375
EXPOSE 2375
VOLUME /.swarm
ENTRYPOINT ["/swarm"]
CMD ["--help"]](https://image.slidesharecdn.com/untitled-150511111724-lva1-app6892/85/Docker-Clustering-Batteries-Included-12-320.jpg)
Docker Clustering - Batteries Included
- 1. Docker Clustering batteries included, but removable Jessie Frazelle @frazelledazzell Qcon London: March 5th , 2015
- 2. InfoQ.com: News & Community Site • 750,000 unique visitors/month • Published in 4 languages (English, Chinese, Japanese and Brazilian Portuguese) • Post content from our QCon conferences • News 15-20 / week • Articles 3-4 / week • Presentations (videos) 12-15 / week • Interviews 2-3 / week • Books 1 / month Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations /docker-clustering
- 3. Presented at QCon London www.qconlondon.com Purpose of QCon - to empower software development by facilitating the spread of knowledge and innovation Strategy - practitioner-driven conference designed for YOU: influencers of change and innovation in your teams - speakers and topics driving the evolution and innovation - connecting and catalyzing the influencers and innovators Highlights - attended by more than 12,000 delegates since 2007 - held in 9 cities worldwide
- 4. Outline - Shortest ever intro to Docker - Intro to Swarm (Docker clustering) - Demo of Swarm - Future of Swarm
- 5. What is Docker? Docker is a runtime for containers. Whoa what’s a container? A container is a concept made from linux namespaces, cgroups, & pivot roots.
- 6. Outline - Shortest ever intro to Docker - Intro to Swarm (Docker clustering) - Demo of Swarm - Future of Swarm
- 7. Intro to Swarm Native Clustering for Docker Serves the standard Docker API Transparently scale Docker API consumers to multiple hosts
- 10. Schedulers out-of-the-box bin-packing (native) options random (native) mesos (coming soon, in the works)
- 11. How to use Swarm # installing swarm $ docker pull swarm # create a cluster $ docker run --rm swarm create 6856663cdefdec325839a4b7e1de38e8 # <- unique <cluster_id> # on each of your nodes, start the swarm agent $ docker run -d swarm join --addr=<node_ip:2375> token://<cluster_id>
- 12. Minimal Image (small tangent) A Whopping 7.19 MB The Dockerfile --> FROM scratch COPY ./swarm /swarm COPY ./certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt ENV SWARM_HOST :2375 EXPOSE 2375 VOLUME /.swarm ENTRYPOINT ["/swarm"] CMD ["--help"]
- 13. How to use Swarm # start the manager on any machine or your laptop $ docker run -d -p <swarm_port>:2375 swarm manage token://<cluster_id> # list nodes in your cluster $ docker run --rm swarm list token://<cluster_id> <node_ip:2375>
- 14. Using the Docker CLI + Swarm # use the regular docker cli $ export DOCKER_HOST=tcp://<swarm_ip:swarm_port> $ docker info $ docker ps $ docker logs ... # manage resources $ docker run -m 1g $ docker run -c 1 $ docker run -p 80:80
- 15. Constraints # standard from docker info # (storagedriver, executiondriver, kernelversion, operatingsystem) $ docker run -e constraint:operatingsystem=debian ... $ docker run -e constraint:storagedriver=btrfs ... # custom with host labels $ docker -d --label init=systemd ... $ docker -d --label init=sysvinit ... $ docker run -e constraint:init!=systemd ... $ docker -d --label environment=production ... $ docker run -e constraint:environment=production ...
- 16. Affinity # containers $ docker run -d --name web1 -p 80:80 nginx $ docker run -d --name stats -e affinity:container==web1 stats # images $ docker run -d -e affinity:image==redis redis $ docker run -d -e affinity:image==nginx nginx
- 17. Other Filters # ports $ docker run -d --name web1 -p 80:80 nginx $ docker run -d --name web2 -p 80:80 nginx ^ defaults on different host # dependency $ docker run --volumes-from some-container ... $ docker run --link some-container:alias ... $ docker run --net container:some-container ...
- 18. Outline - Shortest ever intro to Docker - Intro to Swarm (Docker clustering) - Demo of Swarm - Future of Swarm
- 19. The Servers
- 20. Storage Drivers (medium tangent) Ubuntu Host --> AUFS Fedora Host --> Device Mapper Debian Host --> Overlay
- 21. AUFS - First storage driver implemented - Ubuntu uses it in their default kernel for Live CD where root filesystem is COW (copy-on-write) between CD/DVD/USB Pitfalls: not in mainline kernel
- 22. Device Mapper - Used by RedHat, default to Fedora - In mainline kernel - Creates “pools” of blocks Each container & each image gets its own block device - Each time a new block (or a copy-on- write block) is written, a block is allocated from the pool
- 23. Device Mapper Pitfalls: By default, Docker puts data and metadata on a loop device backed by a sparse file Which is cool but has terrible performance. Each time a container writes to a new block... a block has to be allocated from the pool... and when it's written to... a block has to be allocated from the sparse file... and sparse file performance is not the greatest
- 24. Overlay - The hero we all deserve - In mainline kernel (>=3.18) - works a lot like AUFS in that it does not need its own partition and works out-of-the-box Pitfalls: requires kernel >= 3.18
- 25. BTRFS (not used for demo, but important) - In mainline kernel - Does copy-on-write at filesystem level integrates the snapshot and block pool management features at the filesystem level, instead of the block device level Pitfalls: have to setup partition
- 26. Back to the demo...
- 27. Outline - Shortest ever intro to Docker - Intro to Swarm (Docker clustering) - Demo of Swarm - Future of Swarm
- 28. Future of Swarm - Rescheduling Policies - More backend drivers, Mesos, etc - Leader Election (Distributed State) - Keeping up to date feature-wise with things added to the engine
- 29. Fin. Jessie Frazelle @frazelledazzell jess@docker.com
- 30. Watch the video with slide synchronization on InfoQ.com! http://www.infoq.com/presentations/docker- clustering