OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
1 like96 views
The document outlines the architecture and functionality of Apache Mesos, highlighting its resource management capabilities for diverse frameworks and the evolution of its ecosystem. It discusses container management, network isolation strategies like Project Calico, and persistent storage solutions, emphasizing Mesos's role as a powerful datacenter operating system. Key features include dynamic resource sharing, scalability, and support for various programming languages and applications.
![© 2015 Mesosphere, Inc. All Rights Reserved. 41
● Create/mount external volumes at task startup
● Exposes existing Docker Volume Driver to non-docker tasks
● e.g. RexRay
Docker Volume Driver Isolator Module
{
"id": "my-marathon-app",
"cmd": "while [ true ] ; do touch /var/lib/rexray/volumes/test12345/hello ; sleep 5 ;
done",
…,
"env": {
"DVDI_VOLUME_NAME": "test12345",
"DVDI_VOLUME_OPTS": "size=5,iops=150,volumetype=io1,newfstype=xfs,
overwritefs=true"
}
}](https://image.slidesharecdn.com/joergschad-mesosandthearchitectureofnewdatacenter-160502102054/85/OSDC-2016-Mesos-and-the-Architecture-of-the-New-Datacenter-by-Jorg-Schad-41-320.jpg)
OSDC 2016 - Mesos and the Architecture of the New Datacenter by Jörg Schad
- 2. 2
- 3. Flink Cassandra Rails Spark memcached
- 4. Flink Cassandra Rails Spark memcached
- 5. 5 A naive approach to handling varied app requirements: static partitioning. This can cope with heterogeneity, but is very expensive. KEEP IT STATIC time
- 6. 6 Maintaining sufficient headroom to handle peak workloads on all partitions leads to poor utilization overall. KEEP IT STATIC time
- 7. 7 Multiple frameworks can use the same cluster resources, with their share adjusting dynamically. SHARED RESOURCES time
- 8. Run everything in containers!
- 9. © 2015 Mesosphere, Inc. All Rights Reserved. What about container management? 9
- 10. © 2015 Mesosphere, Inc. All Rights Reserved. What about container management? 10
- 12. 12 THE BIRTH OF MESOS TWITTER TECH TALK The grad students working on Mesos give a tech talk at Twitter. March 2010 APACHE INCUBATION Mesos enters the Apache Incubator. Spring 2009 CS262B Ben Hindman, Andy Konwinski and Matei Zaharia create “Nexus” as their CS262B class project. MESOS PUBLISHED Mesos: A Platform for Fine-Grained Resource Sharing in the Data Center is published as a technical report. September 2010 December 2010 DC/OS April 2016
- 13. 13 Sharing resources between batch processing frameworks ● Hadoop ● MPI ● Spark What does an operating system provide? ● Resource management ● Programming abstractions ● Security ● Monitoring, debugging, logging TECHNOLOGY VISION
- 14. • A top-level Apache project • A cluster resource negotiator • Scalable to 10,000s of nodes • Fault-tolerant, battle-tested • An SDK for distributed apps • Native Docker support 14 Apache Mesos
- 15. © 2015 Mesosphere, Inc. All Rights Reserved. 2-Level Scheduling 15 Launch Tasks Scheduler(s) AllocationUser Work Resource Offers
- 17. 17 ARCHITECTURE MESOS FUNDAMENTALS ● Agents advertise resources to Master ● Master offers resources to Framework ● Framework rejects/uses resources ● Agents report task status to Master
- 18. 18 ARCHITECTURE MESOS FUNDAMENTALS ● Agents advertise resources to Master ● Master offers resources to Framework ● Framework rejects/uses resources ● Agents report task status to Master
- 19. 19 ARCHITECTURE MESOS FUNDAMENTALS ● Agents advertise resources to Master ● Master offers resources to Framework ● Framework rejects/uses resources ● Agents report task status to Master
- 20. 20 ARCHITECTURE MESOS FUNDAMENTALS ● Agents advertise resources to Master ● Master offers resources to Framework ● Framework rejects/uses resources ● Agents report task status to Master
- 21. 21 ARCHITECTURE MESOS FUNDAMENTALS ● Agents advertise resources to Master ● Master offers resources to Framework ● Framework rejects/uses resources ● Agents report task status to Master
- 22. 22 YARN
- 23. 23 YARN
- 24. 24 Distributed Systems SDK„Developers will ask for an API not for a VM to run their software“
- 25. 25 ● Focus on application logic, not on data-center structure ● Avoid networking-related code ● Reuse of built-in fault-tolerance and high availability ● Reuse distributed (infrastructure) frameworks (e.g., storage) Distributed System SDK Mesos
- 26. 26 Scheduler Interface ● Utilise resources ● ResourceOffer ● React on Tasks Events ● TaskStatus Updates Executor Interface (optional) ● Task life-cycle and monitoring ● Provided executors ● Command Executor ● Docker Executor First Framework/Service Mesos
- 27. 27 Introduction Framework ● Different languages ● Java, Python, C++, GO, … ● Scheduler + Executor Try it yourself! https://github.com/mesosphere/RENDLER Rendler Mesos
- 28. © 2015 Mesosphere, Inc. All Rights Reserved. 28 Container Networking
- 29. 29 Containers isolate tasks on the agent, but what about their communication? The status quo in a Mesos cluster: one IP per agent. Many containers per agent: they must share a single IP. CONTAINER NETWORKING Agent ContainerContainer ContainerContainer ContainerContainer
- 30. 30 This causes headaches: ● Port conflicts ● Security compromises ● Performance ● Service discovery CONTAINER NETWORKING Agent ContainerContainer ContainerContainer Web serviceWeb service
- 31. 31 This causes headaches: ● Port conflicts ● Security compromises ● Performance ● Service discovery CONTAINER NETWORKING Agent ContainerContainer ContainerContainer Test serviceProd. service
- 32. 32 This causes headaches: ● Port conflicts ● Security compromises ● Performance ● Service discovery CONTAINER NETWORKING Agent ContainerContainer ContainerContainer ContainerContainer
- 33. 33 CONTAINER NETWORKING This causes headaches: ● Port conflicts ● Security compromises ● Performance ● Service discovery Agent Container Container Agent Container Container
- 34. 34 NETWORK ISOLATION Segregating containers’ network traffic can solve these problems in an elegant, maintainable way.
- 36. 36 NETWORK ISOLATION Segregating containers’ network traffic can solve these problems in an elegant, maintainable way. Implemented as Mesos modules: ● Project Calico ● Port-mapping isolation There are alternative solutions as well!
- 37. 37 CALICO NETWORK ISOLATION Calico Network Virtualizer & IP Address Manager: ● Pure Layer-3 solution ● Uses iptables to route container traffic ● Provides security policies ● Advertises routes to local containers via BGP ● Can assign IP-per-container
- 38. © 2015 Mesosphere, Inc. All Rights Reserved. 38 Persistent Frameworks
- 39. © 2015 Mesosphere, Inc. All Rights Reserved. 39 My Task just died on the Mesos Agent* ● Problem: No guarantees for reoffered resources ● Dynamic Reservations (MESOS-2018) ● Problem: Task’s sandbox is garbage collected ● Persistent Volumes (MESOS-1554) Persistence Primitives
- 40. © 2015 Mesosphere, Inc. All Rights Reserved. 40 Storage backed by third party storage services ● E.g. EMC ScaleIO, EC2, NFS based storage system ● Not Mesos Managed (≠remote storage) ● Not tied to particular agent How can I access those volumes (esp. without docker)? External Volumes
- 41. © 2015 Mesosphere, Inc. All Rights Reserved. 41 ● Create/mount external volumes at task startup ● Exposes existing Docker Volume Driver to non-docker tasks ● e.g. RexRay Docker Volume Driver Isolator Module { "id": "my-marathon-app", "cmd": "while [ true ] ; do touch /var/lib/rexray/volumes/test12345/hello ; sleep 5 ; done", …, "env": { "DVDI_VOLUME_NAME": "test12345", "DVDI_VOLUME_OPTS": "size=5,iops=150,volumetype=io1,newfstype=xfs, overwritefs=true" } }
- 42. 42
- 43. 43 DC/OS is … ● 100% open source (ASL2.0) + A big, diverse community ● An umbrella for ~30 OSS projects + Roadmap and designs + The build tool chain + Docs and tutorials ● Not limited in any way ● Familiar, with a few new features
- 44. Datacenter Operating System (DC/OS) Distributed Systems Kernel (Mesos) DC/OS ENABLES MODERN DISTRIBUTED APPS Big Data + Analytics EnginesMicroservices (in containers) Streaming Batch Machine Learning Analytics Functions & Logic Search Time Series SQL / NoSQL Databases Modern App Components Distributed systems kernel to abstract resources Ecosystem of frameworks & apps Consistent architecture to run on top of kernel User Interface (GUI & CLI) Core system services (e.g., distributed init, cron, service discovery, package mgt & installer, storage) Any Infrastructure (Physical, Virtual, Cloud) 44
- 45. 45
- 46. 46 THE UNIVERSE