Mesos Networking

Christos(Kozyrakis,(Spike(Curtis
KapilArya,(Connor(Doyle,(NiklasNielsen,(TarakParekh,(Alex(Pollitt
Mesos Networking

The/State/of/Mesos Networking
Containers(share(the(slave’s(IP(address
Containers(can(use(any(port(on(the(slave
Service(discovery(using(perAslave(proxies
localhost:8888(on(any(slave(redirects(to(a(specific(service

This/was/OK/Initially
For(clusters(where(
– a(single(framework(manages(all(services
– there(are(only(a(few,(longArunning(services
– there(is(a(single(version(of(each(service

But/it’s/Problematic/Now
For(clusters(where
– services(are(launched(by(tens(of(frameworks
– there(are(thousands(of(services(with(high(churn(
– multiple(version(of(each(service(
prod/test/dev,(US/AMEA/Asia,(…(

Problem/#1:/Port/Conflicts
If(two(apps(want(to(use(same(port(on(a(slave,(one(fails(to(start
Alternative:(port(isolator(enforces(nonAoverlapping(port(ranges
! service(discovery(problem(for(the(app(that(does(not(get(standard(port
Alternative:(bridged(networking
! service(discovery(problem(for(the(app(behind(the(bridge

Problem/#2:/Service/Discovery
How(do(multiple(frameworks(manage(proxy(settings?
How(do(clients(know(which(version(of(a(service(is(at(each(port?
Do(we(update(the(proxies(in(10K(slaves(every(time(a(service(starts?

Problem/#3:/No/Isolation
How(do(we(stop(a(test(app(from(connecting(with(a(prod(app?(
How(we(isolate(different(users,(services,(or(divisions?(
How(do(we(stop(DoS attacks(within(the(cluster?

Mesos Networking/Redux
PerAcontainer(IP(addresses
Routable(within(and,(if(needed,(outside(the(cluster
No(port(conflicts(
DNSAbased(service(discovery
Discovery(using(hostnames((A(&(SRV(records,(HTTP(interface)
Network(isolation
Based(on(coarseAgrain(or(fineAgrain(security(policies

Implementation
One(feature(set,(many(pluggable(implementations
Different(network(virtualization(technologies((L2(or(L3)
Different(IP(address(management(schemes
Different(DNS(servers
First(implementation(based(on(Project(Calico
L3Abased(network(virtualization(&(isolation
Simple,(scalable,(openAsource

IP
Service
Router
Router
Router
BGP BGP
IP
Service
IP
Service
IP
Service
IP
Service
IP
Service
IP
Service
IP
Service
Build/the/DC/network/like/the/Internet

IP
Service
Router
Router
Router
BGP BGP
IP
Service
IP
Service
IP
Service
IP
Service
IP
Service
IP
Service
IP
Service
Mesos Slave
Build/the/DC/network/like/the/Internet
Mesos Slave

Mesos,Slave
Executor,Namespace
Root,Namespace
eth0
eth0 cali34
192.168.0.45
10.0.0.1
Executor,Namespace
eth0 cali89
10.0.0.2
Linux/Kernel/Routing
(you(already(have(this!)
default(via(192.168.0.1(dev eth0(
192.168.0.0/24( dev eth0((src 10.0.2.15((
10.0.0.1/32(dev cali34(scope(global
10.0.0.2/32(dev cali89(scope(global
10.0.1.40/32(via(192.168.0.29(dev eth0
10.0.2.53/32(via(192.168.0.131(dev eth0
vethpair((kernel(version(2.6.24+)
Containers(on(
other(slaves
IP
Calico/Data/Plane

Mesos,Slave
Executor,Namespace
Root,Namespace
eth0
eth0 cali34
192.168.0.45
10.0.0.1
Executor,Namespace
eth0 cali89
10.0.0.2
IP
Linux/Kernel/Filtering/ (iptables)
(you(already(have(this!)
Per$container+distributed+firewall
Calico/Data/Plane

Mesos,Slave
Executor,Namespace
Root,Namespace
eth0
eth0 cali34
192.168.0.45
10.0.0.1
Executor,Namespace
eth0 cali89
10.0.0.2
IP
Felix
Route
ReflectorBGP
Client
Calico/Control/Plane

Mesos – Calico/Integration
Networking(isolator(
Calico(IP(address(management(– IPAM((plugAin)
Calico(network(virtualizer (plugAin)(
Master(cleanup(module(

Update'
task'state
Networking/Workflow
PlugQin/(Calico)SlaveMasterFramework
IPAM
Network
virtualizer
Get,IP
Isolator
module
Isolate,(IP,,policy)
Cleanup
module
Launch,task,(policy)
Launch,task,(policy)
Task,update,(IP)
Task,update,(IP)
Mesos module
Network(plugAin

MesosQDNS
Mesos
Master
Slave Slave Slave Slave Slave…
Mesos
DNS
Watch(ZK(for
master(changes
Pull(task(state
Generate( DNS(records
DNS(&(HTTP
based(discovery
nginx_prod.marathon.mesos ! 10.13.17.95
_nginx_prod._tcp.marathon.mesos !10.13.17.95:8181

Networking/Demo
Mesos cluster(with(2(slaves
Launching(4(probe(tasks
Each(probe(listens(to(port(9000(
Each(probe(tries(to(reach(all(other(probes
We(want(all(4(to(launch(successfully((no(port(conflicts)
We(want(to(isolate(them(into(two(groups(of(2(probes

Roadmap
Code(release((open(source)
Integration(with(Mesosphere(DCOS
Interfaces(for(coarseAgrain(and(fineAgrain(isolation(policies
Other(plugAin(implementations
Flexible(task(naming(in(MesosADNS
Network(QoS

Summary
Mesos networking(features
PerAcontainer(IP(addresses
DNSAbased(service(discovery
Network(isolation
1st implementation(using(Project(Calico(
Try(it(and(contribute!

References
https://mesosphere.com/
http://www.projectcalico.org/
https://github.com/mesosphere/netAmodules
https://github.com/mesosphere/mesosAdns

Mesos Networking

More Related Content

Viewers also liked (11)

Similar to Mesos Networking (10)

More from Mesosphere Inc. (20)

Recently uploaded (20)

Mesos Networking