Inside Architecture of Neutron
- 1. Inside the Architecture of Neutron Mark McClain mmcclain (at) yahoo-inc.com
- 2. Why Create Neutron? • Rich Topologies • Technology Agnostic • Extensible • Advance Services Support • Load Balancing, VPN, Firewall
- 3. The Basics
- 4. What does the user see? Compute API Network API Storage APIGUI, CLI, API Libs KVM ML2 Plugin Ceph
- 5. Abstractions Net1 10.0.0.0/24 Nova Neutron L2 virtual network virtual port virtual server virtual interface (VIF) virtual subnet VM1 10.0.0.2 VM2 10.0.0.2
- 6. Architecture
- 7. Design Goals • Unified API • Small Core • Pluggable Open Architecture • Extensible
- 8. OpenStack The Operator View
- 9. Basic Deployment neutron-server L2 AgentL2 AgentL2 AgentL2 AgentL2 AgentL2 Agent L3 AgentL3 Agent L3 AgentL3 Agent Database L3 Agent DHCP Agent L2 Agent Message Queue Adv Services
- 10. neutron-server REST API SERVICE RPC SERVICE PLUGIN REST API SERVICE • REST API • HTTP(S) Python WSGI Application • Customary TCP port is 9696 • Exposes logical resources • networks, subnets, ports, etc • Request/Response Serialization
- 11. neutron-server REST API SERVICE RPC SERVICE PLUGIN RPC SERVICE • RPC Service • AMQP via Oslo messaging modules • Enables bidirectional agent communication • Optional
- 12. neutron-server REST API SERVICE RPC SERVICE PLUGIN • PLUGIN • Written in Python • Only one active • Must implement V2 API calls • Optional database access • Optional extension support
- 13. The Plugin ML2Plugin core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
- 14. The Plugin NeutronPluginBaseV2 NeutronDbPluginV2 ML2Plugin core_plugin = neutron.plugins.ml2.plugin.Ml2Plugin
- 15. Plugin Extensions • Add logical resources to the REST API • Discovered by server at startup • REST: /v2.0/extensions • Common Extensions • Binding, DHCP, L3, Provider, Quota, Security Group • Other Extensions • Allowed Addresses, Extra Routes, Metering
- 16. Monolithic Plugin • Full implementation of core resources • Two types: • Proxy • Direct control PLUGIN
- 17. ML2: Modular Layer 2 Plugin • Full V2 Plugin Implementation • Delegates calls to proper L2 drivers • Two kinds of drivers • Type Driver • Mechanism Driver Mech Mgr PLUGIN Type Mgr
- 18. L2 Agent
- 19. L2 Agent • Runs on hypervisor • Communicates with server via RPC • Watch and notify when devices added/removed • Wires new devices • Proper network segment • Security Group Rules
- 20. Dive Into the OVS Agent • OVS • What does it actually do? • How do we get isolation? • VLAN, Overlays: GRE, VXLAN • Processing loop
- 21. Linux Network Namespace • Isolated copy of network stack • private loopback • scope limited to namespace • can reuse addresses • Explicit configuration needed to connect • Processes can spawn within namespace lo eth1 eth0 lo eth1 eth0 lo eth1 eth0 Host A B br-int
- 22. L3 Agents
- 23. Network Node L3 Agent • Run on Network Node • Uses Namespaces • Metadata Agent (if enabled) Network Node Core Hypervisor Hypervisor Hypervisor
- 24. L3 Agent How it’s implemented • Manages Collection of Network Namespaces • Isolated IP Stacks • Forwarding Enabled • net.ipv4.ip_forward=1 • Static Routing • Metadata Proxy lo eth1 eth0 lo qg-2 qr-1 lo qg-b qr-e Host A B br-ex
- 26. Configuration Agents: DHCP • RPC based notifications • dnsmasq • Isolation Support via Network Namespaces • Multiple copies for HA
- 27. Configuration Agents: Metadata Proxy • Proxies Metadata requests to Nova • Routed Networks • process embedded in router • Non-routed Networks • static route redirects traffic running in DHCP namespace
- 28. Configuration Agents: Metadata Proxy curl http://168.254.169.254/openstack/latest/meta_data.json Tenant VM Nova Metadata Service Meta NS Proxy Metadata Agent Unix Domain Socket Management Network X-Router-Id: 2bc7c882-d612-438c-a334-0047f2b5c2d7 X-Forwarded-For: 10.0.0.1 X-Instance-ID: aaaaaaaa-aaaa-aaaa-aaaaaaaaaaaa
- 29. Booting a VM • nova boot
- 30. Booting a VM • nova boot • create port • notify DHCP of new port
- 31. Booting a VM • nova boot • create port • notify DHCP of new port • create device • new in Icehouse wait
- 32. Booting a VM • nova boot • create port • notify DHCP of new port • libvirt create device • new in Icehouse wait • wire port
- 33. Booting a VM • nova boot • create port • notify DHCP of new port • libvirt create device • new in Icehouse wait • wire port • boot
- 34. Load Balancer as a Service • Service Plugin • Driver based • Agent w/Driver • Agent communicates over RPC • Open Source requires namespaces • Others interact with other systems LB Agent HAProxy
- 35. VPN as a Service • Service Plugin • Driver based • Agent w/Driver • Communicates over RPC • Openswan L3 Agent Router Metadata Proxy VPN Driver
- 36. Firewall as a Service • Edgewall • Service Plugin • Driver based • Agent w/Driver • Communicates over RPC • Experimental L3 Agent Router Metadata Proxy Firewall Driver
- 37. Differences • Different Design Decisions • Sync with backend system • L2 Agent Optional • Not all implement same extensions
- 38. Summary Open vSwitch / Linux Bridge Ryu OpenFlow Controller • Unified API • Small Core • Pluggable Open Architecture • Multiple Vendor Support • Extensible
- 39. More Information • Cloud Administrator Guide • http://docs.openstack.org/admin-guide-cloud/content/ch_networking.html • Network v2.0 API • http://developer.openstack.org/api-ref-networking-v2.html
- 40. Questions?