VMworld 2015: VMware NSX Deep Dive

Editor's Notes

• #10: Explain each module in little detail.. Showing the value of each feature Port Security : Provides DHCP snooping used by VXLAN module; Port Security – IP spoof guard VXLAN – VTEP ; MTEP – Multicast replication; ARP Proxy Distributed Router – East – West traffic between VXLAN vWires had to go through Edge gateway Distributed Firewall – Better performance Message Bus provides a new communication channel that allows direct communication from NSX manager to the host User World Agent – Communicates with the controller one one side and the kernel modules on the other
• #11: TBD: Properties & Benefits (VM, separation of MP/CP/DP, scale-out, no multicast) - Functions: overlay, L2, L3 dataplane programming Provides control plane to distribute Logical Switching and Logical Routing network information to ESXi hosts NSX Controllers are clustered for scale out and high availability Network information is sliced across nodes in a Controller Cluster Removes Dependencies on Multicast from Physical Networks Provides suppression of ARP broadcast traffic in logical networks
• #12: Functionality NSX for vSphere centralized management plane 1:1 mapping between an NSX Manager and vCenter Server Provides the management UI and API for NSX Configures Controller Cluster Generates certificates to secure control plane communications Installs Logical Switching, Distributed Routing and Firewall kernel modules on ESXi hosts Operationally: Deploys NSX Controller and NSX Edge Virtual Appliances (OVF) vSphere Web Client Plugin Host configuration includes Distributed Firewall and NSX Edges
• #13: NSX Control Plane communication occurs over the management network. The Control plane is protected by: Certificate based authentication SSL NSX Manager generates self-signed certificates for each ESXi Hosts and Controllers These certificates are pushed to the controller and ESXi hosts over secure channels Mutual authentication occurs by verifying these certificates
• #17: Ethernet in IP overlay network Entire L2 frame encapsulated in UDP 50+ bytes of overhead 24 bit VXLAN Network Identifier 16 M logical networks VXLAN can cross Layer 3 network boundaries Overlay between ESXi hosts VMs do NOT see VXLAN ID VTEP (VXLAN Tunnel End Point) VMkernel interface which serves as the endpoint for encapsulation/de-encapsulation of VXLAN traffic Technology submitted to IETF for standardization With Cisco, Citrix, Red Hat, Broadcom, Arista and Others
• #20: VXLAN traffic uses a vmknic which provides VXLAN Virtual Tunnel End Point (VTEP) functionality A single dvPortGroup per VDS is created for all VTEPs A logical switch is a L2 broadcast domain implemented using VXLAN A dvPortGroup is created for each logical switch Provides local switching & isolation VXLAN logical switches can also span multiple VDS
• #21: Support for multiple VXLAN vmknics per host to provide additional options for uplink load balancing DSCP & COS Tag from internal frame copied to external VXLAN encapsulated header Support for Guest VLAN tagging vMotion callback Dedicated TCP/IP stack for VXLAN Ready for VXLAN hardware offloading to network adapters A highly available and secure control plane to distribute VXLAN network information to ESXi hosts Removes dependency on multicast routing/PIM in the physical network Suppress broadcast traffic in VXLAN networks ARP Directory Service & NSX Controller
• #22: In Unicast or Hybrid mode, each ESXi host will select one VTEP in every remote segment from its VTEP mapping table as a proxy. This is per VNI (balances load across proxy VTEPs) In Unicast Mode this proxy is called a UTEP – Unicast Tunnel End Point In Hybrid Mode it is an MTEP – Multicast Tunnel End Point This list of UTEPs/MTEPs is then synced to each VTEP If a UTEP or MTEP leaves a VNI the host will be updated by the Controller and then select a new proxy within the segment Optimized Replication – VTEPs perform software replication of BUM traffic to local VTEPs and one UTEP/MTEP per remote segment The VXLAN header format has been updated in NSX for vSphere A new REPLICATE_LOCALLY bit is used in the VXLAN header for Unicast and Hybrid Modes When an UTEP or MTEP receives a unicast frame with the REPLICATE_LOCALLY bit set it is responsible for re-injecting the frame to the local transport network The behavior of the proxy depends on the traffic replication mode UNICAST MODE Source VTEP role Replicates an encapsulated frame to each remote UTEP via unicast Also replicates the frame to each active VTEP in the local segment UTEP role Delivers a copy of the de-encapsulated inner frame to local VMs Sends the replicated frame to all VTEPs in the local segment Unicast Mode has minimal dependencies on physical network, but the overhead increases as environment scales Configurable per VNI during logical switch provisioning Multicast Addresses are not required in Unicast Mode Hybrid Mode Source VTEP role Replicates an encapsulated frame to each remote MTEP via unicast Also replicates the frame locally via multicast MTEP role Delivers a copy of the de-encapsulated inner frame to it’s local VMs Sends the replicated frame to the local segment using the multicast address assigned to the VNI Hybrid Mode leverages the physical network to reduce the overhead of traffic replication. Overhead increases as VXLAN segments are added Again configurable per VNI Multicast Addresses are required in Hybrid Mode To reduce dependencies on the physical network, ESXi hosts will now send IGMP joins & reports An IGMP Querier on the physical network per transport network is still recommended Multicast Mode Source VTEP role Replicate the VXLAN frame locally via multicast L2 multicast will deliver to all local destination VTEPs No UTEP or MTEPs required Multicast routing will handle delivery to all remote segments Multicast Mode is entirely reliant on multicast support in the physical network for local and remote traffic replication Configurable per VNI Multicast Addresses are required
• #28: Logical Interfaces (LIFs) on a Distributed Logical Router Instance There are internal LIFs and uplink LIFs VM Default Gateway traffic is handled by LIFs on the appropriate network LIFs are distributed across every hypervisor prepared for NSX Up to 1000 LIFs can be configured per Distributed Logical Router Instance 8 Uplink 992 Internal An ARP table is maintained per LIF vMAC is the MAC address of an internal LIF vMAC is same across all hypervisors and it is never seen by the physical network (only by VMs) pMAC is the MAC address of the dvUplink on each host through which traffic flows to the physical network
• #30: The Distributed Logical Router Control Plane is provided by a per instance DLR Control VM and the NSX Controller Supports Dynamic Routing Protocols OSPF BGP Communicates with NSX Manager and Controller Cluster NSX Manager sends LIF information to the Control VM and Controller Cluster Control VM sends Routing updates to the Controller Cluster DLR Control VM and NSX Controller are not in the data path High availability supported through Active-Standby configuration
• #31: VMware NSX provides a faithful reproduction of Network & Security Services in Software VXLAN is the overlay technology empowering those virtual networking capabilities Logical Routing allows for communication between virtual workloads belonging to separate IP subnets Distributed Routing optimizes traffic flows for East-West communication inside the Data Center Centralized Routing handles on-ramp/off-ramp communication with the external physical network Multiple logical topologies can be built combining NSX DLR and Edge functional components Each logical routing components can be deployed redundantly to guarantee a fully resilient design
• #32: Enterprise topology, optimizes as much E-W traffic as possible by adding as many LIFs on the one DLR instance. Unless L2 spans across all clusters it is still common to uses NSX edge gateway even if it’s aggregating the one Distributed Logical Router instance so that VXLAN LIFs are used.
• #35: While we’re focusing on Firewalling here, note that NSX is the security platform offering Antivirus, Intrusion Prevention, Vulnerability Management, Identity and Access Management, Security Policy Management, DLP File Integrity Monitoring and more…
• #40: Traffic Redirection rules are configured using Service Composer (within Security Policy definition) or using Partner Security Services (DFW UI - NSX 6.1). Filtering Module is an extension of DFW. Filtering Module rules are configured within Security Policy definition (Service Composer menu). Traffic Redirection Module define which traffic are steered to Partner Services VM: Using Service Composer: ANY -> SG, SG1 -> SG2, SG -> ANY ANY, TCP/UDP destination port, TCP/UDP source port Predefined Services application & protocols (NSX 6.1) Using Partner Security Services (under DFW UI)
• #44: Challenges Applications are not mobile as they are tied to a physical LB instance Multi-tenancy ? Configuration automation