OSDC 2014 ONIE by Nat Morris

Editor's Notes

• #4: Cumulus Network’s HCL focused on fixed boxes (Leaf/Spine)Same Broadcom silicon as Arista switches, same hardware performance at lower price point.Arista has additional hardware platforms for special purposes Choice – Cumulus focuses on breadth of platforms/vendors for best of breed.Arista supports black boxesArista and supports many different configurations – Cumulus doesn’t need differentiated price points for low end configurations, they are already cheaperCumulus Linux is a Linux OS, and network services apps run on top of it are very rich.Arista in contrast is a Linux-based OS, EoS integrates all apps in one image and control is limited to some Linux containerCloud Networking designs – includes L2/Host Multi-homing*, L3/ECMP, L2 over L3 VXLAN.Customers are moving to L3 CLOS fabrics so L2/Host multi-homing is all that’s needed, not MLAGOrchestration – Comprehensive set of tools today on par with Arista and rapid innovationOur model offers the same Orchestration tool and more due to rapid pace of innovation (ex. Midokura)OpenFlow is supported with other OS such as Big SwitchAutomation.Cumulus Linux has Zero Touch Provisioning, automated install, better DevOps integration (due to unmodified Linux/scripting languages)Application visibility – Leverage server style tools & hardware counters/functionalityArista may have stronger networking tracers, advanced mirroring (DANZ), advanced congestion management (LANZ) tools today. Congestion management/counters will be enabled with switchd file system, more can be done for simplification, but similar capability can be enabled through scriptingProgrammable foundation – drivers abstractions, eAPI, Unmodified Linux Cumulus Linux drivers abstractions are unchanged (in contrast Arista uses sysDB to provide visibility to their own driver), Cumulus Linux networking data structures are unchanged (Arista uses its own so user is limited to management plane/control plane box changes)
• #8: Just like BIOS and PXE allows you to install an OS on a server using a remote image, the combination of U-Boot and ONIE allows that for bare metal switches.We require ONIE preloaded on HCL because U-Boot is different across vendor devices, and U-Boot itself is not very user friendly.We created ONIE and gave it to the Open Compute Project (OCP); it facilitates easy network OS installation of not just Cumulus Linux (Pica8 is a competitive example). Now you have your choice of installing whatever OS you want, not just what comes with the switch (e.g. Cisco IOS– OEM example, or FASTPATH– Broadcom’s OS).Think of ONIE as PXE on steroids. ONIE is a small BusyBox Linux distribution, with a bunch of fetch and execution Bash scripts. It leverages modern ways of discovering networks using what was built into Linux—e.g., IPv6 neighbor discovery, DHCPv6, DHCPv4.U-Boot is very good at probing the bus. U-Boot takes about 1MB. It has boot flash that’s dedicated to booting the hardware, separate from the Operating System flash. ONIE is a way to build on top of this. Takes about 3.5 MB.ONIE is extremely well documented and flexible, and embraced by the open source community. (Source is on GitHubsince summer 2013).
• #31: Within Linux is a construct called netlink,the communication channel between user space and Linux kernel. Everything we see in the User Space box talks to the Kernel through netlink (not shown on diagram). switchd snoops the netlink traffic and can react (e.g. whenever you add or remove a route)Color decode:Green with orange border pushes things down to the kernel