OSDC 2014: Nat Morris - Open Network Install Environment
0 likes517 views
The document discusses the Open Network Install Environment (ONIE), which provides an environment for network operating system (NOS) installer discovery and execution, akin to a pre-installed kickstarter using a modern Linux kernel. It highlights ONIE's anatomy, installation mechanics, ongoing development, and its integration with bare metal switches, along with lessons learned from hardware development. Additionally, the document provides resources and examples of compatible hardware for users implementing ONIE.
OSDC 2014: Nat Morris - Open Network Install Environment
- 1. Open Network Install Environment NETWAYS OSDC 2014, Berlin Nat Morris 9th April 2014
- 2. cumulusnetworks.com 2 • Overview • What Is ONIE? • Lessons Learned • ONIE Development • Demo Agenda
- 3. IP Fabric Networking Landscape cumulusnetworks.com 3 Network Hardware NetworkOS Open Closed
- 4. The Expanding Landscape hardware operating system appapp hardware operating system app app Single Vendor Blob Multi-Vendor Ecosystem app app cumulusnetworks.com 4
- 5. Understanding Characteristics of a Leaf Switch 5cumulusnetworks.com 10/40 Gigabit spine uplink ports Serial console port Ethernet Out-of-Band Management Port 1/10 Gigabit Ethernetnetwork access ports * SFP+ ports can be grouped together into a single QSFP 40G port via reverse connecting breakout cable options * QSFP ports can be broken out into four SFP+ ports via copper or optical transceiver options
- 6. Understanding Characteristics of a Spine Switch 6cumulusnetworks.com Serial console port Ethernet Out-of-Band Management Port * QSFP ports can be broken out into four SFP+ ports via copper or optical breakout cable options 40G / 10G Fabric ports
- 7. Bare Metal Switch Provisioning § Similar approach to installing OS on server § BIOS + PXE = U-Boot + ONIE (Open Network Install Environment) § Supported hardware (HCL) preloaded with ONIE § ONIE available on GitHub • http://onie.github.io/onie/ cumulusnetworks.com 7 bare metal server operating system app app app BIOS and PXE bare metal switch operating system app app app U-Boot and ONIE
- 10. What Is ONIE? § Network OS Install Environment § Provides an environment for network OS installer discovery and execution § Like a pre-installed kickstarter § Defined by its behaviors § Implemented using a modern Linux kernel and BusyBox § An Open Source Project within OCP § http://www.onie.org/ § Evolving … cumulusnetworks.com 10
- 13. ONIE: Anatomy of a Network Switch cumulusnetworks.com 13 ( Management Interfaces ) ( Data Plane ) CPU SoC DRAM Boot Flash Mass Storage Switching ASIC Serial Console Ethernet Mgmt Port 10Gb Port 40Gb Port … 10Gb Port 40Gb Port … PCIe
- 14. ONIE: Uses Management Interfaces cumulusnetworks.com 14 ( Management Interfaces ) ( Data Plane ) CPU SoC DRAM Boot Flash Mass Storage Switching ASIC Serial Console Ethernet Mgmt Port 10Gb Port 40Gb Port … 10Gb Port 40Gb Port … PCIe
- 15. ONIE: Bare Metal Install – First Time Boot Up cumulusnetworks.com 15 Boot Loader (HW Vendor Supplied) ONIE (HW Vendor Supplied) Installer (OS Vendor) Boot Loader • Low Level boot loader, configures CPU complex • Loads and boots ONIE ONIE • Linux Kernel with Busybox • Configures management Ethernet interface • Locates and executes an OS installer • Provides tools and environment for installer OS Installer • Available from network or USB • Linux executable • Installs vendor OS into mass storage Network OS (OS Vendor Supplied) Fetches Installs
- 16. ONIE: Subsequent Reboots – Vendor’s OS is Already Installed cumulusnetworks.com 16 Boot Loader (HW Vendor Supplied) ONIE (HW Vendor Supplied) Boot Loader • Low Level boot loader, configures CPU complex • Loads and boots OS vendor’s installed OS Network OS • Configures Switching ASIC • Runs Network Protocols • Provides CLI Network OS (OS Vendor Supplied) ONIE • SSll exists, but is not used • Available for uninstall / re-‐install operaSons
- 17. ONIE: Network OS Installer Discovery and Install Behavior cumulusnetworks.com 17 Configure Network Interface Locate Installer Run Installer • Uses DHCPv4, DHCPv6 • Configures Ethernet interface for IPv4 / IPv6 • Configures DNS and hostname • Determines the locaSon of an installer executable • Examines local file systems, e.g. USB flash drives • Uses DHCP opSons, DNS Service Discovery, MulScast DNS and IPv6 Neighbors • Downloads installer via URL • Passes various environment variables to installer • Launches installer
- 18. Why Not Use PXE? cumulusnetworks.com 18 • IPv6 and HTTP out of the box • Use exisSng Linux device drivers. No need to write new ones • Integrated automaSon
- 19. ONIE: Other Behaviors § Provides a mechanism for a network OS to invoke the above behaviors. cumulusnetworks.com 19 Reinstall Remove currently installed OS and return to the “out of box” provisioning state Uninstall Completely wipe out everything, except ONIE Rescue Reboot box into ONIE for repair, debug and forensics Update Install a new ONIE version Diag Run HW Vendor’s diag (optional)
- 20. Lessons Learned § Hardware Platforms § Diverse mix of CPU complex designs, even within a single CPU family. § HW Vendors need freedom to customize ONIE § Need mechanism to run HW vendor diag § Recurring Themes § TLV based EEPROM format widely adopted § Common HW designs reduce development time cumulusnetworks.com 20
- 21. ONIE Ongoing Development § Support Multiple CPU Architectures § PowerPC – Today § x86 – VM Prototype available today § Thinking about ARM, MIPS § Maintain ONIE behaviors across architectures § Testing and Compliance § Enhance and extend regression test suites § Develop compliance test suites cumulusnetworks.com 21
- 22. ONIE Ongoing Development § New Features § DHCPv6 § DNS Service Discovery / Multicast DNS discovery § HW Vendor diagnostic § Releases § Quarterly release cadence § Ongoing maintenance § Enhancements cumulusnetworks.com 22
- 23. ONIE x86 Strategy § Use existing BIOS from hardware vendors § During manufacturing install GRUB2 and ONIE-x86 on the mass storage block device § NOS Installer adds partitions, installs software and updates GRUB2 configuration. cumulusnetworks.com 23
- 24. Block Device Partitioning § Partition using GUID Partition Table (GPT) format § GPT supported by Linux, GRUB, UEFI § Supports dual-booting multiple operating systems cumulusnetworks.com 24 Partition # Name R/W Notes 1 GRUB Boot r/o Used by GRUB and GPT 2 ONIE BOOT r/o ONIE Kernel 3 ONIE CONFIG r/w Configuration 4+ For NOS use
- 26. NOS Installer Duties § Create new GPT partition(s) § Create file systems on partition(s) § Install NOS files into partition(s) § Update ONIE-CONFIG using ONIE provided CLIs § onie-boot-entry-add! § onie-boot-entry-remove! § onie-boot-entry-show! § onie-boot-default! § onie-boot-update! cumulusnetworks.com 26
- 27. Resources § ONIE Websites § Main Page: http://www.onie.org/ § Source Code: https://github.com/onie/onie/ § Documentation: http://onie.github.io/onie/ cumulusnetworks.com 27
- 28. © 2013 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis. § Thank You! cumulusnetworks.com 28
- 29. Back Up Slides – Cumulus Background cumulusnetworks.com 29
- 30. Pushing Changes Down cumulusnetworks.com 30 CPU, RAM, Flash, etc. Switch Silicon Front Panel Ports lldpd Routing Tables ARP Table Devices Bridge FDB Filter Tables Bonds VLANs LinuxKernel Virtual Kernel Ports Bridging mstpd ACLRouting Suite Quagga snmpd vconfig iptable ebtable ip6tableiproute2 VXLAN Bridges Switch HAL brctl Switch Driver UserSpace Quagga daemon, Quagga.conf, and vtysh CLI and /etc/network/ interfaces switchd
- 31. Cumulus Linux Hardware Compatibility List Cumulus Networks and Partner Confidential 40G Portfolio Part number Description Merchant Silicon Cumulus Linux Release Dell S6000 32 x 40G-QSFP+ Trident II 2.1 or later Edge-Core AS6700-32X 32 x 40G-QSFP+ Trident II 2.0 or later Penguin Computing Arctica 3200XL 32 x 40G-QSFP+ Trident II 2.0 or later Quanta QCT T5032-LY6* 32 x 40G-QSFP+ Trident II 2.0 or later 10G Portfolio Part number Description Merchant Silicon Cumulus Linux Release Agema AG-7448CU 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident 1.5.0 or later Dell S4810 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident 2.0.x or later** Edge-Core AS5600-52X 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident+ 1.5.0 or later Edge-Core AS5710-54X 48 x 10G-SFP+ and 6 x 40G-QSFP+ Trident II 2.1 or later** Edge-Core AS5710-96X 96 x 10G-SFP+ and 8 x 40G-QSFP+ Trident II 2.1 or later** Penguin Computing Arctica 4804X 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident+ 1.5.1 or later Quanta QCT T-3048-LY2* 48 x 10G-SFP+ and 4 x 40G-QSFP+ Trident+ 1.5.0 or later Quanta QCT T5048-LY8* 48 x 10G-SFP+ and 6 x 40G-QSFP+ Trident II 2.1 or later** 1G Portfolio Part number Description Merchant Silicon Cumulus Linux Release Edge-Core AS4600-54T 48 x 1G-T and 4 x 10G-SFP+ Apollo2 2.0 or later Penguin Computing Artica 4804i 48 x 1G-T and 4 x 10G-SFP+ Triumph2 1.5.1 or later Quanta QCT T1048-LB9* 48 x 1G-T and 4 x 10G-SFP+ FireBolt3 1.5.0 or later 31