XConn: Scalable Memory Expansion and Sharing for AI Computing with CXL Switches
- 1. Scalable Memory Expansion, Pooling and Sharing for AI Computing with CXL Switch XConn Technologies Inc. 1
- 2. The Ever Ballooning AI Models • Requiring scalable memory capacity and bandwidth • Requiring low latency and high bandwidth interconnect between heterogeous computing units https://medium.com/riselab/ai-and-memory-wall-2cb4265cb0b8
- 3. XConn’s CXL 2.0 Switch 2,048 GB/s total BW with 256 lanes Lowest port-to-port latency Lowest power consumption/port • Works with CXL 1.1 server processors, CXL memory devices • Future compatible with the upcoming CXL 2.0 processors • Works in hybrid mode (CXL/PCIe mixed) • CS (customer samples) available now, MP 2Q24 Reduced PCB area Lower TCO World’s First CXL2.0 (XC50256) & PCIe 5.0 (XC51256) switch IC
- 4. CMS: Scalable Memory Pooling & Sharing Enabled by CXL 2.0 Switch • One single XC50256 connects to 32 combined hosts/devices • Fully support CXL Fabric Manager • Support switch cascading for a larger size memory pool Memory Pooling/Sharing with CXL 1.1 Hosts and Single Logical Devices H# H1 H3 H4 H2 D# D1 D3 D4 D2 CXL 2.0 Switch Memory Pooling with CXL 2.0 Hosts and Multiple Logical Devices D# H# D1 H1 D3 H3 D4 H4 D2 H2 CXL 2.0 Switch ManagementHost (CXL FabricManager) ManagementHost (CXL FabricManager)
- 5. Demo of CXL Memory Pooling Apollo Ref. Board CXL Host 0 Management Host (FM) CXL Host 1 CXL Memory Expander 0 CXL Memory Expander 1
- 6. CXL Memory is allocated/released from/back to the pool when demand increases/subsides Elastic Memory Pool with Apollo Switch
- 7. Demo of CXL Memory Sharing Apollo Ref. Board CXL Host 0 Management Host (FM) CXL Host 1 CXL Memory Expander 0 CXL Memory Expander 1
- 8. XConn’s Upcoming CXL3.1/PCIe6.1 Switch for AI Systems: A Game Changer Memory pooling/sharing/expansion Supports All-to-All with scalable large switching capacity Fit for All-reduce, All-gather with super low latency and high bandwidth switching Scablable fabric network with up to 4,096 CXL devices Hybrid CXL/PCIe mode to connect CXL and PCIe devices Works with emerging CXL devices, e.g. In Memory Compute Lower total power consumption to reduce energy cost
- 9. Address: 1245 S. Winchester Blvd San Jose, CA 95128 Web: Https://www.xconn-tech.com Email: JP.Jiang@xconn-tech.com
Editor's Notes
- #4: host