ODSA - OCP Accelerator Module and the Infrastructure
1 like697 views
The document discusses the need for an open accelerator infrastructure to support the growing variety of AI hardware accelerators. It proposes the Open Compute Project Accelerator Module specification to standardize the form factor and interfaces of accelerator modules. It also introduces the concept of a Universal Baseboard that can host multiple accelerator modules and uses a standardized interconnect to provide flexible topologies to scale performance for different AI workloads. The open infrastructure is intended to foster innovation while providing interoperability between accelerator modules and systems.
ODSA - OCP Accelerator Module and the Infrastructure
- 1. Consume. Collaborate. Contribute.Consume. Collaborate. Contribute. OCP Accelerator Module and The Infrastructure ODSA Project Workshop March 28, 2019
- 2. Consume. Collaborate. Contribute. Outline • Motivation • Approach • Examples • Requesting Participation and Feedback
- 4. Consume. Collaborate. Contribute. AI’s rapid evolution is producing an explosion of new types of hardware accelerators for Machine Learning (ML) and Deep Learning (DL) GPU FPGA ASIC NPU TPU NNP IPU xPU…
- 5. Consume. Collaborate. Contribute. Varied Module and System Form Factors
- 6. Consume. Collaborate. Contribute. Different Implementations Targeting Similar Requirements! HPC
- 7. Consume. Collaborate. Contribute. Power CPU Head Node Host Interface High-speed Expansion Links Baseboard CPU Head Node Host Interface Cooling Baseboard PCIe Switch PCIe SwitchChassis Chassis Logical Components for AI Hardware System
- 8. Consume. Collaborate. Contribute. Facebook Big Basin System Baseboard on sliding tray Midplane boardBaseboardIO board
- 9. Consume. Collaborate. Contribute. Big Basin Rack View Head node Tioga Pass • 2S server Tioga Pass as head node • Open Rack v2, 12.6kw • 4 Big Basin per Rack
- 10. Consume. Collaborate. Contribute. Facebook Inference/Video Accelerator Common System
- 11. Consume. Collaborate. Contribute. Facebook Training System
- 12. Consume. Collaborate. Contribute. Varied Module and System Form Factors
- 13. Consume. Collaborate. Contribute. Common Requirements for Accelerator System • Flexibility • Robustness & Serviceability • Configuration, Programming, & Management • Power & Cooling • Inter-module Communication to Scale Up • Input / Output Bandwidth to Scale Out
- 14. Consume. Collaborate. Contribute. “If you want to go Fast, go Alone; If you want go Far, go Together” We have done Fast for Short-term result; It is time to go Far at OCP for Long-term gain!
- 15. Consume. Collaborate. Contribute. We Started from OAM.
- 16. Consume. Collaborate. Contribute. OCP Accelerator Module(OAM) Spec • 102mm x 165mm Module Size • With two high-speed Mirror Mezz connectors • 12V and 48V input DC Power • Up to 350w (12V) and up to 700w (48V) TDP • Up to 450W (air-cooled) and 700W (liquid-cooled) • Support single or multiple ASIC(s) per Module • Up to eight x16 Links (Host + inter-module Links) • Support one or two x16 High speed link(s) to Host • Up to seven x16 high speed interconnect links • Up to 8* Modules per Baseboard • System management and debug interfaces
- 18. Consume. Collaborate. Contribute. OAM Power • Support both 12V and 48V as input • 12V to support up to 350w TDP • 48V to support up to 700w TDP Power Rail Voltage Tolerance # of pins Current Capability Status P12V 11V min to 13.2V max 27 27A (when at 11V) Normal Power P12V Mandatory 11V min to 13.2V max 5 5A (when at 11V) Normal Power P48V 44V min to 60V max 16 16A (when at 44V) Normal Power P3.3V 3.3V±10% (max) 2 2A Normal Power
- 19. Consume. Collaborate. Contribute. OAM Pin Map SerDes 1 X16 SerDes 2 X16 SerDes 3 X16 Host X16 SerDes R X20 SerDes 4 X16 SerDes 5 X16 SerDes 6 X16 Power Connector 0 Connector 1
- 20. Consume. Collaborate. Contribute. Interconnect end-to-end Channel Loss • The module interconnection channel total insertion loss @28Gbps should not be over -8dB • System baseboard IL budget = Die to Die IL from each OAM supplier – 16dB 8-dB package loss 8-dB package lossOAM OAM ~2-3inch Package Package ~2-3inch Baseboard
- 21. Consume. Collaborate. Contribute. System Management/Debugging • Sensor reporting • Error monitoring/Reporting • Firmware Update • Power Capping • FRU Information • IO Calibration • JTAG/I2C/UART interfaces for debugging
- 22. Consume. Collaborate. Contribute. Overview OAM: Mechanical/Thermal Top stiffener Example Bottom stiffenerOAM PCB Example die BOTTOM VIEWTOP VIEW
- 23. Consume. Collaborate. Contribute. Mech Requirements – OAM Bottom Stiffener • 5±0.15mm stiffener as required by connector • 3mm alignment pins that extend 10mm below OAM PCB surface • Die spring (rectangular profile coil spring) to provide unmate force • EMI gaskets for grounding to baseboard SIDE VIEW, FULL ASSEMBLY (BOTTOM STIFFENER IN BLUE) BOTTOM VIEW FRONT VIEW Top stiffener OAM PCB Bottom Stiffener System Baseboard
- 24. Consume. Collaborate. Contribute. Mech Requirements – System Baseboard • Component KOZ 103 x 166mm: 0mm height • Cross-hatched locations: Grounding Pads • EMI grounding pads located north and south of the connectors • 4x Mounting Holes for M3.5 screws • 2x SMT nuts used as alignment features TOP VIEW
- 25. Consume. Collaborate. Contribute. Mech Recommendations – Alignment Features • Notch provides orientation and keying (OPTIONAL, BUT RECOMMENDED) TOP VIEW SMT nuts on baseboard SIDE VIEW Molex Mirror Mezz Connector Gatherability: 0.76mm
- 26. Consume. Collaborate. Contribute. Thermal Requirements – Operation Environment • Ambient Temp: 5°C to 35°C • Approach Temp: 5°C to 48°C • Altitude: sea level to 6000ft • Humidity: 20% to 90% • Cold boot temp limit: TBD • Storage temp: -20°C to 85°C • No ambient temp compensation/de-rating for altitude
- 27. Consume. Collaborate. Contribute. Now we have a industry standard OAM spec, what’s the next?
- 28. Consume. Collaborate. Contribute. We need an Open Accelerator Infrastructure
- 29. Consume. Collaborate. Contribute. • Power and Cooling • Mechanical • Electrical • Security & Management SERVER Hierarchical Base Specification for OAI Well-defined boundaries Fostering Innovation • OAM • UBB (Interconnect Topology) • Switch Board • SCM • Tray • Chassis Designs and Products may be compliant to any or all specifications
- 30. Consume. Collaborate. Contribute. The Universal Baseboard (UBB)
- 31. Consume. Collaborate. Contribute. Different Neural Networks and Frameworks for Model or Data Parallelism Benefit from different Interconnect Topologies
- 32. Consume. Collaborate. Contribute. •Consider a Grid of Planar OAM sites •Standard Volumetric •Protocol Agnostic Interconnects •Wires are Wires! Host Interface Expansion Universal Baseboard Universal Baseboard (UBB)
- 33. Consume. Collaborate. Contribute. With different interconnect topologies Six inter-module Links may create a 3D Mesh or Torus With seven inter-OAM Links and one Host Link A Grid of interconnected OAMs, Max Bisection BW One Hop Away Ready for Expansion With six inter-OAM Links and one Host Link
- 34. Consume. Collaborate. Contribute. OAM Topology Examples Fully Connected w/ 7 links
- 35. Consume. Collaborate. Contribute. OAM Topology Examples Almost Fully Connected w/ 6 links
- 36. Consume. Collaborate. Contribute. OAM Topology Examples Rings w/ 4 links Port 4/5/6/R for AISC which has 4 links on Conn1 Only
- 37. Consume. Collaborate. Contribute. OAM Topology Examples Hybrid Cube Mesh w/ 4 links Port 4/5/6/R for AISC which has 4 links on Conn1 Only
- 38. Consume. Collaborate. Contribute. OAM Topology Examples Hybrid Cube Mesh w/ 6 links
- 39. Consume. Collaborate. Contribute. Summary for OAI/OAM • Rev 0.85 of the OAM spec is available in OCP Wiki • Join the Project and further develop interoperable Modules for an Open Accelerator Infrastructure(UBB, PSB, SCM, Tray, Chassis…) • We invite you to join the OAI subgroup for further collaboration: Register for the Mailing List: https://ocp-all.groups.io/g/OCP-OAI Wiki under OCP Server Project: https://www.opencompute.org/wiki/Server/OAI
- 40. Consume. Collaborate. Contribute. Accelerator Form Factors • Different Form Factors PCIe CEM OAM M.2/Dual M.2 OCP NIC Others • Different Accelerator Targets Training Inference Video Others