Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems (ECOC2021)
- 1. Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems ECOC Market Focus New Technologies & Systems/New Markets/New Packaging Platforms Wed, 15 September 2021 Jeff Hutchins / Ranovus OIF PLL Working Group Co-Packaging Vice Chair Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 1
- 2. Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 2
- 3. OIF Membership and Activities Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 3 Technical Committee Networking & Operations WG Interoperability WG Physical & Link Layer WG Physical Layer User Group Network Operator WG Interoperability Demonstrations Implementation Agreements Market Awareness and Education Committee https://www.oiforum.com/technical- work/implementation-agreements-ias/ Educational Workshops & Presentations
- 4. Motivation for Co-Packaging? Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 4 The portion of data center power that networking consumes is growing with each generation and has become a key driver for co-packaging Co-packaging can significantly reduce the “network” power by the reduction in electrical interface power – leaving more power for ”compute” But it also enables new applications needing high-density, high-capacity, and disaggregated resources Networking Power is becoming a larger portion of the data center power leaving less for useful computation Adapted from OIF Lightwave CPO Webinar, Ram Huggahalli , 10-14-20
- 5. What are Some Configurations? Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 5 The Co-Packaging Assembly (CPA) is a Multi-Chip Module (MCM) with either socketed … Co-Packaging Architectures Host PCB Co-Packaged Assembly Substrate ASIC Engine Substrate socketed OE/EE XSR Some Engine locations can be populated with a passive Copper Cable Assembly (CCA) to connect to engines which might be too large to fit in the co-packaged assembly Copper Cable Host PCB Co-Packaged Assembly Substrate ASIC Engine Substrate OE/EE XSR Host PCB (high performance material) ASIC ASIC Substrate / RDL RDL Optical Engine Or Pluggable MR/LR SerDes VSR Pluggable or On-Board Architectures ASIC ASIC Substrate / RDL RDL OE/EE Host PCB Or the ASIC may be packaged and mounted on a high-quality substrate XSR+ Near Packaging Architectures Pluggable module at the front panel or on-board module
- 6. What are the Co-Packaging Applications? Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 6 OIF CPO Webinar, Ram Huggahalli 10-14-20 Optics Chiplet Electrical Chiplet Data Center Networking AI Training / ML Disaggregation End Node Switching Node Dense Sparse Low Latency I/F Ethernet I/F Poster Child Application
- 7. Co-Packaged Card Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 7 51.2T Ethernet Switch Card with 100G Lanes oif2021.178.00 Ted Schmidt / Lumentum Optical Engine Connector Co-Packaged Assembly Optical Engine Switch Card PCB Remote Lasers if needed Switch ASIC
- 8. Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 8 Studying a Front Panel Pluggable Form Factor • Easy to replace upon failure • Using an external laser can improve the reliability of the optical engine • Employs a blind mate optical connector Scope • Mechanical form factor and thermal power classes • Electrical and optical specifications • Eye safety requirements Specifying a Form Factor Eye safety requirements are considered Remote Laser Standardization
- 9. Extra connector Extra connector Optical Connector Contribution to Link Loss Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 9 Optical Engine Front Panel Pluggable Optical Engine Front Panel Pluggable Data Center Cabling MDI MDI TP2 Data Center Cabling TP3 Sometimes it’s reasonable to convolve these loss distributions In either scenario, the Link Loss from TP2 to TP3 is fixed, per standards OE Manufacturing Test Point OE Manufacturing Test Point Front Panel Pluggable Link Co- Packaged Given optical requirements for TP2 and TP3, the requirements for the OE test point can be adjusted for the extra connector losses
- 10. Rework and Connector Loss Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 10 • Let’s assume that a particular connector has a 0.1% chance on first mating to exhibit an IL > IL_LIMIT Then for PF = 0.1%, the assembly yield “Y” is 99.9% for a single mating • The chance of of failing one or two of two fiber connections: or • For the general case of ”N” fiber connections, the chance of failure one or more connections: or • So, we must choose a PF such that we achieve the desired card 1st pass assembly yield loss which drives rework 1 − 𝑌2 1 − 𝑌𝑁 𝑌 = (1 − 𝑃𝐹) 1 − 1 − PF N 1 − 1 − 𝑃𝐹 2 With PF = 0.1% The target card rework rate drives the connector IL failure probability limit
- 11. How Should a Remote Laser & OE be Controlled? Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 11 Host for MIS ELS Optical Engine Host for MIS ELS Optical Engine Host for MIS Integrated Lasers Optical Engine Host for MIS Integrated Lasers Optical Engine CMIS I/F CMIS I/F CMIS I/F MIS I/F MIS I/F ? I/F ? I/F ? I/F ? I/F The 3.2T OE for Ethernet switching will support both internal and external laser sources The OIF is addressing the management interface architecture in its CMIS project Laser Internal To Optical Engine Laser External To Optical Engine Controller directly controls Laser Looks like pluggable optics
- 12. Co-Packaging Electrical Interfaces for Ethernet Switching Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 12 Host PCB Co-Packaged Assembly Substrate ASIC Engine Substrate socketed OE/EE XSR ≤ 10dB Optical Engine in a co-packaged architecture The OIF is supporting both Near Packaging (XSR+) and Co-Packaging (XSR) electrical interfaces Leveraging OIF’s existing XSR project ASIC ASIC Substrate / RDL RDL OE/EE Host PCB XSR+ ≤ 13dB Optical Engine in a near-packaged architecture Newly created project in the electrical track
- 13. XSR+ 3Q2021 How is Co-Packaging Being Tackled by the OIF? Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 13 Framework IA 4Q2020 Initiated a ”Framework” umbrella level project to study co-packaging • Defines the application spaces and explore technological considerations 3.2T OE 1Q2021 Initiated a ”3.2T OE” project for 51.2T Ethernet Switching Application • Defines an optical engine module, 16 of which can be used to provide optical interfaces for a 51.2Tb/s switch using XSR electrical interfaces and FR & DR optical interfaces ELSFP 2Q2021 Initiated a ”ELSFP” (External Laser Small Form Factor) project • Defines a front panel pluggable laser source for co-packaging Co-Packaging The OIF has initiated multiple projects to study and address Co-packaging Initiated an ”XSR+” electrical interface project to to CEI to support NPO • Adds new clauses to support D2D and D2IOE at 13dB loss at ~26.5GHz CMIS Created a “Management Interface” Track and CPO MIS project • Defines the management interface for co-packaging
- 14. Co-Packaged Optics: The OIF’s Work to Standardize Higher Integration Levels for Next-Generation Data Center Systems Copyright © 2021 14 Thank-you Follow-on standards efforts Co-Packaging Framework Framework IA