Creating Your Own PCI Express System Using FPGAs: Embedded World 2010
6 likes10,640 views
This document discusses creating PCI Express systems using FPGA devices. It provides an overview of PCI Express, describing its key functional elements like the root complex and endpoints. It also outlines PCI Express support in Altera FPGAs, including both hard IP blocks and soft IP cores that enable PCI Express connectivity. The hard IP blocks perform the various PCI Express layers and reduce resource usage compared to soft cores.
Creating Your Own PCI Express System Using FPGAs: Embedded World 2010
- 1. Creating Your Own PCI Express System with FPGA Devices Learning Zone @ Embedded World © 2010 Altera Corporation - Public
- 2. PCI Express Overview PCI Express (Peripheral CPU CPU Component Interconnect PCI Express) is a computer Graphics Express North-bridge (high Memory expansion standard BW, low latency) introduced by Intel in 2004 Serial ATA HDD − Officially abbreviated as PCIe PCI PCI Express (PCI-E is also commonly used) Mbyte down devices South-bridge (I/O bridge) PCIe replaces PCI, PCI-X, USB 2.0 and AGP Gigabit Ethernet* LPC External PCIe complements SERDES- Add ins modules LPC based bus interface to the CPU © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 2
- 3. PCIe Functional Elements Root complex CPU CPU − Connects host CPU/memory complex to PCIe hierarchy − One or more downstream ports (= rootports) Root Memory complex − Rootport discovers, initializes, and enumerates PCIe topology Downstream port Switch PCIe link Upstream port − Assembly of logical PCIe-to-PCIe bridges − One upstream port directed towards Switch root complex − One or more downstream ports Endpoint PCIe PCIe Legacy Switch − Legacy and native endpoints e.g. to PCI-X endpoint endpoint bridge terminal point to PCI, USB, InfiniBand, Fibre Channel, Ethernet… PCI-X expansion slot © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 3
- 4. PCIe Technology Timeline Raw bit rate Link BW BW/lane/way PCIe 1.x 2.5 GT/s 2 Gbps ~250 MBps Future PCIe 2.0 5.0 GT/s 4 Gbps ~500 MBps FPGA PCIe 3.0 8.0 GT/s 8 Gbps ~1 GBps Releases Total Bandwidth / Gbps 100 90 • Gen3: 8 GT/s signaling 80 • Lower latencies, improved PM • Enhanced software model 70 60 50 PCIe Gen2 40 @ 5 GT/s 30 PCIe Gen1 @ 2.5 GT/s 20 10 PCI/PCI-X 0 1999 2001 2003 2005 2007 2009 2011 2013 Note: Link BW = 0.8 x (raw bit rate) in Gen1/Gen2 due to 8B/10B encoding Dotted line is for projected numbers, Total bandwidth = Link Bandwithx8 Lanes © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 4
- 5. PCIe Support in Altera Devices Hard IP Gen1/2 Soft IP Gen1 Device PCI Express PCI Express Supported Supported configuration configuration Cyclone IV GX x1, x2, x4 x1, x4 (PCIe Gen1 only) 1 per device Arria II GX x1, x4, x8 x1, x4, x8 (PCIe Gen1 only) 1 per device Stratix IV GX x1, x4, x8 x1, x4, x8 2 to 4 per device HardCopy IV GX x1, x4, x8 x1, x4, x8 1 to 2 per device Stratix IV GT x1, x4, x8 x1, x4, x8 2 to 4 per device Save up to 40K LEs and no License Fees with Hard IP © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 5
- 6. PCI Express Hard IP Block Performs transaction, data link, PHYMAC layer and functionality Supports − PCI Express Gen 1 and Gen 2 − x1, x2, x4 & x8 lane configurations − Root port and endpoint applications Connects directly to embedded transceivers using internal PIPE interface − Shared by two adjacent transceiver blocks Enabled through the PCI Express Compiler Wizard FPGA PCI Express Hard IP Block Embedded Transceiver Block PIPE Transceiver Block n Transaction Data Link PHYMAC Transceiver Block 2 Layer Layer Layer Transceiver Block 1 To / from Transceiver Block 0 Slot or cable © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 7. PCI Express Hard IP Diagram Configurable maximum payload size Transceiver Block PCI Express Hard IP Block PLD Fabric − 128, 256, 512, 1024*, or 2048* bytes Clock & Reset Selection 1 or 2* Virtual Channels (VC) PMA PCS 16-Kbyte receive buffer per VC TL FPGA Interface PCI Express Interface Adapter 2-Kbyte transmit retry buffer PIPE Application Transceiver Block Protocol Stack Layer 128-bit or 64-bit application datapath width Local Interrupt support (legacy, MSI Mgmt IF & MSI-X) PMA PCS (LMI) Retry VC RX Advanced error reporting Buffer Buffer (AER) support Power management support Local management interface (LMI) to access configuration registers Status & debug interface * Stratix IV GX/GT only © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 8. PCI Express Solutions © 2010 Altera Corporation - Public
- 9. Industrial PC or Controller USB Processor DSP Graphics (Intel Atom) (Optional) Hard Disk PCIe RapidIO DVI Isolation and RJ-45 Industrial Motor Control Ethernet Power Stage Isolation PWM and RJ-45 Sensor Camera ADC/DAC OPTO I/O © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 10. CNC EtherCAT Drives Demo CNC application runs on 3S software and positions motors via an EtherCAT Industrial Ethernet network connected to an LCD for data display and I/O control options © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 11. PCI Express Testing © 2010 Altera Corporation - Public
- 12. PCIe Hard IP Testing Altera PCIe Gen2 development card Interoperability testing − Compatibility/functionality with chipsets Core generator with verifying payloads (built in DMA engine) − Test for PCIe compliance using PCI-SIG tests Performance testing − Testing throughput of PCIe link PCIe Gen2 motherboard − Stress test using the chain DMA architecture PCle Gen2 platforms Test configurations Motherboard Chipset Processor − Modes AMD Asus M3A78-T 790GX/SB750 AMD790GX PCI Express Gen1 x1, x4, x8 PCI Express Gen2 x1, x4, x8 Intel Asus P5Q-EN Intel-645 LGA775 Intel DX58S0 Tylersburg Core I7 © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 12
- 13. Preliminary Results of PCI Express Hard IP Bandwidth Testing – Tylersburg Chipset Simultaneous read/ HIP configuration DMA write DMA read write HIP:Gen1:x8:128 bits 1,691 MBps 1,695 MBps 1,683/1,449 MBps HIP:Gen1:x8:64 bits 1,774 MBps 1,522 MBps 1,524/1,631 MBps HIP:Gen2:x4:64 bits 1,772 MBps 1,528 MBps 1,528/1,634 MBps HIP:Gen2:x4:128 bits 1,670 MBps 1,613 MBps 1,607/1,402 MBps HIP:Gen2:x8:256 bits 3,400 MBps 3,300 MBps 3,173/2,155 MBps Theoretical maximum bandwidth Demo Here − Gen2 x8: 4,000 MBps Stratix IV GX hard IP is close to theoretical maximum now © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 13
- 14. PCI Express Development Kits © 2010 Altera Corporation - Public
- 15. PCI Express Development Kits Stratix IV GX FPGA Arria II GX FPGA Cyclone IV GX Transceiver Development Kit Development Kit Starter Kit Applications Applications Applications −Develop and test PCI −Develop and test PCI −Develop and test PCI Express Gen2 (to x8 lane) Express Gen1 (to x4 lane) Express Gen1 (to x1 lane) −Develop and test memory −Develop and test memory −Develop and test memory subsystems subsystems subsystems −System Development −System Development −FPGA prototyping & power −Supported other protocols −Supported other protocols measurement via (HSMC) Daughter Cards via (HSMC) Daughter Cards −Testing transceiver signal quality to 2.5Gbps DK-DEV-4SGX230N/C2 $4,495 DK-DEV-2AGX125N $1,495 DK-START-4CGX15N $395.00 Now Now Coming shipping shipping March 2010 © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 16. Altera PCI Express Solutions Summary Industry-leading solutions − Stratix IV GX FPGA is industry’s only shipping FPGA solution with hard IP support for PCIe Gen2 − Arria II GX FPGA is the industry’s first low-cost 40-nm FPGA with hard IP support for PCIe Gen1 x1, x4, x8 − Cyclone IV GX is the industry’s lowest-cost FPGA with hard IP support for PCIe Gen1 x1, x4 Low-risk, hardware-verified solutions − PCI-SIG compliance workshops − Interoperability with multiple ASSP vendors − 5 generations of transceiver-based FPGAs with PCI Express support − Development kits/demo boards © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 16
- 17. PCI Express Quickstart Guide with SOPC Builder PCI Express Endpoint using SOPC Builder Lab © 2010 Altera Corporation - Public
- 18. Example PCIe-SOPC Builder System Memory DMADMA DMA © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 19. PCIe Endpoint Design Example © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 20. Backup © 2010 Altera Corporation - Public
- 21. Auto Speed Negotiation Initially link trains to L0 (normal operation) in 2.5G data rate Speeds advertised in training sequence (TS) ordered sets − Supported speeds by the other component noted Speed change occurs when the link is in electrical idle 2.5 Gbps Detect 2.5 Gbps Polling.Active 2.5 Gbps 2.5 Gbps Polling.Config Recovery.RcvrLock 2.5 Gbps 5.0 Gbps 5.0 Gbps 2.5 Gbps 2.5 Gbps Config 5.0 Gbps Recovery.RcvrCfg Recovery.Speed 2.5 Gbps 2.5 Gbps L0 Recovery.Idle 5.0 Gbps 5.0 Gbps Dedicated PCIe (PIPE) Clock Switch Circuitry in Stratix IV GX/GT and Hardcopy IV GX © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 21
- 22. MSC Hpe IRP Platform © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 23. PCI-SIG Industry Testing – PCI-SIG PCI-SIG FPGA/PHY Config. PCIe IP qualification Cyclone IV GX (Gen 1) x1, x4 Hard IP Q110 Arria II GX (Gen1) x1, x4, x8 Hard IP Q209 Stratix IV GX (Gen1 and Gen2) x1, x4, x8 Hard IP 1Q09 Arria GX (Gen1) x1,x4 Soft IP 2Q07 Stratix II GX (Gen1) x1, x4, x8 Soft IP 2Q06 Cyclone II/TI x1 (Gen1) x1 Soft IP 1Q06 Cyclone II/Philips x1 V (Gen1) x1 Soft IP 4Q05 Stratix GX (Gen1) PCI-SIG qualification across all target lane configurations IP x1, x4 Soft 1Q05 © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 23
- 24. PCI Express TX Eye Mask Test Data Summary − The Stratix IV GX device TX eye passes the PCI Express Gen 2 mask tests © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 24
- 25. Development Kits © 2010 Altera Corporation - Public
- 26. Stratix IV GX and HardCopy IV GX Shipping Shipping Development Kit now now PCI Express short form factor Included daughtercards − HSMC loop-back − HSMC debug Included software − Quartus II Development Kit Edition 12-month license − Example designs Documentation − User guide − Reference manual − Schematics − Layout design files © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S.
- 27. Arria II GX Development Kit Shipping Shipping now now PCI Express short form factor Included daughtercards − HSMC loop-back − HSMC debug Included software − Quartus II Development Kit Edition 12-month license − Example designs Documentation − User guide − Reference manual − Schematics − Layout design files © 2010 Altera Corporation - Public ALTERA, ARRIA, CYCLONE, HARDCOPY, MAX, MEGACORE, NIOS, QUARTUS & STRATIX are Reg. U.S. Pat. & Tm. Off. and Altera marks in and outside the U.S. 27