QPACE QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)

QPACE QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)

Heiko Joerg Schick
Firmware Project and Bring-up Lead
Böblingen, 2009-06-29

IBM Deutschland Research & Development GmbH 10/15/09 © Copyright IBM Corporation 2009

Agenda

Overview

QPACE Architecture

QPACE Node Card

2 IBM Deutschland Research & Development GmbH 10/15/09 © Copyright IBM Corporation 2009

Overview

QPACE = QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)

The QPACE project is a research collaboration of IBM Development and European
universities and research institutes with the goal to build a prototype of a cell processor-
based supercomputer.

The major part of this project will be funded by the German Research Foundation (DFG –
Deutsche Forschungsgemeinschaft) as part of a Collaborative Research Center (SFB –
Sonderforschungsbereich [TR55]). Additional funds come the PRACE project.


QPACE Architecture

Architecture

System:
 Node card with PowerXCell 8i processor and network processor (NWP)
 Commodity processor interconnected by a custom network
 256 node-cards per rack = 26 Tflops peak performance

Network:
 3-dimensional Torus Network: nearest-neighbour communication, 3-dimensional torus topology
 Tree Network: evaluation of global conditions and synchronization
 Gigabit Ethernet: One link per node, rack-level switches

Applications:
 Optimized for calculations in theoretical particle physics: Simulation of Quantum Chromodynamics
 Target sustained performance of 20-30%

Liquid cooling system:
 Closed node card housing acts as heat conductor
 Housing is connected to liquid-cooled “cold plate”
 Cold Plate is placed between two rows of node cards


QPACE Architecture

Root Card
(16 per rack) Backplane
(8 per rack)

Node Card
(256 per rack)

Power Supply and Power Adapter Card
(24 per rack)
Rack


QPACE Node Card

Features

Components:
 PowerXCell 8i processor 3.2 GHZ
 4 Gigabyte DDR2 memory 800 MHZ with ECC
 Network processor (NWP) Xilinx FPGA LX110T FPGA
 Ethernet PHY
 6 x 1GB/s external links using PCI Express physical layer
 Service Processor (SP) Freescale 52211
 FLASH (firmware and FPGA configuration)
 Power subsystem
 Clocking

Network Processor:
 FLEXIO interface to PowerXCell 8i processor, 2 bytes with 3 GHZ bit rate
 Gigabit Ethernet
 UART FW Linux console
 UART SP communication
 SPI Master (boot flash)
 SPI Slave for training and configuration
 GPIO


QPACE Node Card

Network Processor Network PHYs
PowerXCell 8i (FPGA)
Memory Processor


QPACE Node Card

Block Diagram

DDR2 DDR2
DDR2 DDR2

800MHz

I2C
Power SPI RW
Subsystem (Debug)
PowerXCell 8i

FLEXIO FLEXIO
Clocking
6GB/s 6GB/s

RS232
SPI
I2C
SP FPGA Virtex-5
UART
Freescale
MCF52211 GigE PHY

SPI 384 IO@250MHZ
Flash
4*8*2*6 = 384 IO
680 available (LX110T)
6x 1GB/s PHY

Compute Network


QPACE Node Card

FPGA Architecture
FLEXIO

Rocket IO IBM:
• RocketIO Logic
IOC (IOIF)
IOC (IOIF)
FELX iO • IOC Logic
• GBIF Logic

Slave(BE) GBIF Master(BE)

Receive Requests Make Requests

Switch / Address Decode / FIFOs
Bus Controller

Academic Partners:
• Network Processor Logic

6 x 1GB/S


QPACE Node Card

FlexIO Processor Interface

Interface:
 High bandwidth interface between IBM PowerXCell 8i processor and Xilinx Viretx-5 FPGA via an
interface implementation from Rambus Inc.
 Optimized for intra-board environments.

Challenges:
 QPACE FlexIO connection is very challenging:
– Speed, Latency, Bandwidth and Timing (Clock)
– 3 Gbyte/sec communication channel
– 2 Byte link wide

Requirements:
 FlexIO requires link training after power-on:
– Phase calibration (aligns the data for optimal sampling point)
– Parallel calibration (synchronizes the receive deserializer with the transmit serializer)
– Levelization calibration (aligns all data lanes)


Thank you very much for your attention.

Weitere Themen

Disclaimer

IBM®, DB2®, MVS/ESA, AIX®, S/390®, AS/400®, OS/390®, OS/400®, iSeries, pSeries, xSeries, zSeries, z/
OS, AFP, Intelligent Miner, WebSphere®, Netfinity®, Tivoli®, Informix und Informix® Dynamic ServerTM,
IBM, BladeCenter and POWER and others are trademarks of the IBM Corporation in US and/or other
countries.

Cell Broadband Engine is a trademark of Sony Computer Entertainment, Inc. in the United States,
other countries, or both and is used under license there from. Linux is a trademark of Linus Torvalds
in the United States, other countries or both.

Other company, product, or service names may be trademarks or service marks of others.
The information and materials are provided on an "as is" basis and are subject to change.


QPACE QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)

More Related Content

What's hot (20)

Similar to QPACE QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.) (20)

More from Heiko Joerg Schick (16)

Recently uploaded (20)

QPACE QCD Parallel Computing on the Cell Broadband Engine™ (Cell/B.E.)