The OptiPuter, Quartzite, and Starlight Projects: A Campus to Global-Scale Testbed for Optical Technologies Enabling LambdaGrid Computing

"The OptIPuter, Quartzite, and Starlight Projects: A Campus to Global-Scale Testbed for Optical Technologies Enabling LambdaGrid Computing” Invited Talk Optical Fiber Communication Conference (OFC2005) Anaheim, CA March 9, 2005 Dr. Larry Smarr Director, California Institute for Telecommunications and Information Technology Harry E. Gruber Professor, Dept. of Computer Science and Engineering Jacobs School of Engineering, UCSD

Abstract and OFC Invited Paper Abstract Dedicated optical connections have significant advantages over shared internet connections. The OptIPuter project (www.optiputer.net) uses medical and earth sciences imaging as application drivers. Quartzite (UCSD) and Starlight (Chicago) create unique combinations of OEO routers and OOO and wavelength-selective optical switches. Invited Paper for OFC 2005 The OptIPuter, Quartzite, and Starlight Projects: A Campus to Global-Scale Testbed for Optical Technologies Enabling LambdaGrid Computing By Larry Smarr, Harry E. Gruber Professor, Department of Computer Science and Engineering, UCSD Director, California Institute of Telecommunications and Information Technology With Shaya Fainman, Joseph Ford, Phil Papadopoulos University of California, San Diego and Tom DeFanti, Maxine Brown, and Jason Leigh Electronic Visualization Laboratory University of Illinois at Chicago

The Evolution from Supercomputer-Centric to a Net-Centric Architecture Megabit/s Gigabit/s Terabit/s Source: Timothy Lance, President, NYSERNet 1 GFLOP Cray2 60 TFLOP Altix Bandwidth of NYSERNet Research Network Backbones T1 32 10Gb “ Lambdas”

Calit2 -- Research and Living Laboratories on the Future of the Internet www.calit2.net University of California San Diego & Irvine Campuses Faculty & Staff Working in Multidisciplinary Teams With Students, Industry, and the Community One Focus Area is Net-Centric Optical Architectures

The OptIPuter Project – Bringing the Power of Lambdas to End Users NSF Large Information Technology Research Proposal Calit2 (UCSD, UCI) and UIC Lead Campuses—Larry Smarr PI Partnering Campuses: USC, SDSU, NW, TA&M, UvA, SARA, NASA Industrial Partners IBM, Sun, Telcordia, Chiaro, Calient, Glimmerglass, Lucent $13.5 Million Over Five Years Linking Global Scale Science Projects to User’s Linux Clusters NIH Biomedical Informatics NSF EarthScope and ORION http://ncmir.ucsd.edu/gallery.html siovizcenter.ucsd.edu/library/gallery/shoot1/index.shtml Research Network

Opt ical Networking, I nternet P rotocol, Comp uter Bringing the Power of Lambdas to Users Extending Grid Middleware to Control: Cluster Enpoints- Storage, Visualization, & Computing Linux Clusters With 1 or 10 Gbps I/O per Node Scalable Visualization Displays with OptIPuter Clusters Jitter-Free, Fixed Latency, Predictable Optical Circuits One or Parallel Dedicated Light-Pipes 1 or 10 Gbps WAN Lambdas Uses Internet Protocol, But Does NOT Require TCP Exploring Both Intelligent Routers and Passive Switches Applications Drivers: Earth and Ocean Sciences Biomedical Imaging

Tiled LCD Displays Driven by Linux Graphics Clusters Allow for Both Global Context and High Levels of Detail "Source: Data from JPL/Mica; Display UCSD NCMIR, David Lee" 150 MPixel Rover Image on 40 MPixel OptIPuter Visualization Node Display

Interactively Zooming In Using EVL’s JuxtaView on NCMIR’s Sun Microsystems Visualization Node "Source: Data from JPL/Mica; Display UCSD NCMIR, David Lee"

Highest Resolution Zoom on NCMIR 40 MPixel OptIPuter Display Node "Source: Data from JPL/Mica; Display UCSD NCMIR, David Lee"

LambdaRAM: Clustered Memory To Provide Low Latency Access To Large Remote Data Sets Giant Pool of Cluster Memory Provides Low-Latency Access to Large Remote Data Sets Data Is Prefetched Dynamically LambdaStream Protocol Integrated into JuxtaView Montage Viewer 3 Gbps Experiments from Chicago to Amsterdam to UIC LambdaRAM Accessed Data From Amsterdam Faster Than From Local Disk all 8-14 none all 8-14 1-7 Displayed region Visualization of the Pre-Fetch Algorithm none Data on Disk in Amsterdam Local Wall Source: David Lee, Jason Leigh

OptIPuter Challenge is to Couple Cluster Endpoints to WAN DWDM Dedicated Photonic Channels Scalable Adaptive Graphics Environment (SAGE) Controls: 100 Megapixels Display 55-Panel 1/4 TeraFLOP Driven by 30 Node Cluster of 64 bit Dual Opterons 1/3 Terabit/sec I/O 30 x 10GE interfaces Linked to OptIPuter 1/8 TB RAM 60 TB Disk Source: Jason Leigh, Tom DeFanti, EVL@UIC OptIPuter Co-PIs NSF LambdaVision MRI@UIC

UCSD Campus LambdaStore Architecture Dedicated Lambdas to Labs Creates Campus LambdaGrid SIO Ocean Supercomputer IBM Storage Cluster Extreme Switch with 2 Ten Gbps Uplinks Streaming Microscope Source: Phil Papadopoulos, SDSC, Calit2

OptIPuter Software Architecture--a Service-Oriented Architecture Integrating Lambdas Into the Grid Distributed Applications/ Web Services Telescience Vol-a-Tile SAGE JuxtaView Visualization Data Services LambdaRAM PIN/PDC Lambdas IP+TP Discovery and Control GTP XCP UDT LambdaStream CEP RBUDP DVC Configuration Distributed Virtual Computer (DVC) API DVC Runtime Library Globus XIO DVC Services DVC Core Services DVC Job Scheduling DVC Communication Resource Identify/Acquire Namespace Management Security Management High Speed Communication Storage Services GRAM GSI RobuStore

A Photonics-Centric View of UCSD’s ECE Department - Many Are Involved with Calit2 Photonics Program J. Ford S. Esener Y.H. Lo M. Heller S. Bhatia Bio Engineering Chemistry C Tu Communications EM waves RF Electronic Circuits Computer Engineering Electronic Devices & Materials Optical Interconnects Thrust Bio Photonics Thrust Photonic Networks Thrust G. Papen S. Fainman Ed Yu C. Guest Signal Processing Systems Materials P.K Yu S. Radic D. Wang S. Mookherjea Most Are Involved with Calit2 Photonics Program

UCSD ECE, Jacobs School, and Calit2 Set Photonics as a High Priority for Faculty Recruiting All Joined UCSD in Last 2.5 Years Shayan Mookherjea , Asst. Professor, Electrical and Computer Engineering Expertise: Optical Devices and Optical Communication Networks, Including Photonics, Lightwave Systems and Nano-Scale Optics Ph.D. : Electrical Engineering, Caltech, 2003 Background: 2003 Wilts Prize for Best Thesis in Caltech Electrical Engineering Stojan Radic , Assoc. Professor, Electrical and Computer Engineering Expertise: Optical Communication Networks; All-Optical Processing; Parametric Processes in High-confinement Fiber and Semiconductor Devices Ph.D. : Institute of Optics, University of Rochester, 1995 Background: Corning research, Bell Labs Trans. Dept., Nortel Chair Assoc. Prof., Duke Deli Wang , Asst. Professor, Electrical and Computer Engineering Expertise: Nanoscale Science and Technology; Semiconductor Nanomaterials and Devices for Electronic, Optoelectronic and Biological Applications Ph.D. : Materials, UC Santa Barbara, 2001 Background: Postdoctoral Fellow, Harvard University Joseph Ford , Assoc. Professor, Electrical and Computer Engineering Expertise: Optoelectronic Subsystems Integration (MEMS, Diffractive Optics, VLSI); Fiber Optic and Free-Space Communications Ph.D. : Applied Physics, UCSD, 1992 Background: Bell Labs Adv. Photonics Dept., Chief Scientist, Optical Micro-Machines. George Papen , Professor, Electrical and Computer Engineering Expertise: Advanced Photonic Systems Including Optical Communication Systems, Optical Networking, and Environmental And Atmospheric Remote Sensing Ph.D. : Electrical and Computer Engineering from the University of Wisconsin, 1989 Background: University of Illinois at Urbana-Champaign UCSD Photonics

NSF Quartzite Research Instrumentation Award ECE Faculty Augment OptIPuter Testbed Hybrid System of Packet-Based and Circuit-Based Devices Match the Network to the Number of Existing Endpoints Greatly Increase the Number of 10Gb Optical Paths Evaluating DWDM and CWDM Technologies for Campus Scale Hybrid Network “Switch Stack” at Campus Collocation Point Packet Switch—Chiaro Networks Transparent Optical Switch--Glimmerglass Physically Build New Topologies Without Physical Rewiring Experimental Pre-Commercial Devices Lucent Wavelength-Selective Switch Experimental Academic Devices Radic/Ford Packet-Rate Wavelength Routing and Multicasting Source: Phil Papadopoulos, SDSC, Calit2—Quartzite PI

The Optical Core of the UCSD Campus-Scale Testbed -- Evaluating Packet Routing versus Lambda Switching Goals by 2007: >= 50 endpoints at 10 GigE >= 32 Packet switched >= 32 Switched wavelengths >= 300 Connected endpoints Approximately 0.5 TBit/s Arrive at the “Optical” Center of Campus Switching will be a Hybrid Combination of: Packet, Lambda, Circuit -- OOO and Packet Switches Already in Place Source: Phil Papadopoulos, SDSC, Calit2 Funded by NSF MRI Grant Lucent Glimmerglass Chiaro Networks

UCSD Quartzite Testbed -- Lucent 1xK Wavelength-Selective Switch MEMS Wavelength Switching Concept: Modular Transparent DWDM Network Provisioning 64 Channel 4×4 WS-OXC Prototype Micro-Electro-Mechanical Switching + Free-Space Optical Wavelength MUX Millisecond-Rate Provisioning for DWDM & CWDM Networks

Packet-Rate Wavelength Routing and Multicasting Parametric λ -Conversion + Passive Waveguide Routing Conventional 10 GbE Terminals Connected via Transparent Passive Router + NLO Ultra-fast Parametric λ - Conversion 1 2 3 K N-1 N 1 2 3 l M-1 M Node IN Node OUT … and Routed to Arbitrary Output Input Packet  -Translated… AWG Router Waveguide Wavelength Band Translation 1 to 100nm Translation Amplification & 2R Regeneration Time-of-Flight Transparent Routing Passive Silica Waveguide 40x40 Channels, 50 GHz Passband Source: Joseph Ford, Stojan Radic, ECE, UCSD A B C P Q NxN  UCSD Photonics

Experimental Demo: λ -Conversion @ 40 Gb/s UCSD Parametric Processing Laboratory OC-768 Packet Switched in Primitive Parametric Cell Parametric Processing Nonlinear Processing in High Confinement Fiber / SOA / QD Sub-Picosecond Response Time + Time-of-Flight Advantages over Conventional O-E-O Routing Data Rate / Format Independent: Transparent to 1 Tb/s Routes Signals by Multichannel Band (Not Single λ ) Selective Conjugation Supports Long-Haul Transmission Source: Joseph Ford, Stojan Radic, ECE, UCSD UCSD Photonics

Scalable Intelligent Optical Networks (SION) Photonics Research Testbed To Enable Cross-Integrational Photonics Systems Research Fundamentally novel device capabilities New component capabilities Specific component performance specs Network-enabling components needed Performance metrics and drivers Novel networking capabilities Devices & Subsystems Network Architectures Physical Layer System Control Photonics Testbed UCSD Photonics

NLR Will Provide an Experimental Network Infrastructure for U.S. Scientists & Researchers First Light September 2004 “ National LambdaRail” Partnership Serves Very High-End Experimental and Research Applications 4 x 10Gb Wavelengths Initially Capable of 40 x 10Gb wavelengths at Buildout Links Two Dozen State and Regional Optical Networks DOE and NASA Using NLR

The OptIPuter LambdaGrid is Rapidly Expanding 1 GE Lambda 10 GE Lambda Source: Greg Hidley, Aaron Chin, Calit2 UCSD StarLight Chicago UIC EVL NU CENIC San Diego GigaPOP CalREN-XD 8 8 NetherLight Amsterdam U Amsterdam NASA Ames NASA Goddard NLR NLR 2 SDSU CICESE via CUDI CENIC/Abilene Shared Network PNWGP Seattle CAVEwave/NLR NASA JPL ISI UCI CENIC Los Angeles GigaPOP 2 2

OptIPuter Has Built on the Lessons Learned from the OMNInet Metro Area OOO Testbed NTON 10 Gb Lambdas StarLight: the Largest 1 GE & 10 GE Exchange for Supporting U.S. / International Research & Education Networks NTONC C DWDM RAM

Dedicated Research 10Gb Optical Circuits in 2005 North America, Europe and Japan US IRNC (black) – 20Gb NYC—Amsterdam – 10Gb LA—Tokyo GEANT/I2 (orange) – 30Gb London, etc.—NYC UK to US (red) – 10Gb London—Chicago SURFnet to US (light blue) – 10Gb Amsterdam—NYC – 10Gb Amsterdam—Chicago Canadian CA*net4 to US (white) – 30Gb Chicago-Canada-NYC – 30Gb Chicago-Canada-Seattle Japan JGN II to US (grey) – 10Gb Chicago—Tokyo European (not GEANT) (yellow) – 10Gb Amsterdam—CERN – 10Gb Prague—Amsterdam – 2.5Gb Stockholm—Amsterdam – 10Gb London—Amsterdam IEEAF lambdas (dark blue) – 10Gb NYC—Amsterdam – 10Gb Seattle—Tokyo CAVEwave/PacificWave (purple ) – 10Gb Chicago—Seattle—SD – 10Gb Seattle—LA—SD Northern Light UKLight PNWGP Japan Manhattan Landing CERN

Calient Lambda Switches Now Installed at StarLight and NetherLight Source: Maxine Brown, OptIPuter Project Manager University of Amsterdam is an OptIPuter Partner Now Supporting 10GE International Lambdas

Multiple HD Streams Over Lambdas Will Radically Transform Campus Collaboration U. Washington JGN II Workshop Osaka, Japan Jan 2005 Prof. Osaka Prof. Aoyama Prof. Smarr Source: U Washington Research Channel Telepresence Using Uncompressed 1.5 Gbps HDTV Streaming Over IP on Fiber Optics

Calit2 Collaboration Rooms Testbed UCI to UCSD In 2005 Calit2 will Link Its Two Buildings via CENIC-XD Dedicated Fiber over 75 Miles Using OptIPuter Architecture to Create a Distributed Collaboration Laboratory UC Irvine UC San Diego UCI VizClass UCSD NCMIR Source: Falko Kuester, UCI & Mark Ellisman, UCSD

September 26-30, 2005 University of California, San Diego California Institute for Telecommunications and Information Technology The Networking Double Header of the Century Will Be Driven by LambdaGrid Applications i Grid 2 oo 5 T H E G L O B A L L A M B D A I N T E G R A T E D F A C I L I T Y Maxine Brown, Tom DeFanti, Co-Organizers www.startap.net/igrid2005/ http://sc05.supercomp.org

The OptiPuter, Quartzite, and Starlight Projects: A Campus to Global-Scale Testbed for Optical Technologies Enabling LambdaGrid Computing

More Related Content

What's hot (20)

Viewers also liked (20)

Similar to The OptiPuter, Quartzite, and Starlight Projects: A Campus to Global-Scale Testbed for Optical Technologies Enabling LambdaGrid Computing (20)

More from Larry Smarr (20)

Recently uploaded (20)

The OptiPuter, Quartzite, and Starlight Projects: A Campus to Global-Scale Testbed for Optical Technologies Enabling LambdaGrid Computing