05 Preparing for Extreme Geterogeneity in HPC

ORNL is managed by UT-Battelle, LLC for the US Department of Energy
Preparing for Extreme Heterogeneity
in High Performance Computing
Jeffrey S. Vetter
With many contributions from FTG Group and Colleagues
R-CCS International Symposium
Kobe
18 Feb 2020
ORNL is managed by UT-Battelle
for the US Department of Energy http://ft.ornl.gov vetter@computer.org

1616
Highlights
• Recent trends in extreme-scale HPC paint an ambiguous future
– Contemporary systems provide evidence that power constraints are driving architectures to change rapidly
– Multiple architectural dimensions are being (dramatically) redesigned: Processors, node design, memory systems, I/O
– Complexity is our main challenge
• Applications and software systems are all reaching a state of crisis
– Applications will not be functionally or performance portable across architectures
– Programming and operating systems need major redesign to address these architectural changes
– Procurements, acceptance testing, and operations of today’s new platforms depend on performance prediction and
benchmarking.
• We need portable programming models and performance prediction now more than ever!
– Heterogeneous processing
• OpenACC->FGPAs
• Intelligent runtime system (IRIS)
• Clacc – OpenACC support in LLVM (not covered today)
• OpenACC dialect of MLIR for Flang Fortran (not covered today)
– Emerging memory hierarchies (NVM)
• DRAGON – transparent NVM access from GPUs (not covered today)
• NVL-C – user management of nonvolatile memory in C (not covered today)
• Papyrus – parallel aggregate persistent storage (not covered today)
• Performance prediction is critical for design and optimization (not covered today)

3030
History
Q: Think back 10 years. How
many of you would have
predicted that many of our
top HPC systems would be
GPU-based architectures?
Yes
No
Revisionists 

3232
Future
Q: Think forward 10 years.
How many of you predict
that most of our top HPC
systems will have the
following architectural
features?
Assume general purpose multicore CPU
GPU
FPGA/Reconfigurable processor
Neuromorphic processor
Deep learning processor
Quantum processor
RISC-V processor
Some new unknown processor
All/some of the above in one SoC

3434
Implications
Q: Now, imagine you are building
a new application with an
expected ~3M LOC and 20 team
members over the next 10 years.
What on-node programming
model/system do you use?
C, C++ XX, Fortran XX
Metaprogramming, etc (e.g., AMP, Kokkos, RAJA, SYCL)
CUDA, cu***, HIP, OpenCL
Directives: OpenMP XX, OpenACC XX
R, Python, Matlab, etc
A Domain Specific Language (e.g., Claw, PySL)
A Domain Specific Framework (e.g., PetSc)
Some new unknown programming approach
All/some of the above

35
The FTG Vision
Architectures
Multicore CPU GPU FPGA AI Accelerator SoC DSP Deep Memory
Persistent
Memory
Neuromorphic
Applications
Science and
Engineering (e.g., CFD,
Materials, Fusion)
Streaming
(e.g., SW Radio,
Experimental instrument)
Sensing
(e.g., SAR, vision)
Deep learning
(e.g., CNN)
Analytics
(e.g., graphs)
Robotics
(e.g., sense and react)
Programming Systems
Compiler
Domain Specific
Languages
Just-in-time
Compilation
Metaprogramming Scripting Libraries Autotuning
Runtime and Operating Systems
Discovery
Task Scheduling
and Mapping
Data Orchestration IO Synchronization Load balancing
Performance
Productivity
EnergyEfficiency

36
The FTG Vision | Applications
Architectures
Persistent
Memory
Neuromorphic
Applications
Science and
Materials, Fusion)
Streaming
(e.g., SW Radio,
Sensing
(e.g., SAR, vision)
Deep learning
(e.g., CNN)
Analytics
(e.g., graphs)
Robotics
Programming Systems
Compiler
Domain Specific
Languages
Just-in-time
Compilation
Discovery
Task Scheduling
and Mapping
Performance
Productivity
EnergyEfficiency

37
National security
Stockpile
stewardship
Next-generation
electromagnetics
simulation of hostile
environment and
virtual flight testing for
hypersonic re-entry
vehicles
Energy security
Turbine wind plant
efficiency
High-efficiency,
low-emission
combustion engine
and gas turbine
design
Materials design for
extreme
environments of
nuclear fission
and fusion reactors
Design and
commercialization
of Small Modular
Reactors
Subsurface use
for carbon capture,
petroleum extraction,
waste disposal
Scale-up of clean
fossil fuel combustion
Biofuel catalyst
design
Scientific discovery
Find, predict,
and control materials
and properties
Cosmological probe
of the standard model
of particle physics
Validate fundamental
laws of nature
Demystify origin of
chemical elements
Light source-enabled
analysis of protein
and molecular
structure and design
Whole-device model
of magnetically
confined fusion
plasmas
Earth system
Accurate regional
impact assessments
in Earth system
models
Stress-resistant crop
analysis and catalytic
conversion
of biomass-derived
alcohols
Metagenomics
for analysis of
biogeochemical
cycles, climate
change,
environmental
remediation
Economic security
Additive
manufacturing
of qualifiable
metal parts
Reliable and
efficient planning
of the power grid
Seismic hazard
risk assessment
Urban planning
Health care
Accelerate
and translate
cancer research
ECP applications target national problems in 6 strategic areas
https://exascaleproject.org/

38
DARPA Domain Specific System on Chip Program is investigating Performance
Portability of Software Defined Radio
• Signal processing: An open-
source implementation of IEEE-
802.11 WIFI a/b/g with GR OOT
modules.
• Input / Output file support via
Socket PDU (UDP server) blocks
• Image/Video transcoding with
OpenCL/OpenCV
Video/Image
Files
GR IEEE-802.11 Transmit (TX)
UDP
Antenna
UDP
IEEE-802.11 Receive (RX)
Xavier SoC #1 Xavier SoC #2

39
The FTG Vision | Architectures
Architectures
Persistent
Memory
Neuromorphic
Applications
Science and
Materials, Fusion)
Streaming
(e.g., SW Radio,
Sensing
(e.g., SAR, vision)
Deep learning
(e.g., CNN)
Analytics
(e.g., graphs)
Robotics
Programming Systems
Compiler
Domain Specific
Languages
Just-in-time
Compilation
Discovery
Task Scheduling
and Mapping
Performance
Productivity
EnergyEfficiency

46
Contemporary devices are approaching fundamental limits
I.L. Markov, “Limits on fundamental limits to computation,” Nature, 512(7513):147-54,
2014, doi:10.1038/nature13570.
Economist, Mar 2016
R.H. Dennard, F.H. Gaensslen, V.L. Rideout, E. Bassous, and A.R. LeBlanc, “Design of ion-implanted
MOSFET's with very small physical dimensions,” IEEE Journal of Solid-State Circuits, 9(5):256-68, 1974,
Dennard scaling has already ended. Dennard observed that voltage and
current should be proportional to the linear dimensions of a transistor: 2x
transistor count implies 40% faster and 50% more efficient.

4747
End of Moore’s Law : what’s your prediction ??
Economist, Mar 2016
“The number of people predicting the death of Moore’s Law doubles every two years.” – Peter Lee, Microsoft

50
Sixth Wave of Computing
http://www.kurzweilai.net/exponential-growth-of-computing
Transition
Period
6th wave

5151
Predictions for Transition Period
Optimize Software and
Expose New Hierarchical
Parallelism
• Redesign software to
boost performance on
upcoming
architectures
• Exploit new levels of
parallelism and
efficient data
movement
Architectural
Specialization and
Integration
• Use CMOS more
effectively for specific
workloads
• Integrate components
to boost performance
and eliminate
inefficiencies
• Workload specific
memory+storage
system design
Emerging Technologies
• Investigate new
computational
paradigms
• Quantum
• Neuromorphic
• Advanced Digital
• Emerging Memory
Devices

5252
Parallelism
upcoming
architectures
parallelism and
efficient data
movement
Architectural
Specialization and
Integration
• Use CMOS more
workloads
and eliminate
inefficiencies
memory+storage
system design
• Investigate new
computational
paradigms
• Quantum
• Neuromorphic
• Emerging Memory
Devices

5353
Parallelism
upcoming
architectures
parallelism and
efficient data
movement
Architectural
Specialization and
Integration
• Use CMOS more
workloads
and eliminate
inefficiencies
memory+storage
system design
• Investigate new
computational
paradigms
• Quantum
• Neuromorphic
• Emerging Memory
Devices

5454
Quantum computing: Qubit design and fabrication
have made recent progress but still face challenges
Science 354, 1091 (2016) – 2 December
http://nap.edu/25196

58
Fun Question: when was the field effect transistor patented?
https://www.edn.com/electronics-blogs/edn-moments/4422371/Lilienfeld-patents-field-effect-
transistor--October-8--1926

5959
Parallelism
upcoming
architectures
parallelism and
efficient data
movement
Architectural
Specialization and
Integration
• Use CMOS more
workloads
and eliminate
inefficiencies
memory+storage
system design
• Investigate new
computational
paradigms
• Quantum
• Neuromorphic
• Emerging Memory
Devices

6060
https://www.thebroadcastbridge.com/content/entry/1094/altera-announces-arria-10-2666mbps-ddr4-memory-fpga-interface
Pace of Architectural Specialization is Quickening
• Industry, lacking Moore’s Law, will need to continue to
differentiate products (to stay in business)
– Use the same transistors differently to enhance performance
• Architectural design will become extremely important,
critical
– Dark Silicon
– Address new parameters for benefits/curse of Moore’s Law
• 50+ new companies focusing on hardware for Machine
Learning
http://www.wired.com/2016/05/google-tpu-custom-chips/
D.E. Shaw, M.M. Deneroff, R.O. Dror et al., “Anton, a special-purpose machine for molecular dynamics
simulation,” Communications of the ACM, 51(7):91-7, 2008.
http://www.theinquirer.net/inquirer/news/2477796/intels-nervana-
ai-platform-takes-aim-at-nvidias-gpu-techology
https://fossbytes.com/nvidia-volta-gddr6-2018/
Xilinx ACAP
HotChips 2018
HotChips 2018

6262
Analysis of Apple A-* SoCs
http://vlsiarch.eecs.harvard.edu/accelerators/die-photo-analysis

6767
Intel Stratix 10 FPGA
Experimental Computing Lab (ExCL) managed by the ORNL Future Technologies Group
• Intel Stratix 10 FPGA and four banks of DDR4 external
memory
– Board configuration: Nallatech 520 Network Acceleration Card
• Up to 10 TFLOPS of peak single precision performance
• 25MBytes of L1 cache @ up to 94 TBytes/s peak
bandwidth
• 2X Core performance gains over Arria® 10
• Quartus and OpenCL software (Intel SDK v18.1) for
using FPGA
• Provide researcher access to advanced FPGA/SOC
environment
https://excl.ornl.gov/
Mar 2019
For more information or to apply for an account, visit https://excl.ornl.gov/

68
NVIDIA Jetson AGX Xavier SoC
• NVIDIA Jetson AGX Xavier:
• High-performance system on a chip for autonomous
machines
• Heterogeneous SoC contains:
– Eight-core 64-bit ARMv8.2 CPU cluster (Carmel)
– 1.4 CUDA TFLOPS (FP32) GPU with additional
inference optimizations (Volta)
– 11.4 DL TOPS (INT8) Deep learning accelerator
(NVDLA)
– 1.7 CV TOPS (INT8) 7-slot VLIW dual-processor
Vision accelerator (PVA)
– A set of multimedia accelerators (stereo, LDC,
optical flow)
• Provides researchers access to advanced high-
performance SOC environment
Mar 2019

69
Qualcomm 855 SoC (SM8510P) Snapdragon™
Adreno 640
Hexagon 690
Kyro 485
Kyro 485 (8-ARM Prime+BigLittle Cores)
Prime
Core
Hexagon 690 (DSP + AI)
7nm TSMC
Adreno 640
• Vulkan, OpenCL, OpenGL ES 3.1
• Apps: HDR10+, HEVC, Dolby, etc
• Enables 8k-360o VR video playback
• 20% faster compared to Adreno 630
• Quad threaded Scalar Core
• DSP + 4 Hexagon Vector Xccelerators
• New Tensor Xccelerator for AI
• Apps: AI, Voice Assistance, AV codecs
• Snapdragon X24 LTE (855 built-in) modem LTE Category 20
• Snapdragon X50 5G (external) modem (for 5G devices)
• Qualcomm Wi-Fi 6-ready mobile platform: (802.11ax-ready,
802.11ac Wave 2, 802.11ay, 802.11ad)
• Qualcomm 60 GHz Wi-Fi mobile platform: (802.11ay,
802.11ad)
• Bluetooth Version: 5.0
• Bluetooth Speed: 2 Mbps
• High accuracy location with dual-frequency GNSS.
Connectivity (5G)
Spectra 360 ISP
• New dedicated Image Signal Processor (ISP)
• Dual 14-bit CV-ISPs; 48MP @ 30fps single camera
• Hardware CV for object detection, tracking, streo depth process
• 6DoF XR Body tracking, H265, 4K60 HDR video capture, etc.
Spectra 360
5G
Qualcomm Development Board connected to (mcmurdo) HPZ820
• Connected Qualcomm board to HPZ820 through USB
• Development Environment: Android SDK/NDK
• Login to mcmurdo machine
$ ssh –Y mcmurdo
• Setup Android platform tools and development environment
$ source /home/nqx/setup_android.source
• Run Hello-world on ARM cores
$ git clone https://code.ornl.gov/nqx/helloworld-android
$ make compile push run
• Run OpenCL example on GPU
$ git clone https://code.ornl.gov/nqx/opencl-img-processing
• Run Sobel edge detection
$ make compile push run fetch
• Login to Qualcomm development board shell
$ adb shell
$ cd /data/local/tmp
Created by Narasinga Rao Miniskar, Steve Moulton
© Qualcomm Inc.
© Qualcomm Inc.

7070
Growing Open Source Hardware Movement Enables Rapid Chip Design
RISC-V Summit, 2018

7171
DARPA ERI Programs Aiming for Agile (and Frequent) Chip Creation
A. Olofsson, 2018

7272
Summary:
Transition Period will be Disruptive – Opportunities and Pitfalls Abound
• New devices and architectures may not
be hidden in traditional levels of
abstraction
• Examples
– A new type of CNT transistor may be
completely hidden from higher levels
– A new paradigm like quantum may require
new architectures, programming models, and
algorithmic approaches
Layer Switch, 3D NVM Approximate Neuro Quantum
Application 1 1 2 2 3
Algorithm 1 1 2 3 3
Language 1 2 2 3 3
API 1 2 2 3 3
Arch 1 2 2 3 3
ISA 1 2 2 3 3
Microarch 2 3 2 3 3
FU 2 3 2 3 3
Logic 3 3 2 3 3
Device 3 3 2 3 3
Adapted from IEEE Rebooting Computing Chart

103103
LLNL
IBM/NVIDIA
Department of Energy (DOE) Roadmap to Exascale Systems
An impressive, productive lineup of accelerated node systems supporting DOE’s mission
ANL
IBM BG/Q
ORNL
Cray/AMD/NVIDIA
LBNL
Cray/AMD/NVIDIA
LANL/SNL
TBD
ANL
Intel/Cray
ORNL
AMD/Cray
LLNL
TBD
LANL/SNL
Cray/Intel Xeon/KNL
2012 2016 2018 2020 2021-2023
ORNL
IBM/NVIDIA
LLNL
IBM BG/Q
Sequoia (10)
Cori (12)
Trinity (6)
Theta (24)Mira (21)
Titan (9) Summit (1)
NERSC-9
Perlmutter
Aurora
ANL
Cray/Intel KNL
LBNL
Cray/Intel Xeon/KNL
First U.S. Exascale Systems
Sierra (2)
Pre-Exascale Systems [Aggregate Linpack (Rmax) = 323 PF!]
Jan 2018
Heterogeneous Cores
Deep Memory incl NVM
Plateauing I/O Performance

118118
Frontier Continues the Accelerated Node Design
• Partnership between ORNL, Cray, and AMD
• The Frontier system will be delivered in 2021
• Peak Performance greater than 1.5 EF
• Composed of more than 100 Cray Shasta cabinets
– Connected by Slingshot™ interconnect with adaptive routing, congestion control,
and quality of service
• Accelerated Node Architecture:
– One purpose-built AMD EPYC™ processor
– Four HPC and AI optimized Radeon Instinct™ GPU accelerators
– Fully connected with high speed AMD Infinity Fabric links
– Coherent memory across the node
– 100 GB/s injection bandwidth
– Near-node NVM storage

119
Comparison of Titan, Summit, and Frontier Systems
System Specs
Titan Summit Frontier
Peak 27 PF 200 PF ~1.5 EF
# cabinets 200 256 > 100
Node
1 AMD Opteron CPU
1 NVIDIA Kepler GPU
2 IBM POWER9™ CPUs
6 NVIDIA Volta GPUs
1 AMD EPYC CPU
4 AMD Radeon Instinct GPUs
On-node
interconnect
PCI Gen2
No coherence
across the node
NVIDIA NVLINK
Coherent memory
across the node
AMD Infinity Fabric
Coherent memory
across the node
System
Interconnect
Cray Gemini network
6.4 GB/s
Mellanox Dual-port EDR IB network
25 GB/s
Cray four-port Slingshot network
100 GB/s
Topology 3D Torus Non-blocking Fat Tree Dragonfly
Storage
32 PB, 1 TB/s, Lustre
Filesystem
250 PB, 2.5 TB/s, IBM Spectrum
Scale™ with GPFS™
2-4x performance and capacity
of Summit’s I/O subsystem.
On-node NVM No Yes Yes
Power 9 MV 13 MV 29 MV

120
Complex architectures yield…
System: MPI, Legion, HPX, Charm++, etc
Low overhead
Resource contention
Locality
Node: OpenMP, Pthreads, U-threads, etc
SIMD
NUMA, HBM
Cores: OpenACC, CUDA, OpenCL, OpenMP4, …
Memory use, coalescing Data orchestration Fine grained parallelism Hardware features
Complex
Programming
Models

121
During this Sixth Wave transition, Complexity is our major challenge!
Design
How do we design future systems so
that they are better than current
systems on important applications?
• Simulation and modeling are more difficult
• Entirely possible that the new system will be
slower than the old system!
• Expect ‘disaster’ procurements
Programmability
How do we design applications with
some level of performance portability?
• Software lasts much longer than transient
hardware platforms
• Proper abstractions for flexibility and
efficiency
• Adapt or die

124
The FTG Vision | Programming Systems
Architectures
Persistent
Memory
Neuromorphic
Applications
Science and
Materials, Fusion)
Streaming
(e.g., SW Radio,
Sensing
(e.g., SAR, vision)
Deep learning
(e.g., CNN)
Analytics
(e.g., graphs)
Robotics
Programming Systems
Compiler
Domain Specific
Languages
Just-in-time
Compilation
Discovery
Task Scheduling
and Mapping
Performance
Productivity
EnergyEfficiency

125
What more to say ?!?!? 

131131
Directive-based Strategy with OpenARC: Open Accelerator
Research Compiler
• Open-Sourced, High-Level Intermediate
Representation (HIR)-Based, Extensible
Compiler Framework.
– Perform source-to-source translation from
OpenACC C to target accelerator models.
• Support full features of OpenACC V1.0 ( + array
reductions and function calls)
• Support both CUDA and OpenCL as target accelerator
models
– Provide common runtime APIs for various back-
ends
– Can be used as a research framework for various
study on directive-based accelerator computing.
• Built on top of Cetus compiler framework, equipped with
various advanced analysis/transformation passes and
built-in tuning tools.
• OpenARC’s IR provides an AST-like syntactic view of the
source program, easy to understand, access, and
transform the input program.
S. Lee and J.S. Vetter, “OpenARC: Open Accelerator Research Compiler for Directive-Based, Efficient Heterogeneous Computing,”
in ACM Symposium on High-Performance Parallel and Distributed Computing (HPDC). Vancouver: ACM, 2014

149149
FPGAs| Approach
• Design and implement an OpenACC-to-FPGA translation
framework, which is the first work to use a standard and portable
directive-based, high-level programming system for FPGAs.
• Propose FPGA-specific optimizations and novel pragma
extensions to improve performance.
• Evaluate the functional and performance portability of the
framework across diverse architectures (Altera FPGA, NVIDIA
GPU, AMD GPU, and Intel Xeon Phi).
S. Lee, J. Kim, and J.S. Vetter, “OpenACC to FPGA: A Framework for Directive-based High-Performance Reconfigurable Computing,” Proc. IEEE
International Parallel & Distributed Processing Symposium (IPDPS), 2016, 10.1109/IPDPS.2016.28.

153
FPGA OpenCL Architecture
FPGA
Memory
Local Memory
Interconnect
Local Memory
Interconnect
Local Memory
Interconnect
Memory
Memory
Memory
Memory
Memory
Global Memory Interconnect
PCIe
External Memory
Controller and PHY
External Memory
Controller and PHYHostProcessor
External DDR Memory External DDR Memory
Kernel
PipelineKernel
PipelineKernel
PipelineKernel
Pipeline
Kernel
PipelineKernel
PipelineKernel
PipelineKernel
Pipeline
Kernel
PipelineKernel
PipelineKernel
PipelineKernel
Pipeline
Pipeline
Depth
Vector
Width
Number of Replicated Compute Units

156156
Kernel-Pipelining Transformation Optimization
• Kernel execution model in OpenACC
– Device kernels can communicate with
each other only through the device
global memory.
– Synchronizations between kernels are
at the granularity of a kernel
execution.
• Altera OpenCL channels
– Allows passing data between kernels
and synchronizing kernels with high
efficiency and low latency
Global Memory
Kernel 1 Kernel 2
Global Memory
Kernel 1 Kernel 2Channel
Kernel communications through
global memory in OpenACC
Kernel communications with
Altera channels

157
Kernel-Pipelining Transformation Optimization (2)
#pragma acc data copyin (a) create (b) copyout (c)
{
#pragma acc kernels loop gang worker present (a, b)
for(i=0; i<N; i++) { b[i] = a[i]*a[i]; }
#pragma acc kernels loop gang worker present (b, c)
for(i=0; i<N; i++) {c[i] = b[i]; }
}
channel float pipe_b;
__kernel void kernel1(__global float* a) {
int i = get_global_id(0);
write_channel_altera(pipe_b, a[i]*a[i]);
}
__kernel void kernel2(__global float* c) {
int i = get_global_id(0);
c[i] = read_channel_altera(pipe_b);
}
(a) Input OpenACC code
(b) Altera OpenCL code with channels
Global Memory
Kernel 1 Kernel 2
Global Memory

158
Kernel-Pipelining Transformation Optimization (3)
#pragma acc data copyin (a) create (b) copyout (c)
{
#pragma acc kernels loop gang worker present (a, b)
for(i=0; i<N; i++) { b[i] = a[i]*a[i]; }
#pragma acc kernels loop gang worker present (b, c)
for(i=0; i<N; i++) {c[i] = b[i]; }
}
(a) Input OpenACC code
(c) Modified OpenACC code for kernel-pipelining
Global Memory
Kernel 1 Kernel 2
Global Memory
#pragma acc data copyin (a) pipe (b) copyout (c)
{
#pragma acc kernels loop gang worker pipeout (b) present (a)
For(i=0; i<N; i++) { b[i] = a[i]*a[i]; }
#pragma acc kernels loop gang worker pipein (b) present (c)
For(i=0; i<N; i++) {c[i] = b[i];}
}
Kernel-pipelining
transformation
Valid under
specific conditions

175
Overall Performance of OpenARC FPGA Evaluation
FPGAs prefer applications with deep execution pipelines (e.g., FFT-1D and
FFT-2D), performing much higher than other accelerators.
For traditional HPC applications with abundant parallel floating-point operations,
it seems to be difficult for FPGAs to beat the performance of other accelerators,
even though FPGAs can be much more power-efficient.
• Tested FPGA does not contain dedicated, embedded floating-point
cores, while others have fully-optimized floating-point computation units.
Current and upcoming high-end FPGAs are equipped with hardened floating-
point operators, whose performance will be comparable to other accelerators,
while remaining power-efficient.

187
The FTG Vision | Runtime and Operating Systems
Architectures
Persistent
Memory
Neuromorphic
Applications
Science and
Materials, Fusion)
Streaming
(e.g., SW Radio,
Sensing
(e.g., SAR, vision)
Deep learning
(e.g., CNN)
Analytics
(e.g., graphs)
Robotics
Programming Systems
Compiler
Domain Specific
Languages
Just-in-time
Compilation
Discovery
Task Scheduling
and Mapping
Performance
Productivity
EnergyEfficiency

189
IRIS: Mapping Strategy for Heterogeneous Architectures and Native Programming
Models
ARM CPU
NVIDIA GPU
OpenACC
Intel FPGA
CUDA
OpenMP
Intel OpenCL
General
Accelerators
OpenCL
CPU/Xeon PhiOpenMP
SYCL
HIP AMD GPU
IRISCommonRuntimeAPI
IRIS offers a common API for diverse heterogeneous
devices and also allows intermixing of multiple
programming models (mix CUDA, OpenMP, OpenCL, etc.).
Support more
programming
models.

190
IRIS: An Intelligent Runtime System for Extremely Heterogeneous
Architectures
• Provide programmers a unified programming
environment to write portable code across
heterogeneous architectures (and preferred
programming systems)
• Orchestrate diverse programming systems
(OpenCL, CUDA, HIP, OpenMP for CPU) in a single
application
– OpenCL
• NVIDIA GPU, AMD GPU, ARM GPU, Qualcomm GPU, Intel
CPU, Intel Xeon Phi, Intel FPGA, Xilinx FPGA
– CUDA
• NVIDIA GPU
– HIP
• AMD GPU
– OpenMP for CPU
• Intel CPU, AMD CPU, PowerPC CPU, ARM CPU,
Qualcomm CPU

191191
The IRIS Architecture
• Platform Model
– A single-node system equipped with host CPUs
and multiple compute devices (GPUs, FPGAs,
Xeon Phis, and multicore CPUs)
• Memory Model
– Host memory + shared device memory
– All compute devices share the device memory
• Execution Model
– DAG-style task parallel execution across all
available compute devices
• Programming Model
– High-level OpenACC, OpenMP4, SYCL* (*
planned)
– Low-level C/Fortran/Python IRIS host-side
runtime API + OpenCL/CUDA/HIP/OpenMP
kernels (w/o compiler support)

192
Supported Architectures and Programming Systems by IRIS
ExCL* Systems Oswald Summit-node Radeon Xavier Snapdragon
CPU Intel Xeon IBM Power9 Intel Xeon ARMv8
Qualcomm
Kryo
Programming Systems • Intel OpenMP
• Intel OpenCL
• IBM XL OpenMP • Intel OpenMP
• Intel OpenCL
• GNU GOMP • Android NDK
OpenMP
GPU NVIDIA P100 NVIDIA V100 AMD Radeon VII NVIDIA Volta
Qualcomm
Adreno 640
Programming Systems • NVIDIA CUDA
• NVIDIA OpenCL
• NVIDIA CUDA • AMD HIP
• AMD OpenCL
• NVIDIA CUDA • Qualcomm OpenCL
FPGA
Intel/Altera
Stratix 10
Programming Systems • Intel OpenCL
* ORNL Experimental Computing Laboratory (ExCL) https://excl.ornl.gov/

193
IRIS Booting on Various Platforms

194194
Task Scheduling in IRIS
• A task
– A scheduling unit
– Contains multiple in-order commands
• Kernel launch command
• Memory copy command (device-to-host, host-to-device)
– May have DAG-style dependencies with other tasks
– Enqueued to the application task queue with a device
selection policy
• Available device selection policies
– Specific Device (compute device #)
– Device Type (CPU, GPU, FPGA, XeonPhi)
– Profile-based
– Locality-aware
– Ontology-base
– Performance models (Aspen)
– Any, All, Random, 3rd-party users’ custom policies
• The task scheduler dispatches the tasks in the
application task queue to available compute devices
– Select the optimal target compute device according to
task’s device selection policy

195195
SAXPY Example on Xavier
• Computation
– S[] = A * X[] + Y[]
• Two tasks
– S[] = A * X[] on NVIDIA GPU (CUDA)
– S[] += Y[] on ARM CPU (OpenMP)
• S[] is shared between two tasks
• Read-after-write (RAW), true dependency
• Low-level Python IRIS host code +
CUDA/OpenMP kernels
– saxpy.py
– kernel.cu
– kernel.openmp.h

196
SAXPY: Python host code & CUDA kernel code
saxpy.py (1/2)
#!/usr/bin/env python
import iris
import numpy as np
import sys
iris.init()
SIZE = 1024
A = 10.0
x = np.arange(SIZE, dtype=np.float32)
y = np.arange(SIZE, dtype=np.float32)
s = np.arange(SIZE, dtype=np.float32)
print 'X', x
print 'Y', y
mem_x = iris.mem(x.nbytes)
mem_y = iris.mem(y.nbytes)
mem_s = iris.mem(s.nbytes)
saxpy.py (2/2)
kernel0 = iris.kernel("saxpy0")
kernel0.setmem(0, mem_s, iris.iris_w)
kernel0.setint(1, A)
kernel0.setmem(2, mem_x, iris.iris_r)
off = [ 0 ]
ndr = [ SIZE ]
task0 = iris.task()
task0.h2d_full(mem_x, x)
task0.kernel(kernel0, 1, off, ndr)
task0.submit(iris.iris_gpu)
kernel1.setmem(0, mem_s, iris.iris_rw)
kernel1.setmem(1, mem_y, iris.iris_r)
task1 = iris.task()
task1.h2d_full(mem_y, y)
task1.d2h_full(mem_s, s)
task1.submit(iris.iris_cpu)
print 'S =', A, '* X + Y', s
iris.finalize()
kernel.cu (CUDA)
extern "C" __global__ void saxpy0(float* S, float
A, float* X) {
int id = blockIdx.x * blockDim.x + threadIdx.x;
S[id] = A * X[id];
}
extern "C" __global__ void saxpy1(float* S,
float* Y) {
int id = blockIdx.x * blockDim.x + threadIdx.x;
S[id] += Y[id];
}

197
SAXPY: Python host code & OpenMP kernel code
kernel.openmp.h (OpenMP)
#include <iris/iris_openmp.h>
static void saxpy0(float* S, float A, float* X,
IRIS_OPENMP_KERNEL_ARGS) {
int id;
#pragma omp parallel for shared(S, A, X)
private(id)
IRIS_OPENMP_KERNEL_BEGIN
S[id] = A * X[id];
IRIS_OPENMP_KERNEL_END
}
static void saxpy1(float* S, float* Y,
IRIS_OPENMP_KERNEL_ARGS) {
int id;
#pragma omp parallel for shared(S, Y) private(id)
IRIS_OPENMP_KERNEL_BEGIN
S[id] += Y[id];
IRIS_OPENMP_KERNEL_END
}
saxpy.py (1/2)
import iris
import numpy as np
import sys
iris.init()
SIZE = 1024
A = 10.0
print 'X', x
print 'Y', y
saxpy.py (2/2)
off = [ 0 ]
ndr = [ SIZE ]
task0 = iris.task()
task1 = iris.task()
print 'S =', A, '* X + Y', s
iris.finalize()

198
Memory Consistency Management
saxpy.py (1/2)
import iris
import numpy as np
import sys
iris.init()
SIZE = 1024
A = 10.0
print 'X', x
print 'Y', y
saxpy.py (2/2)
off = [ 0 ]
ndr = [ SIZE ]
task0 = iris.task()
task1 = iris.task()
print 'S =', A, '* X + Y', s
iris.finalize()
mem_s is
shared between
GPU and CPU

199199
Locality-aware Device Selection Policy
saxpy.py (1/2)
import iris
import numpy as np
import sys
iris.init()
SIZE = 1024
A = 10.0
print 'X', x
print 'Y', y
saxpy.py (2/2)
off = [ 0 ]
ndr = [ SIZE ]
task0 = iris.task()
task1 = iris.task()
task1.submit(iris.iris_data)
print 'S =', A, '* X + Y', s
iris.finalize()
iris_data selects
the device that
requires
minimum data
transfer to
execute the
task

201
The FTG Vision
Architectures
Persistent
Memory
Neuromorphic
Applications
Science and
Materials, Fusion)
Streaming
(e.g., SW Radio,
Sensing
(e.g., SAR, vision)
Deep learning
(e.g., CNN)
Analytics
(e.g., graphs)
Robotics
Programming Systems
Compiler
Domain Specific
Languages
Just-in-time
Compilation
Discovery
Task Scheduling
and Mapping
Performance
Productivity
EnergyEfficiency
IRIS
OpenARC

204204
Recap
• Motivation: Recent trends in computing
paint an ambiguous future
– Multiple architectural dimensions are being
(dramatically) redesigned: Processors, node
design, memory systems, I/O
– Complexity is our main challenge
• Applications and software systems across
many areas are all reaching a state of crisis
– Need a focus on performance portability
• ORNL FTG investigating design and
programming challenges for these trends
– Performance modeling and ontologies
– Performance portable compilation to many
different heterogeneous architectures/SoCs
– Intelligent scheduling system to automate
discovery, device selection, and data movement
– Targeting wide variety of existing and future
architectures (DSSoC and others)
• Visit us
– We host interns and other visitors year
round
• Faculty, grad, undergrad, high school,
industry
• Jobs in FTG
– Postdoctoral Research Associate in
Computer Science
– Software Engineer
– Computer Scientist
– Visit https://jobs.ornl.gov
• Contact me vetter@ornl.gov

205205
Final Report on Workshop on Extreme Heterogeneity
1. Maintaining and improving programmer productivity
– Flexible, expressive, programming models and languages
– Intelligent, domain-aware compilers and tools
– Composition of disparate software components
• Managing resources intelligently
– Automated methods using introspection and machine learning
– Optimize for performance, energy efficiency, and availability
• Modeling & predicting performance
– Evaluate impact of potential system designs and application mappings
– Model-automated optimization of applications
• Enabling reproducible science despite non-determinism & asynchrony
– Methods for validation on non-deterministic architectures
– Detection and mitigation of pervasive faults and errors
• Facilitating Data Management, Analytics, and Workflows
– Mapping of science workflows to heterogeneous hardware and software services
– Adapting workflows and services to meet facility-level objectives through learning approaches
https://orau.gov/exheterogeneity2018/ https://doi.org/10.2172/1473756

05 Preparing for Extreme Geterogeneity in HPC

More Related Content

What's hot (20)

Similar to 05 Preparing for Extreme Geterogeneity in HPC (20)

More from RCCSRENKEI (20)

Recently uploaded (20)

05 Preparing for Extreme Geterogeneity in HPC