Hpc 3
Download as PPTX, PDF0 likes722 views
This document discusses the convergence of parallel architectures. It describes how historically parallel architectures were tied to specific programming models, but there is now a more standardized approach. A modern layered framework includes programming models, communication abstractions, operating system support, and communication hardware. Shared memory, message passing, and data parallel programming models are discussed in detail. The document also examines how architectural models have evolved from being tailored to specific programming models to a more generalized approach, contributing to the convergence of parallel architectures.
Hpc 3
- 1. HIGH PERFORMANCE COMPUTING LECT_03 Convergence of Parallel Architectures-I BATCH: 11BS(IT) PREPARED BY: MUKHTIAR AHMED Asst. Prof. I.T Department
- 2. 2 History Application Software System Software SIMD Message Passing Shared Memory Dataflow Systolic Arrays Architecture • Uncertainty of direction paralyzed parallel software development! Historically, parallel architectures tied to programming models • Divergent architectures, with no predictable pattern of growth.
- 3. 3 Today Extension of “computer architecture” to support communication and cooperation • OLD: Instruction Set Architecture • NEW: Communication Architecture Defines • Critical abstractions, boundaries, and primitives (interfaces) • Organizational structures that implement interfaces (hw or sw) Compilers, libraries and OS are important bridges today
- 4. 4 Modern Layered Framework CAD Multiprogramming Shared address Message passing Data parallel Database Scientific modeling Parallel applications Programming models Communication abstraction User/system boundary Compilation or library Operating systems support Communication hardware Physical communication medium Hardware/software boundary
- 5. 5 Programming Model What programmer uses in coding applications Specifies communication and synchronization Examples: • Multiprogramming: no communication or synch. at program level • Shared address space: like bulletin board • Message passing: like letters or phone calls, explicit point to point • Data parallel: more regimented, global actions on data – Implemented with shared address space or message passing
- 6. 6 Communication Abstraction User level communication primitives provided • Realizes the programming model • Mapping exists between language primitives of programming model and these primitives Supported directly by hw, or via OS, or via user sw Lot of debate about what to support in sw and gap between layers Today: • Hw/sw interface tends to be flat, i.e. complexity roughly uniform • Compilers and software play important roles as bridges today • Technology trends exert strong influence Result is convergence in organizational structure • Relatively simple, general purpose communication primitives
- 7. 7 Communication Architecture = User/System Interface + Implementation User/System Interface: • Comm. primitives exposed to user-level by hw and system-level sw Implementation: • Organizational structures that implement the primitives: hw or OS • How optimized are they? How integrated into processing node? • Structure of network Goals: • Performance • Broad applicability • Programmability • Scalability • Low Cost
- 8. 8 Evolution of Architectural Models Historically machines tailored to programming models • Prog. model, comm. abstraction, and machine organization lumped together as the “architecture” Evolution helps understand convergence • Identify core concepts • Shared Address Space • Message Passing • Data Parallel Others: • Dataflow • Systolic Arrays Examine programming model, motivation, intended applications, and contributions to convergence
- 9. 9 Shared Address Space Architectures Any processor can directly reference any memory location • Communication occurs implicitly as result of loads and stores Convenient: • Location transparency • Similar programming model to time-sharing on uniprocessors – Except processes run on different processors – Good throughput on multiprogrammed workloads Naturally provided on wide range of platforms • History dates at least to precursors of mainframes in early 60s • Wide range of scale: few to hundreds of processors Popularly known as shared memory machines or model • Ambiguous: memory may be physically distributed among processors
- 10. 10 Shared Address Space Model Process: virtual address space plus one or more threads of control Portions of address spaces of processes are shared •Writes to shared address visible to other threads (in other processes too) •Natural extension of uniprocessors model: conventional memory operations for comm.; special atomic operations for synchronization •OS uses shared memory to coordinate processes St or e P1 P2 Pn P0 Load P0 pr i vat e P1 pr i vat e P2 pr i vat e Pn pr i vat e Virtual address spaces for a collection of processes communicating via shared addresses Machine physical address space Shared portion of address space Private portion of address space Common physical addresses
- 11. 11 Communication Hardware Also natural extension of uniprocessor Already have processor, one or more memory modules and I/O controllers connected by hardware interconnect of some sort Memory capacity increased by adding modules, I/O by controllers •Add processors for processing! •For higher-throughput multiprogramming, or parallel programs I/O ctrlMem Mem Mem Interconnect Mem I/O ctrl Processor Processor Interconnect I/O dev ices