Communication model of parallel platforms
Download as PPTX, PDF3 likes2,795 views
The document discusses two primary data exchange methods in parallel platforms: shared-address-space machines and message-passing platforms. Shared-address-space platforms allow multiple processors to access a common memory, distinguished by uniform (UMA) and non-uniform memory access (NUMA) architectures, each impacting algorithm design and programming complexity. Message-passing platforms, on the other hand, consist of independent processors with their own exclusive memory, and while requiring less hardware support, they are more challenging to emulate in a shared-address-space environment.
Communication model of parallel platforms
- 1. Communication Model of Parallel Platforms • There are two primary forms of data exchange between parallel tasks - accessing a shared data space and exchanging messages. • Platforms that provide a shared data space are called shared-address- space machines or multiprocessors. • Platforms that support messaging are also called message passing platforms or multicomputers.
- 2. Shared-Address-Space Platforms • Part (or all) of the memory is accessible to all processors. • Processors interact by modifying data objects stored in this shared- address-space. • If the time taken by a processor to access any memory word in the system global or local is identical, the platform is classified as a uniform memory access (UMA), else, a non-uniform memory access (NUMA) machine.
- 3. NUMA and UMA Shared-Address-Space Platforms Typical shared-address-space architectures: (a) Uniform-memory access shared-address-space computer; (b) Uniform-memory- access shared-address-space computer with caches and memories; (c) Non-uniform-memory-access shared-address-space computer with local memory only.
- 4. NUMA and UMA Shared-Address-Space Platforms • The distinction between NUMA and UMA platforms is important from the point of view of algorithm design. NUMA machines require locality from underlying algorithms for performance. • Programming these platforms is easier since reads and writes are implicitly visible to other processors. • However, read-write data to shared data must be coordinated (this will be discussed in greater detail when we talk about threads programming). • Caches in such machines require coordinated access to multiple copies. This leads to the cache coherence problem. • A weaker model of these machines provides an address map, but not coordinated access. These models are called non cache coherent shared address space machines.
- 5. Shared-Address-Space vs. Shared Memory Machines • It is important to note the difference between the terms shared address space and shared memory. • We refer to the former as a programming abstraction and to the latter as a physical machine attribute. • It is possible to provide a shared address space using a physically distributed memory.
- 6. Message-Passing Platforms • These platforms comprise of a set of processors and their own (exclusive) memory. • Instances of such a view come naturally from clustered workstations and non-shared-address-space multicomputers. • These platforms are programmed using (variants of) send and receive primitives. • Libraries such as MPI and PVM provide such primitives.
- 7. Message Passing vs. Shared Address Space Platforms • Message passing requires little hardware support, other than a network. • Shared address space platforms can easily emulate message passing. The reverse is more difficult to do (in an efficient manner).