Introduction to MPI

Introduction to MPI
Yaman Dua,
Research Scholar,
Department of CSE, IITBHU

Content

Basics of MPI

MPI program structure

MPI functions

Sample Programs

Message Passing Interface

Message passing model for communication in parallel
programming.

A library specification - Not a language or compiler specification

Standard for distributed memory, message passing, parallel
computing

Why MPI?

Standardization – virtually all HPC platforms

Portability – same code runs on another platform

Performance – vendor implementations should exploit native
hardware features

Functionality – 115 routines

Availability – a variety of implementations available in various
languages

MPI programming

All MPI operations are performed with routine calls

Basic definitions in

mpi.h for C/C++

mpif.h for Fortran 77 and 90

To compile a single file filename.c -

mpicc -c filename.c

To link the output and make an executable, use

mpicc -o ouput_filename filename.c

Combining compilation and linking in a single command is a
convenient way to build simple programs

MPI programming

To execute MPI program -

mpirun -np X output_filename

Will run X copies of program in the current run time environment

np - specifies number of copies of program

Basic model of MPI

Communicators and Groups

Group

ordered set of processes

each process is associated with a unique integer rank

rank from 0 to (N-1) for N processes

an object in system memory accessed by handle

A special pre-defined group -

MPI_GROUP_EMPTY - a group with no members

The predefined constant MPI_GROUP_NULL is the value used for
invalid group handles.

Basic model of MPI

Communicator

Group of processes that may communicate with each other

MPI messages must specify a communicator

An object in memory

Handle to access the object

There is a default communicator (automatically defined):

MPI_COMM_WORLD - identify the group of all processes

Intra-Communicator – All processes from the same group

Inter-Communicator – Processes picked up from several groups

Properties of Communicator and Group
For a programmer, group and communicator are one

Allows to organize tasks, based upon function, into task groups

Enable Collective Communications operations across a subset of
related tasks

Allows safe communications

Dynamic – can be created and destroyed at run time

Process may be in more than one group/communicator – unique rank
in every group/communicator

Allows implementing user defined virtual topologies like grid topology,
graph topology

MPI program Structure
MPI include file
“ # include mpi.h”
Initialize MPI environment
MPI_Init
--------------------------
----------
MPI_Xxxxxx();
-----------
Terminate MPI environment
MPI_Finalize

MPI Functions

MPI_Init –

Must be called, by all MPI programs, only once and before any other MPI
functions are called

int MPI_Init(int *pargc, char ***pargv);

Passes command line arguments to all processes

MPI_Comm_size – size of group associated with the communicator

MPI_Comm_rank –

identify the rank of the calling process within the communicator - can be
called task ID

int MPI_Comm_rank(MPI_Comm comm, int *rank);

MPI Functions

Unique rank, between 0 and (p-1), for a process in each communicator it
belongs to

Used to identify work for the processor

MPI_Send – Send messages to processor

MPI_Recv – Receive messages from processor

MPI_Finalize – Terminates the MPI execution environment, last routine
to be called in any MPI program

More MPI Functions

int MPI_Init (&flag)

Check if MPI_Initialized has been called

int MPI_Wtime()

Returns elapsed wall clock time in seconds (double precision) on the
calling processor

int MPI_Wtick()

Returns the resolution in seconds (double precision) of MPI_Wtime()

Classification of MPI Functions

Environment Management

MPI_Init, MPI_Finalize

Point-to-Point Communication

MPI_Send, MPI_Recv

Collective Communication

MPI_Reduce, MPI_Bcast

MPI_Alltoall - rearranges n items of data such that the nth node gets the nth item of
data

Information on the Processes

MPI_Comm_rank, MPI_Get_processor_name

Hello world program
#include "mpi.h"
#include <stdio.h>
int main( int argc, char *argv[] )
{
int rank, size;
MPI_Init( &argc, &argv );
MPI_Comm_rank( MPI_COMM_WORLD, &rank );
MPI_Comm_size( MPI_COMM_WORLD, &size );
printf( "I am %d of %dn", rank, size );
MPI_Finalize();
return 0;
}
//MPI_COMM_WORLD- identifies all processes involved in a computation

Vector addition program/* The master process broadcasts the computed initial values to all the other processes.*/
MPI_Bcast ( array, N, MPI_DOUBLE, master, MPI_COMM_WORLD );
/* Each process adds up its entries.*/
sum = 0.0;
for ( i = 0; i < N; i++ )
{
sum = sum + array[i] * ( double ) my_id;
}
printf ( "n" );
printf ( "SUM - Process %d:n", my_id );
printf ( " My contribution to the sum is %fn", sum );
/* Each worker process sends its sum back to the master process.*/
if ( my_id != master )
{ MPI_Send ( &sum, 1, MPI_DOUBLE, master, 1, MPI_COMM_WORLD ); }
else {
sum_all = sum;
for ( i = 1; i < numprocs; i++ )
{
MPI_Recv ( &sum, 1, MPI_DOUBLE, MPI_ANY_SOURCE, 1,
MPI_COMM_WORLD, &status );
sum_all = sum_all + sum / MPI_COMM_WORLD;
}
}
if ( my_id == master )
{
printf ( "n");
printf ( "SUM - Master process:n");
printf ( " The total sum is %.16fn", sum_all );
}

Vector addition program(cont..)/* Terminate MPI.*/
MPI_Finalize ( );
/* Terminate.*/
if ( my_id == master )
{
printf ( "n");
printf ( "SUM - Master process:n");
printf ( " Normal end of execution.n");
printf ( "n" );
timestamp ( );
}
return 0;
# undef N
}
void timestamp ( )
{
# define TIME_SIZE 40
static char time_buffer[TIME_SIZE];
const struct tm *tm;
time_t now;
now = time ( NULL );
tm = localtime ( &now );
strftime ( time_buffer, TIME_SIZE, "%d %B %Y %I:%M:%S %p", tm );
printf ( "%sn", time_buffer );
return;
# undef TIME_SIZE
}

Matrix multiplication program#include <stdio.h>
#include <sys/time.h>
#include <stdlib.h>
#include "mpi.h"
#define N 4 /* number of rows and columns in matrix */
MPI_Status status;
double a[N][N],b[N][N],c[N][N];
int main(int argc, char **argv)
{
int numtasks,taskid,numworkers,source,dest,rows,offset,i,j,k;
struct timeval start, stop;
MPI_Init(&argc, &argv);
MPI_Comm_rank(MPI_COMM_WORLD, &taskid);
MPI_Comm_size(MPI_COMM_WORLD, &numtasks);
numworkers = numtasks-1;
/*---------------------------- master ----------------------------*/
if (taskid == 0) {
for (i=0; i<N; i++) {
for (j=0; j<N; j++) {
a[i][j]= 1.0;
b[i][j]= 2.0;
}
}
gettimeofday(&start, 0);
/* send matrix data to the worker tasks */
rows = N/numworkers;
offset = 0;
for (dest=1; dest<=numworkers; dest++)
{
MPI_Send(&offset, 1, MPI_INT, dest, 1, MPI_COMM_WORLD);
MPI_Send(&rows, 1, MPI_INT, dest, 1, MPI_COMM_WORLD);
MPI_Send(&a[offset][0], rows*N, MPI_DOUBLE,dest,1, MPI_COMM_WORLD);
MPI_Send(&b, N*N, MPI_DOUBLE, dest, 1, MPI_COMM_WORLD);
offset = offset + rows;
}

Matrix multiplication program(cont..)/* wait for results from all worker tasks */
for (i=1; i<=numworkers; i++)
{ source = i;
MPI_Recv(&offset, 1, MPI_INT, source, 2, MPI_COMM_WORLD, &status);
MPI_Recv(&rows, 1, MPI_INT, source, 2, MPI_COMM_WORLD, &status);
MPI_Recv(&c[offset][0], rows*N, MPI_DOUBLE, source, 2, MPI_COMM_WORLD, &status);
}
gettimeofday(&stop, 0);
printf("Here is the result matrix:n");
for (i=0; i<N; i++) {
for (j=0; j<N; j++)
printf("%6.2f ", c[i][j]);
printf ("n");
}
fprintf(stdout,"Time = %.6fnn",
(stop.tv_sec+stop.tv_usec*1e-6)-(start.tv_sec+start.tv_usec*1e-6)); }
/*---------------------------- worker----------------------------*/
if (taskid > 0) {
source = 0;
MPI_Recv(&offset, 1, MPI_INT, source, 1, MPI_COMM_WORLD, &status);
MPI_Recv(&rows, 1, MPI_INT, source, 1, MPI_COMM_WORLD, &status);
MPI_Recv(&a, rows*N, MPI_DOUBLE, source, 1, MPI_COMM_WORLD, &status);
MPI_Recv(&b, N*N, MPI_DOUBLE, source, 1, MPI_COMM_WORLD, &status);
/* Matrix multiplication */
for (k=0; k<N; k++)
for (i=0; i<rows; i++) {
c[i][k] = 0.0;
for (j=0; j<N; j++)
c[i][k] = c[i][k] + a[i][j] * b[j][k];
}
MPI_Send(&offset, 1, MPI_INT, 0, 2, MPI_COMM_WORLD);
MPI_Send(&rows, 1, MPI_INT, 0, 2, MPI_COMM_WORLD);
MPI_Send(&c, rows*N, MPI_DOUBLE, 0, 2, MPI_COMM_WORLD);
} MPI_Finalize(); }

Las Vegas randomization algorithm
Las Vegas algorithm -

Algorithm that runs for an unpredictable amount of time but always succeeds

It can be converted back into one that runs in bounded time by declaring that it fails
if it runs too long

QuickSort is an example of a Las Vegas algorithm.

Monte Carlo algorithm -

Fails with some probability, but we can’t tell when it fails

If the failure probability is significantly less than 1/2,it can be reduced by running it
many times and taking a majority of the answers

Ex- polynomial equality-testing, pie estimation

MPI monte carlo pie estimation
MPI_Init(&argc,&argv);
MPI_Comm_size(MPI_COMM_WORLD,&numprocs);
MPI_Comm_rank(MPI_COMM_WORLD,&myid);
if (argc <=1) {
fprintf(stderr,"Usage: monte_pi_mpi number_of_iterationsn");
MPI_Finalize();
exit(-1);}
sscanf(argv[1],"%d",&niter); /* 1st argument is the number of iterations*/
/* initialize random numbers */
stream_id = init_sprng(myid,numprocs,SEED,SPRNG_DEFAULT);
mycount=0;
for ( i=0; i<niter; i++) {
x = (double)sprng(stream_id);
y = (double)sprng(stream_id);
z = x*x+y*y;
if (z<=1) mycount++;
}

MPI monte carlo pie estimation(cont)
if (myid ==0) { /* if I am the master process gather results from others */
count = mycount;
for (proc=1; proc<numprocs; proc++) {
MPI_Recv(&mycount,1,MPI_REAL,proc,tag,MPI_COMM_WORLD,&status);
count +=mycount;
}
pi=(double)count/(niter*numprocs)*4;
printf("n # of trials= %d , estimate of pi is %g n",niter*numprocs,pi);
}
else { /* for all the slave processes send results to the master */
printf("Processor %d sending results= %d to master processn",myid,mycount
);
MPI_Send(&mycount,1,MPI_REAL,master,tag,MPI_COMM_WORLD);
}
MPI_Finalize(); /* let MPI finish up */
}

Las Vegas conversion of Pie Estimation
Algo:
Specify error factor for value of pie, specified_error = 0.0000001
sample_size = 100
while(error>=specified_error)
{
Execute monte carlo method with sample size = sample_size
Calculate error = estimated_pie - actual_pie
sample_size * = 10;
}

Introduction to MPI

More Related Content

What's hot (20)

Similar to Introduction to MPI (20)

Recently uploaded (20)

Introduction to MPI