1. Basic Concepts

Types of Communication:

• Point-to-Point Communication: Send-Receive pair, involving two processes
• Collective Communication: such as Broadcast for data movement and Reduce for collective computation, involving a collection of processes

Communicators

• The combination of group and context (a mechanism to protect safe communication, we will come to this later)
• There can be multiple communicators in an application, and a process may be a member of more than one communicator.
• The rank is only relevant to one communicator, so if one process is in more than one communicator it may have a different rank in each .
• All massage passing are carried out within a specific communicator (intra-communicator) or two specific communicators (inter-communicator).

• When a MPI application is initialized by calling MPI_Init function, the library will set up an initial default communicator, MPI_COMM_WORLD, which all processes are a number of.
Say on falcon6, you run a MPI program as
% mpirun -np 4 mpi_prog1
falcon6 will look at a file:
 
/usr/local/mpi/util/machines/machines.alpha
to obtain the other 3 hosts.

Such a  machines.alpha file is shown below:

% more machines.alpha
falcon2.cz3.nus.edu.sg
falcon3.cz3.nus.edu.sg
falcon4.cz3.nus.edu.sg
............
falcon20.cz3.nus.edu.sg

falcon6 will use itself, falcon2, falcon3, and falcon4 to set up MPI_COMM_WORLD with a group of 4 processes.
 

Thus if all of you run a MPI job like this, it will end up with overloading in falcon2, falcon3, and falcon4. In order to avoid this, you can copy:
/usr/local/mpi/util/machines/machines.alpha
to your own directory, and randomize the machine order such as:
 
falcon10.cz3.nus.edu.sg
falcon7.cz3.nus.edu.sg
falcon4.cz3.nus.edu.sg
............
falcon5.cz3.nus.edu.sg
 
Then run job as
mpirun -machinefile machines.alpha -np 4 mpi_prog1
 
 

\% more machines.alpha \\
 

• Like other communicators, MPI_COMM_WORLD is represented by an opaque communication object, which is invisible to user and can only be accessed by specific accessors.

MPI_Comm_size(MPI_Comm MPI_COMM_WORLD,int *size)
MPI_Comm_rank(MPI_Comm MPI_COMM_WORLD,int *rank)
MPI_Comm_group(MPI_Comm MPI_COMM_WORLD, MPI_Group *group)

The last accessor gets from MPI_COMM_WORLD a pointer to an opaque group object, which in turn can only be accessed by its accessors, such as

MPI_Group_size(MPI_Group group,int *size)

• A set of data in C is given by data_type, length, and starting address as

int A[30];

• A set of message data is also specified by these three components.
• Starting address is given by an address operator &.
• Length is in terms of number of elements, not number of bytes, because the former is machine independent.
• The basic data_types for MPI correspond to the basic datatypes of C language with prefix MPI, such as MPI_CHAR, MPI_SHORT, MPI_INT.
• Derived datatypes: more general, mixed, and noncontiguous data constructed in an application.

• Hello World (C code, Fortran code).
• Sum (C code, Fortran code).
• Product (C code, Fortran code).

Homework 2.