Initialization and Completion The call to MPI_FINALIZE should occur on the same thread that initialized MPI. We call this thread the main thread. The call should occur only after all the process threads have completed their MPI calls, and have no pending communications or I/O operations.
 
 
 
 Rationale.  
 
This constraint simplifies implementation.  
 ( End of rationale.) 
 
Multiple threads completing the same request. A program where two threads block, waiting on the same request, is erroneous. Similarly, the same request cannot appear in the array of requests of two concurrent MPI_WAIT{ANY|SOME|ALL} calls. In MPI, a request can only be completed once. Any combination of wait or test which violates this rule is erroneous.
 
 
 
 Rationale.  
 
This is consistent with the view that a multithreaded execution  
corresponds to an interleaving of the  MPI calls.  
In a single threaded implementation, once a wait is  
posted on a request  
the request handle will be nullified before it is possible to  
post a second wait on the same handle.  
With threads, an  MPI_WAIT{ANY|SOME|ALL}  
may be blocked without having nullified its request(s) so it  
becomes the user's responsibility to avoid using the same request  
in an  MPI_WAIT on another thread.  
This constraint also simplifies  
implementation, as only one thread will be blocked on any  
communication or I/O event.  
 ( End of rationale.) 
 
Probe A receive call that uses source and tag values returned by a preceding call to MPI_PROBE or MPI_IPROBE will receive the message matched by the probe call only if there was no other matching receive after the probe and before that receive. In a multithreaded environment, it is up to the user to enforce this condition using suitable mutual exclusion logic. This can be enforced by making sure that each communicator is used by only one thread on each process.
Collective calls Matching of collective calls on a communicator, window, or file handle is done according to the order in which the calls are issued at each process. If concurrent threads issue such calls on the same communicator, window or file handle, it is up to the user to make sure the calls are correctly ordered, using interthread synchronization.
Exception handlers An exception handler does not necessarily execute in the context of the thread that made the exception-raising MPI call; the exception handler may be executed by a thread that is distinct from the thread that will return the error code.
 
 
 
 Rationale.  
 
The  MPI implementation may be multithreaded, so that part of the  
communication protocol may execute on a thread that is distinct from  
the thread that made the  MPI call.  
The design allows the exception handler to be executed on the  
thread  
where the exception occurred.  
 ( End of rationale.) 
 
Interaction with signals and cancellations The outcome is undefined if a thread that executes an MPI call is cancelled (by another thread), or if a thread catches a signal while executing an MPI call. However, a thread of an MPI process may terminate, and may catch signals or be cancelled by another thread when not executing MPI calls.
 
 
 
 Rationale.  
 
Few C library functions are signal safe, and many have cancellation  
points --- points where the thread executing them may be cancelled.  The  
above restriction simplifies implementation (no need for the  MPI  
library to be ``async-cancel-safe'' or ``async-signal-safe.''    
 ( End of rationale.) 
 
 
 
 Advice to users.  
 
Users can catch signals in separate, non- MPI threads (e.g., by  
masking signals on  MPI calling threads, and unmasking them in one or  
more non- MPI threads).    
  
A good programming practice is to have a distinct thread blocked  
in a call to  sigwait for each user expected signal that may occur.  
Users must not catch signals used by the  MPI implementation; as   
each  MPI implementation is required to document the signals used   
internally, users can avoid these signals.  
  
 ( End of advice to users.) 
 
 
 
 Advice  
        to implementors.  
 
The  MPI library should not invoke  library calls that are  
not thread safe, if multiple threads execute.  
 ( End of advice to implementors.)