Introduction to Collective Operations in MPI

Collective operations are called by all processes in a communicator.
MPI_BCAST distributes data from one process (the root) to all others in a communicator.
MPI_REDUCE combines data from all processes in communicator and returns it to one process.
In many numerical algorithms, SEND/RECEIVE can be replaced by BCAST/REDUCE, improving both simplicity and efficiency.

Collective operations are called by all processes in a communicator.