F21DP FAQ
This page discusses frequently asked questions in the context of the course on Distributed and Parallel Technologies.
How do I get started with MPI?
Use one of the hello-world examples, and go through the steps listed below. See the rest of the page for explanations on these steps.
> wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/hello2.c > mpd & > mpicc -Wall -O -o hello2 hello2.c > mpirun -n 4 hello2
You should see something like this as output:
Hello, I am 0 of 4 (hostname is bwlf01) Hello, I am 1 of 4 (hostname is bwlf01) Hello, I am 3 of 4 (hostname is bwlf01) Hello, I am 2 of 4 (hostname is bwlf01)
Read on to learn how to configure MPI to use several hosts. Note that each host is an 8-core machine, so you can run parallel programs with up to 8 tasks and obtain speedups in this local configuration already.
How do I configure MPI?
Check the slides of Lecture 4. In short
- Use the machines in our Beowulf cluster: bwlf01-bwlf31
- You must have /usr/bin in your PATH
- Start the MPI demon in the background mpd & The first time you do this you might be asked to create a file .mpd.conf (if it doesn't exist). Follow the instructions that are printed together with the error message.
- To test the setup run mpdtrace which should print the name of all machines included in your MPI configuration (by default, just the current machine).
- If this succeeds, you are ready to run MPI-compiled programs.
How do I compile C + MPI programs?
Check the slides of Lecture 4. In short, if your program is in file.c, type mpicc -Wall -O -o file file.c.
How do I run C + MPI programs?
Check the slides of Lecture 4. In short, if your executable is file and you have configured MPI (see above), type mpirun -np 4 file to run the program on 4 machines.
How do I launch an MPI configuration with several hosts?
Assuming that the file ${HOME}/etc/mpi4 contains a list of all hostnames, just type
mpdboot -n 4 -f ${HOME}/etc/mpi4
and test it with
> mpdtrace
How do I reset MPI?
To shut down an existing MPI configuration, just type
> mpdallexit
On just one local machine, you can also simply kill any running mpd process (well, if it's your process).
How do I get rid of the password prompt when launching my MPI configuration?
You have to set up a password-less ssh login to do so. There are many good web pages discussing this, for example here or here. In short, generate a password-less ssh key using
> ssh-keygenand then do is:
> ssh-copy-id `whoami`@bwlf01.macs.hw.ac.uk
How do I run a shell command on all cluster nodes?
Set the environment variable BEONODES and then call /u1/staff/hwloidl/bin/beotools/brsh, eg:
bwlf01[102](3.2)> export BEONODES="bwlf01 bwlf02 bwlf03 bwlf05" bwlf01[103](3.2)> echo $BEONODES bwlf01 bwlf02 bwlf03 bwlf05 bwlf01[104](3.2)> which brsh /u1/staff/hwloidl/bin/beotools/brsh bwlf01[105](3.2)> brsh w | fgrep "average" bwlf01 21:05:37 up 13 days, 16:18, 5 users, load average: 0.03, 0.01, 0.00 bwlf02 21:05:37 up 13 days, 16:27, 0 users, load average: 1.43, 2.16, 2.32 bwlf03 21:05:39 up 13 days, 15:16, 0 users, load average: 1.09, 1.96, 2.15 bwlf05 21:05:41 up 10 days, 17:57, 0 users, load average: 0.68, 1.75, 2.29
How do I check whether all cluster nodes are up?
Use bping to contact all machines listed in the BEONODES variable.
How do I get a list of all cluster nodes that are currently up?
echo -n > bwlfLive
for host in `cat ~hwloidl/etc/mpi32`; do
ping -q -c 1 -w 1 $host && echo $host >> bwlfLive
done
How do I get a list of cluster nodes with a load of, say, less than 1?
for i in `cat ~hwloidl/etc/mpi32` ; do if [ `ssh $i w | sed -ne '/average/!d;s/^.*average: \([0-9]*\).*$/\1/;p' ` -lt 1 ] ; then echo $i ; fi ; done
How do I generate speedup graphs and such?
A simple tool for producing speedup graphs, histograms and such is Gnuplot (see also this quick reference card). It is a command-line based tool. Typically, you feed it a file, that contains x and y values on each line, eg. It then produces a graph, either interactively in a window or in a file, which you can then import in your report.
You can automate the generation of this data file, using Unix command such as sed to extract the relevant data from a log of several executions. For example, if you run tests on 2 to 8 processors like this
for ((i=2; i<9; i++)) ; do echo "mpirun -np $i matrix8 2MAT_2000_10_65536" >>LOG ; mpirun -np $i matrix8 2MAT_2000_10_65536 >> LOG ; doneyou can use the following commands to extract the relevant data and show a graph of runtimes:
cat LOG | sed -e '/secs/a\X' | sed -e 's/^.*np \([0-9]*\).*$/\1/;s/^.* \([.0-9]*\) secs.*$/\1/' | sed ':a;N;$!ba;s/\n/ /g' | sed 's/X/\n/g' >rt.dat echo "set term x11; plot \"rt.dat\" with lines; pause 10 " | gnuplot
This will show a graph of runtimes over varying numbers of processors in a new window. More commonly, you put all gnuplot commands in a file (eg. speedups.gp) and feed it to gnuplot
gnuplot speedups.gp
This will generate a file speedups.pdf, which you can include in your report discussing the parallelisation.
How can I profile OpenMP programs?
The necessary steps are discussed in more detail on the OpenMP Lab pages.
How can I profile OpenMP programs?
A slightly dated, but detailed, OpenMP profiler is OmpP. It can be downloaded here and documentation is here. In short, to profile an OpenMP program in file c_simple.c, do
export PATH=/u1/staff/dsg/OPT/x86_64-unknown-linux/bin:$PATH kinst-ompp gcc -fopenmp -o c c_simple.c ./c less c.4-1.ompp.txt
How do I get started with Haskell?
From the command line, on any of the Linux machines, type
ghcito launch the Haskell interpreter. Then, follow the instructions on the slides and do the Haskell exercises. A lot of useful information is in the early chapters of the Real World Haskell book.
How do I profile sequential Haskell programs?
Detailed information is available here. In short, to download, compile and profile two versions of the Taylor series example (power.hs, power_circ.hs) do
# download the examples wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/power.hs wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/power_circ.hs # compile the first program; note the -prof and -auto-all options ghc -cpp -O2 -rtsopts -prof -auto-all -o p_hs power.hs # run the program ./p_hs 2000 10 +RTS -pT -hC -sstderr # this file contains a summary and split up of execution time less p_hs.prof # display the heap consumption profile hp2ps p_hs.hp ps2pdf p_hs.ps evince p_hs.ps # compile the second program; note the -prof and -auto-all options ghc -cpp -O2 -rtsopts -prof -auto-all -o p_circ power_circ.hs # run the program ./p_circ 2000 10 +RTS -pT -hC -sstderr # compare the summary of execution time with the previous one less p_circ.prof # display the heap consumption profile hp2ps p_circ.hp ps2pdf p_circ.ps evince p_circ.pdf
This has been tested with this ghc version:
which ghc /u1/staff/dsg/OPT/x86_64-unknown-linux/bin/ghc
How do I compile and run a (parallel) Haskell program?
The necessary steps are discussed in more detail on the GpH Lab pages.
In short, to compile and run parfib the following sequence should work (either on lxpara3 or any of the bwlf machines):
wget http://www.macs.hw.ac.uk/~trinder/ParDistr/Examples/parfib.hs wget http://hackage.haskell.org/packages/archive/deepseq/1.3.0.1/deepseq-1.3.0.1.tar.gz tar xfz deepseq-1.3.0.1.tar.gz wget http://hackage.haskell.org/packages/archive/parallel/3.2.0.3/parallel-3.2.0.3.tar.gz tar xfz parallel-3.2.0.3.tar.gz wget http://hackage.haskell.org/packages/archive/haskell98/2.0.0.2/haskell98-2.0.0.2.tar.gz tar xfz haskell98-2.0.0.2.tar.gz ~pm175/sw/ghc/7.4.1/x86_64/bin/ghc-7.4.1 -cpp -ideepseq-1.3.0.1 -iparallel-3.2.0.3 -ihaskell98-2.0.0.2 -XBangPatterns -threaded -rtsopts -o parfib parfib.hs ./parfib 37 +RTS -N6 -sstderr
Is there an easier way of compiling Haskell programs?
GHC uses its own package manager for libraries, called cabal, and for larger projects this is the recommended way of using libraries. This is an alternative to downloading the .tar.gz bundles above. Detailed documentation for Cabal is available here.
To tell GHC that it should use the latest parallel package, available from the online database of Haskell packages, Hackage, do the following:
cabal install parallelThis should download and install the libraries in local directory (usually somewhere under ~/.cabal). Now, Control.Parallel.Strategies etc are in scope and you should be able to compile a parallel program like this:
ghc -cpp -O2 -threaded -rtsopts -o p parfib.hsThis has been tested with GHC 7.6.1:
> which ghc /u1/staff/dsg/OPT/x86_64-unknown-linux/bin/ghc
How do I use threadscope for visualisation?
/u1/staff/dsg/OPT/x86_64-unknown-linux/bin/threadscope
How do I compile and run UPC code?
You need to login to lxpara3 to get full UPC support. First, adjust your environment to pick up the installed upcc compiler:
> export PATH=/u1/staff/dsg/OPT/local/berkeley_upc/bin:$PATHThen, you can compile a UPC program in file vecadd4.c like this:
> upcc --pthreads -o v4 vecadd4.cand run like this
> export UPC_PTHREADS_PER_PROC=1; upcrun -n=8 v4 555555See the documentation on statically or dynamically fixing the number of processors etc. To launch a sequence of measurements, do
> for ((n=1; n<9; n++)) ; do upcrun -n=$n v4 555555 ; done
How do I compile and run code under the Hadoop system for MapReduce?
See the notes in the Hadoop Lab for details. In short, the sequence of steps is this:
To set up the Hadoop infrastructure do the following:
export HADOOP_HOME=/scratch/hadoop-0.23.0
export JAVA_HOME=/usr/java/default
export PATH=${HADOOP_HOME}/bin:${JAVA_HOME}/bin:${PATH}
To compile a simple example, download the WordCount.java example, put into a fresh directory and compile it:
wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/WordCount.java
mkdir wordcount_classes
javac -classpath "${HADOOP_HOME}/lib/*:${HADOOP_HOME}/share/hadoop/common/*:${HADOOP_HOME}/share/hadoop/common/lib/*" -d wordcount_classes WordCount.java
jar -cvf wordcount.jar -C wordcount_classes .
You need to provide input like this:
mkdir input wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/WaD.txt mv WaD.txt input/file01 wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/TMM.txt mv TMM.txt input/file02
Now, you can run a single-node Hadoop application like this:
java -cp wordcount.jar:${HADOOP_HOME}/*:${HADOOP_HOME}/lib/*:${HADOOP_HOME}/share/hadoop/common/*:${HADOOP_HOME}/share/hadoop/common/lib/* org.myorg.WordCount input output
You can view the result, i.e. the number of occurrences for each word in the input files, like this:
less output/part-00000
Make sure to delete, or rename, the output directory before you re-run the same application.
How do I compile and run Eden programs?
Full instructions can be found here: http://www.mathematik.uni-marburg.de/~eden/?content=build_progs&navi=build
Preferably use the compiler in the main DSG directory and download the sample code from the Eden tutorial section:
export PATH=/u1/staff/dsg/OPT/x86_64-unknown-linux/bin:$PATH wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/ParMap.hs wget http://www.macs.hw.ac.uk/~hwloidl/Courses/F21DP/srcs/mandel.hsThen compile the mandelbrot example like this:
ghc-7.6.2-eden -parcp -eventlog -O2 -rtsopts --make mandel.hsand run it like this
./mandel 0 200 1 -out +RTS -qp4 > out.ppmThis generates a mandelbrot picture in the file out.ppm which you can view like this:
display out.ppm
Follow the instructions at the end of the Eden slide-set and do some of the exercises.
To run on the cluster (i.e. more than one node) e.g. bwlf01-04 do the following:
- list the nodes in a file (e.g. mpihosts):
bwlf01 bwlf02 bwlf03 bwlf04
- add current directory to path:
PATH=`pwd`:$PATH
- start mpd (mpi process) with 4 nodes (i.e 8x4=32 processors in total):
mpdboot -n 4 -f mpihosts
Note: you might need to run `mpdallexit` first to exit existing running mpd instances. - run program as in step 4, specifying -N up to 32 processors.
You can find many more examples, and an in-depth Eden tutorial on this Eden CEFP tutorial page.
How do I use EdenTV for visualisation?
Compile Eden programs, enabling visualisation by tracing the execution like this:
ghc-7.6.2-eden -parcp -eventlog -O2 -rtsopts --eventlog --make mandel.hs
Run Eden programs, generating a trace file ending with .parevents like this:
./mandel 0 100 1 -out +RTS -qp4 -l > out.ppm
Visualise the execution, by calling the EdenTV trace viewer, and open a trace file generated from
/u1/staff/dsg/OPT/x86_64-unknown-linux/bin/edentv