This page collects material for my part of the course F21DP --- Parallel and Distributed Technology. Check this outline of the entire course and these learning outcomes for my part of it.
Material for Phil Trinder's part of it can be found here.
Purpose and Learning Objectives
Software is no longer sequential. Many programs are now concurrent, parallel, distributed or mobile. This module explores technologies that enable, the design issues that concern, and the hierarchy of architectures that deliver, distributed and parallel systems.
The parallel strand starts by reviewing basic concepts including brief overviews of the need for parallelism, parallel architectures and programming approaches. The main focus is then on parallel programming models, namely
- MPI with C,
- Glasgow parallel Haskell,
- OpenMP on multicores,
- GPU processing.
Following a review of C and an introduction to essential MPI concepts and constructs, parallel design patterns and their realisations as algorithmic skeletons are explored. The distributed strand also starts by reviewing basic concepts, including concurrency, threads and sockets. Existing technologies covered include fault tolerance, remote evaluation (RPC/RMI), and distributed object-based systems including object request brokers like CORBA. Several emerging technologies are covered, and compared to existing technologies. These include high-level coordination languages, mobility, Clouds and Grids.
Pre-requisites:
- F29OC: Operating Systems & Concurrency
- Solid C programming skills (there will be a quick C revision)
Course Structure
See this course outline.
- Week 1 (HWL): Introducing Parallelism (HWL/PWT); C Revision I; C Revision II;
- Week 2 (HWL): C Revision II; Communication Libraries (MPI) I; C Tutorial;
- Week 3 (HWL): Communication Libraries II; Parallel Performance Tuning (HWL); MPI Tutorial I;
- Week 4: (PWT); Haskell I Haskell II Glasgow parallel Haskell (GpH) I Haskell Lab;
- Week 5 (PWT): GpH II; GpH III; GpH Lab;
- Week 6: Programming Week and GpH Lab;
- Week 7 (HWL): Higher-order Parallelism (Skeletons) I; Higher-order Parallelism (Skeletons) II;
- Week 8 (PWT): OpenMP I; OpenMP II; OpenMP Lab;
- Week 9 (HWL):
Warehouse Computing Frameworks I;
Warehouse Computing Frameworks II;
Hadoop Lab;
Coursework due - Week 10 (PWT): Introducing Distribution & Sockets; Remote Evaluation (RPC/RMI); Concurrent & Distributed Haskell
- Week 11 (PWT): Cloud & Grid Computing (GridGpH) Distributed Object-based systems (CORBA) Mobile & Global Computation (mHaskell)
- Week 12 (PWT/HWL): Revision
Assessment:
- Assessed Coursework: 20%
- Exam: 80%
Learning Material
Slides (HWL):
- Week 1: Lecture 1: Overview, Lecture 2: C Revision (Part 1) Lecture 3: C Revision (Part 2)
- Week 2: Lecture 4: Communication Libraries: MPI (Part 1) Lecture 5: Communication Libraries: MPI (Part 2)
- Week 3: Lecture 6: Parallel Performance Tuning
- Week 4: slides for Haskell and GpH part of the course are here
- Week 5: slides for Haskell and GpH part of the course are here
- Week 6: Programming week (see coursework below)
- Week 7:
Lecture 8: Parallel Program Design
- Week 7: Guest Lecture: The unique challenges of producing compilers for GPUs, by Andrew Richards, CEO Codeplay Software Ltd, Edinburgh
- Week 8: slides for Haskell and GpH part of the course are here
- Week 9: Lecture 9: Algorithmic Skeletons Lecture 10: Data Center and Warehouse Computing
- Week 10:
slides are here
Guest Lecture: Parallel Programming in Eden - Week 12: Revision: Revision
Sample Sources:
- Sources from Lecture 2: C Revision (Part 1)
- Sources from Lecture 3: C Revision (Part 2)
- MPI-style Hello world
- Thinker/Guesser example using MPI
- Sequential matrix multiplication (2D arrays)
- Sequential matrix multiplication (array of vectors)
- Sequential matrix multiplication (using transpose)
- Input matrices: 5x5, 80%, max value 99
- Input matrices: 2000x2000, density 10%, max value 65536
- Parallel matrix multiplication (using transpose)
- Parallel matrix multiplication (send rows once)
- Parallel matrix multiplication (send result by rows)
- Parallel matrix multiplication (arbitrary number of processors)
- seq exponential function and a partial solution to the parallel exponential function
- Parallel quick-sort
- Parallel bubble-sort
- Parallel bubble-sort (with instrumentation)
- Input: 10000 element list
- Input: 4141382 element list (density: 80%, max value: 65535)
Lab exercises and tutorials are on a separate page.
Coursework
The assessed coursework for 2012/13 can be found here. The sequential source code for the Totient function is available in C and in Haskell. The assessed coursework will be handed out in Week 6. Submission deadline is Week 9.
Reading List
No single book or web resource covers the course, but recommended reading is associated with many lectures. Some useful links are on my web page on parallel computing.
Specific topics:
- C programming: The standard textbook on C progamming is the one by Kernighan and Ritchie, available from the library. The C book, by Mike Banahan, Declan Brady and Mark Doran, is a completely on-line version of another C introduction.
- Skeletons and Parallel Patterns:
"Structured Parallel Programming"
Michael McCool, James Reinders, Arch Robison.
ISBN10: 0124159931 ISBN13: 9780124159938
Publisher: Elsevier Science & Technology Morgan Kaufmann Publishers; Format: Paperback; Publication date: 31 Jul 2012
Other Resources
News :
Week 12 (28/29 March):- Revision Lecture: Thu 13:15 G45