Parallel Haskell on MultiCores and Clusters: Milan'15
This page covers the course on "Parallel Haskell on MultiCores and Clusters" at the University of Milan, June/July 2015.
Course Overview
While most programming languages struggle to incorporate parallelism into their natural computation models, purely functional languages provide parallel execution for free: due to the absence of side effects, any two nonoverlapping subexpressions can be evaluated in parallel. This offers the programmer an opportunity to exploit the power of modern multicore architectures with only a fraction of the effort that is needed in traditional, stateful programming languages. In particular, the purely functional language Haskell provides several mechanisms to introduce parallelism, ranging from nonintrusive annotationstyle language extensions (such as GpH or Eden), best suited for multicores and smallscale clusters, to more explicit ways of encoding coordination (such Cloud Haskell or HdpH), best suited for largescale distribution. In this lecture series, I will give an introduction to parallel programming in several of these languages extensions, discuss correctness and performance aspects of the resulting parallel programs, and give practical examples for developing parallel code in these extensions. In the first section of the series, after a general introduction to sequential Haskell, I will give an overview of parallel programming in the Glasgow parallel Haskell (GpH) [1,2]. This model adds one primitive to Haskell, which enables the programmer to annotate an expression, marking it for (potential) parallel execution. This approach retains the purely functional nature of the language, is minimally intrusive by only requiring local changes to the program, and delegates most of the cumbersome coordination aspects of the parallel program to an adaptive runtimesystem. I will discuss evaluation strategies as an abstraction over this basic programming model that provides a clean separation between computation and coordination. Finally, I will discuss lazy, dataoriented strategies that make use of laziness in order to further improve the parallel performance in operating on large data structures [3].
In this way, an expert programmer can profit from advantages of functional languages in terms of abstraction and modularity to engineer complex orchestrations of largescale parallel code.
For each of the parallel programming models practical exercises will be given, demonstrating their use on a range of architectures, discussing common techniques for performance tuning, and demonstrating the use of available tools to in the parallel program development and tuning process.
Course Structure & Learning Material
Draft handouts of the complete slide sets are available here.
Draft handouts of the complete slide sets are available here (4up version).
Haskell Introduction
- Haskell Characteristics
- Values and Types
- Functions
- Type Classes and Overloading
- Example: Caesar Cipher
- Exercises on Haskell
Parallel Programming Overview
- Trends in Parallel Hardware
- Parallel Programming Models
- Parallel Programming Example
GpH --- Parallelism in a side-effect free language
- GpH --- Concepts and Primitives
- Evaluation Strategies
- Parallel Performance Tuning
- Further Reading \& Deeper Hacking
- Exercises on GPH
Case study: Parallel Matrix Multiplication
Advanced GpH Programming
- Advanced Strategies
- Further Reading & Deeper Hacking
- Courswork to be used as advanced GPH exercises
Dataflow Parallelism: The Par Monad
- Patterns
- Example
- Further Reading & Deeper Hacking
- Exercises on ParMonad
Skeletons
- Overview
- Implementing Skeletons
- Map/Reduce Skeleton
- More Implementations
- Eden: a skeleton programming language
- Further Reading & Deeper Hacking