The course is a modern introduction to Parallel Processing. The course covers theoretical parallel algorithms and practical tools such as: MPI, OpenMP, OpenCilk, and parallel profiling tools. Other topics include: Code optimization, and High-Throughput Computing -HTCondor.
The course is more practical than theoretical and it is oriented toward Scientific Computing.
Basic Linux and C skills are mandatory and expected to be self taught by lecture 3.
| Meeting | Topic | Lecture Material | Complementary Material |
| Begining of Semester | |||
| 1 | Introduction - Course Overview, administration, and requirements | Slides |
Parallel Computing Explained In 3 Minutes Guy Tel-Zur lecture: Introduction to parallel Programming - Message Passing Interface (MPI) |
| 2 | Introduction - Course Overview (continued) | Slides
|
Parallel Computing Explained In 3 Minutes Guy Tel-Zur lecture: Introduction to parallel Programming - Message Passing Interface (MPI) |
| 3 | Parallel Programming in Distributed Memory
- The Message Passing Interface - MPI |
Slides |
MPI MPI Tutorials |
| 4 | Hands-on & demos:
- Getting to know Linux - Hands-on practice with MPI - Parallel Profilers (Jumpshot, Scalasca) |
Slides
Example programs: ring.c cpi.c MPI-2 get and put: cpi2.c |
MPI reference card
Linux tutorials in PDF  |
| 5 |
Algorithms 1
- Embarrassingly Parallel Computations - Partitioning and Divide and Conquer Home assignment #1 |
Slides | |
| 6 |
Algorithms 2
- Synchronous Computations | Slides |
More onIterative methods |
7 |
Algorithms 3
- Numerical Linear Algebra - MPI Topologies Home assignment #2 |
Slides
Example codes: oned.c wave.c dgemm_example.c dgemm_demo.c dgemm-mpi.c pdgemv.c mm_pblas.c |
ScaLAPACK: a portable linear algebra library for distributed
memory computers - design issues and performance  |
| 8 |
Algorithms 4
- Sorting Algorithms, - Pipeline - Load balancing | Slides | |
| 9 |
Parallel Programming in Shared Memory
- Introduction - OpenMP |
Slides | OpenMP home
OpenMP quick reference card  |
| 10 | OpenMP - continued
- OpenMP Tasking - Open Cilk - Hybrid MPI and OpenMP Home assignment #3 |
Slides | Charles E. Leiserson receives 2014 ACM/IEEE Computer Society Ken Kennedy Award |
| 11 |
Optimizations
- Vectorization - The Roofline model - Aos and Soa - Loops optimizations |
Slides |
|
| 12 |
Distributed Computing - High Throughput Computing
- Introduction to Grid Computing - Introduction to HTCondor - Introduction to Pegasus Home assignment #4 |
Slides HTCondor Tutorial |
|
| 13 |
- Parallel Matlab
- Introduction to OpenACC - Image Processing - Course summary |
Slides
Preparation to the exam |