An Introduction to Parallel Processing

About the course



Meeting Date Topic Lecture Material Complementary Material
17/10/21 Begining of Semester
1 18/10/21 Introduction - Course Overview and requirements Administration
Lecutre:lecture01.pdf
For the most recent version please download the file by clicking on this link (dropbox)
Parallel Computing Explained In 3 Minutes

Guy Tel-Zur lecture: Introduction to parallel Programming - Message Passing Interface (MPI)
2 25/10/21 Introduction to Parallel Processing. Basic Message Passing -MPI The course virtual machine - instructions how to download and install. new
Message Passing Computing
local files: part 1 and part 2.

MPI
MPI Tutorials
3 1/11/21 Lab #1: Virtual (Building 33, Room 330)
Getting to know Linux
Hands-on practice withMPI,
Embarrassingly Parallel Computation

Parallel Profilers (Jumpshot, Scalasca)
Lecture

Example programs:
ring.c
cpi.c
cpi.c(local)
MPI reference card

Linux tutorials in PDF

scalasca
4 8/11/21 Performance Tools
Jumpshot,Scalascaoverview(another one), tutorial

IPM

Torque-Open-Source Resource Manager

MPI Collective routines
Embarrassingly Parallel Computations

Home assignment #1
Tools and demos

The educational cluster

Embarrassingly Parallel Computations

5 15/11/21 Algorithms:


Partitioning and Divide and Conquer

- Demos: MPI on Windows OS
Lecture


Partitioning and Divide and Conquer Strategies

Synchronous Computations

Code: All to all example: c_ex07.c
Top500
More onIterative methods
6 22/11/21 Scalapack
some code examples

Home assignment #2
Lecture6

Lecture7

code:
oned.c
wave.c
dgemm_example.c
dgemm_demo.c
dgemm-mpi.c
pdgemv.c
mm_pblas.c
ScaLAPACK: a portable linear algebra library for distributed
memory computers - design issues and performance

scalapack
7 29/11/21 MPI-2 get and put

Parallel Programming in Shared Memory
- Introduction
- OpenMP
Lecture

MPI-2 get and put
cpi2.c

Programming with Shared Memory

A "Hands-On" Introduction to OpenMP, Mattson and Meadows (a shorter version)

OpenMP home

Wikipedia

OpenMP quick reference card

OpenMP in Visual C/C++

"Reap the Benefits of Multithreading without All the Work"


Introduction to OpenMP and
Advanced OpenMP
from: "Programming Techniques for Supercomputers 2011" by Prof. Gerhard Wellein, RRZE HPC Services

openmp
8 6/12/21 OpenMP - continued

Lecture

Hands-On Introduction to OpenMP, Mattson and Meadows, from SC08 (a shorter version)

Shared Memory(Intel)

Profiling OpenMP
An Overview of OpenMP - Ruud van der Pas - Sun Microsystems

Shared Memory Programming with OpenMP

9 13/12/21 Sorting Algorithms

OpenMP - continued

Hybrid MPI and OpenMP:

Home assignment #3
Sorting Algorithms

OpenMP Tasking Explained


Advanced OpenMP (version 4.5) and Vectorization

Hybrid MPI+OpenMP

10 20/12/21 CilkPlus

Parallel Matlab -MatlabMPI

CilkPlus
Open cilk

Parallel Matlab

CilkPlus
Charles E. Leiserson receives 2014 ACM/IEEE Computer Society Ken Kennedy Award
11 27/12/21 Code modernization and optimization

Introduction to Grid Computing
Intel's Xeon Phi

Lecture

Materials thanks to:Colfax Research

Introduction

N-body(slides)

youtube demo

Paper

Introduction to Grid Computing
Xeon Phi
12 3/1/22
A very short introduction to HTCondor

Home assignment #4
HTCondor and Pegasus

HTCondor Tutorial

HTCondorWeek 2013:
Basic Introduction to using HTCondor by Karen Miller, Center for High Throughput Computing

if time permits: Pegasus short tutorial
13 10/1/22 More theoretical topics... Image Processing

Pipeline computations

Load Balancing and Termination Detection

Numerical Algorithms

Preparation to the exam

Course summary
14/1/22 End of Semester
TBA exam - מועד א TBD
TBA exam - מועד ב TBD