An Introduction to Parallel Processing

About the course



Meeting Date Topic Lecture Material Complementary Material
20/3/22 Begining of Semester
21/3/22 No class - Purim Party
1 28/3/22 Introduction - Course Overview and requirements Administration
Lecutre:lecture01.pdf
For the most recent version please download the file by clicking on this link (dropbox)
Parallel Computing Explained In 3 Minutes

Guy Tel-Zur lecture: Introduction to parallel Programming - Message Passing Interface (MPI)
2 4/2/22 Introduction to Parallel Processing. Basic Message Passing -MPI The course virtual machine - instructions how to download and install. new
Message Passing Computing
local files: part 1 and part 2.

MPI
MPI Tutorials
3 11/4/22 Lab #1: Virtual (Building 33, Room 330)
Getting to know Linux
Hands-on practice withMPI,
Embarrassingly Parallel Computation

Parallel Profilers (Jumpshot, Scalasca)
Lecture

Example programs:
ring.c
cpi.c
cpi.c(local)
MPI reference card

Linux tutorials in PDF

scalasca
18/4/22 No class - Passover vacation
4 25/4/22 Performance Tools
Jumpshot,Scalascaoverview(another one), tutorial

IPM

Torque-Open-Source Resource Manager

MPI Collective routines
Embarrassingly Parallel Computations

Home assignment #1
Tools and demos

The educational cluster

Embarrassingly Parallel Computations
(a local and updated version)
5 2/5/22 Algorithms:


Partitioning and Divide and Conquer

- Demos: MPI on Windows OS
Lecture


Partitioning and Divide and Conquer Strategies

Synchronous Computations

Code: All to all example: c_ex07.c
Top500
More onIterative methods
6 9/5/22 Scalapack
some code examples

Home assignment #2
Lecture6

Lecture7

code:
oned.c
wave.c
dgemm_example.c
dgemm_demo.c
dgemm-mpi.c
pdgemv.c
mm_pblas.c
ScaLAPACK: a portable linear algebra library for distributed
memory computers - design issues and performance

scalapack
7 16/5/22 MPI-2 get and put

Parallel Programming in Shared Memory
- Introduction
- OpenMP
Lecture

MPI-2 get and put
cpi2.c

Programming with Shared Memory

A "Hands-On" Introduction to OpenMP, Mattson and Meadows (a shorter version)

OpenMP home

Wikipedia

OpenMP quick reference card

OpenMP in Visual C/C++

"Reap the Benefits of Multithreading without All the Work"


Introduction to OpenMP and
Advanced OpenMP
from: "Programming Techniques for Supercomputers 2011" by Prof. Gerhard Wellein, RRZE HPC Services

openmp
8 23/5/22 OpenMP - continued

Lecture

Hands-On Introduction to OpenMP, Mattson and Meadows, from SC08 (a shorter version)

Shared Memory(Intel)

Profiling OpenMP
An Overview of OpenMP - Ruud van der Pas - Sun Microsystems

Shared Memory Programming with OpenMP

9 30/5/22 Sorting Algorithms

OpenMP - continued

Hybrid MPI and OpenMP:

Home assignment #3
Sorting Algorithms

OpenMP Tasking Explained

Advanced OpenMP (version 4.5) and Vectorization

a demo with Roffline

Hybrid MPI+OpenMP

10 6/6/22 CilkPlus

Parallel Matlab -MatlabMPI

CilkPlus
Open cilk

Parallel Matlab

CilkPlus
Charles E. Leiserson receives 2014 ACM/IEEE Computer Society Ken Kennedy Award
11 13/6/22 Code modernization and optimization Intel's Xeon Phi

Lecture

Materials thanks to:Colfax Research

Introduction

N-body(slides)

youtube demo

Paper

An Introduction to OpenACC
Xeon Phi
12 20/6/22
An introduction to HTCondor

Home assignment #4

Introduction to Grid Computing

HTCondor and Pegasus

HTCondor Tutorial

HTCondorWeek 2013:
Basic Introduction to using HTCondor by Karen Miller, Center for High Throughput Computing

if time permits: Pegasus short tutorial
13 27/6/22 More theoretical topics... Image Processing

Pipeline computations

Load Balancing and Termination Detection

Numerical Algorithms

Preparation to the exam

Course summary
1/7/22 End of Semester
TBA exam - מועד א TBD
TBA exam - מועד ב TBD