An Introduction to Parallel Processing

About the course



Meeting Date Topic Lecture Material Complementary Material
23/10/22 Begining of Semester
24/10/22 No class
1 31/10/22 Introduction - Course Overview and requirements Administration
Lecutre:lecture01.pdf
For the most recent version please download the file by clicking on this link (dropbox)
Parallel Computing Explained In 3 Minutes

Guy Tel-Zur lecture: Introduction to parallel Programming - Message Passing Interface (MPI)
2 7/11/22 Introduction to Parallel Processing. Basic Message Passing -MPI The course virtual machine - instructions how to download and install. new
Message Passing Computing
local files: part 1 and part 2.

MPI
MPI Tutorials
3 14/11/22 Lab #1: Virtual (Building 33, Room 330)
Getting to know Linux
Hands-on practice withMPI,
Embarrassingly Parallel Computation

Parallel Profilers (Jumpshot, Scalasca)
Lecture

Example programs:
ring.c
cpi.c
cpi.c(local)
MPI reference card

Linux tutorials in PDF

scalasca
4 21/11/22 Performance Tools
Jumpshot,Scalascaoverview(another one), tutorial

IPM

Torque-Open-Source Resource Manager

MPI Collective routines
Embarrassingly Parallel Computations

Home assignment #1
Tools and demos

The educational cluster

Embarrassingly Parallel Computations
(a local and updated version)
5 28/11/22 Algorithms:


Partitioning and Divide and Conquer

- Demos: MPI on Windows OS
Lecture


Partitioning and Divide and Conquer Strategies

Synchronous Computations

Code: All to all example: c_ex07.c
Top500
More onIterative methods
6 5/12/22 Scalapack
some code examples

Home assignment #2
Lecture6

Lecture7

code:
oned.c
wave.c
dgemm_example.c
dgemm_demo.c
dgemm-mpi.c
pdgemv.c
mm_pblas.c
ScaLAPACK: a portable linear algebra library for distributed
memory computers - design issues and performance

scalapack
7 12/12/22 MPI-2 get and put

Parallel Programming in Shared Memory
- Introduction
- OpenMP
Lecture

MPI-2 get and put
cpi2.c

Programming with Shared Memory

A "Hands-On" Introduction to OpenMP, Mattson and Meadows (a shorter version)

OpenMP home

Wikipedia

OpenMP quick reference card

OpenMP in Visual C/C++

"Reap the Benefits of Multithreading without All the Work"


Introduction to OpenMP and
Advanced OpenMP
from: "Programming Techniques for Supercomputers 2011" by Prof. Gerhard Wellein, RRZE HPC Services

openmp
8 19/12/22 OpenMP - continued

Lecture

Hands-On Introduction to OpenMP, Mattson and Meadows, from SC08 (a shorter version)

Shared Memory(Intel)

Profiling OpenMP
An Overview of OpenMP - Ruud van der Pas - Sun Microsystems

Shared Memory Programming with OpenMP

9 26/12/22 Sorting Algorithms

OpenMP - continued

Hybrid MPI and OpenMP:

Home assignment #3
Sorting Algorithms

OpenMP Tasking Explained

Advanced OpenMP (version 4.5) and Vectorization

a demo with Roffline

Hybrid MPI+OpenMP

10 2/1/23 CilkPlus

Parallel Matlab -MatlabMPI

CilkPlus
Open cilk

Parallel Matlab

CilkPlus
Charles E. Leiserson receives 2014 ACM/IEEE Computer Society Ken Kennedy Award
11 9/1/23 Code modernization and optimization Intel's Xeon Phi

Lecture

Materials thanks to:Colfax Research

Introduction

N-body(slides)

youtube demo

Paper

An Introduction to OpenACC
Xeon Phi
12 16/1/23
An introduction to HTCondor

Home assignment #4

Introduction to Grid Computing

HTCondor and Pegasus

HTCondor Tutorial

HTCondorWeek 2013:
Basic Introduction to using HTCondor by Karen Miller, Center for High Throughput Computing

if time permits: Pegasus short tutorial
13 TBD More theoretical topics... Image Processing

Pipeline computations

Load Balancing and Termination Detection

Numerical Algorithms

Preparation to the exam

Course summary
20/1/23 End of Semester
23/1/23 exam - מועד א Location: TBD
20/2/23 exam - מועד ב Location: TBD