Parallel programming

This two-week intensive course introduces the fundamental concepts, tools, and APIs for debugging, testing, and parallelizing scientific applications on modern HPC clusters. You will learn how to efficiently develop and run scientific codes on both shared-memory (OpenMP) and distributed-memory (MPI) architectures.

The first week combines lectures and guided hands-on sessions, covering topics from sequential code optimization to parallel programming with OpenMP and MPI. Using a simple numerical solver as a running example, you will apply standard tools such as gdb, valgrind, perf, and Intel Amplifier to analyze and improve performance, culminating in a hybrid parallel implementation.

The second week focuses on a personal project, where you will apply the techniques learned to your own (or a provided) scientific application. By the end of the course, you will be able to design, analyze, and parallelize real-world scientific programs effectively on HPC systems.

Prerequisites: Basic programming knowledge in C, C++, Fortran, or Python.
Note: GPUs and accelerators are not covered in this course.
Students must bring their own laptop.

Introduction, software development cycle, node optimization

Parallel programming theory, shared memory programming with OpenMP

Distributed memory programming with the Message Passing Interface API (MPI)

Distributed memory programming with advanced MPI

Hybrid programming : MPI + OpenMP

Projects presentation

Personal project weeks

Students have 2 weeks to finish a personal project on their code