Parallel Programming of High Performance Systems 2020

Topic outline

• 

  General

     

  This course, a collaboration of Erlangen Regional Computing Center (RRZE) and Leibniz Supercomputing Center (LRZ), is targeted at students and scientists with interest in programming modern HPC hardware, specifically the large scale parallel computing systems available in Jülich, Stuttgart and Munich, but also smaller clusters in Tier-2 centers and departments.
  
  The exercises given below are suggestions - you are free to play with any code you like. If you are a beginner you will probably also not be able to complete all tasks.
  

  Course date & time:

  March 9-13, 9:00 a.m. - 6:00 p.m.
  

  Course room:

  Friedrich-Alexander-Universität Erlangen-Nürnberg
  Department of Computer Science
  Martensstr. 3
  D-91058 Erlangen
  

  CIP pool 02.135 (2nd floor)
  

  Contact:

  Dr. Georg Hager (georg.hager@fau.de)
  Dr. Volker Weinberg (weinberg@lrz.de)

  
  

  Note: There is now a general discussion forum (see right below). You can post in this forum, but only if you are a registered attendee of the course (this is a requirement of the Moodle system - sorry). You can set up an account (no real name required) at: https://moodle.rrze.uni-erlangen.de/login/signup.php?

  After registering you have to enroll into this course. There should be a link in the box on the left once you're logged in.
  

  
  

  • General dicsussion forum
  • Hotel information for Erlangen Page
  • Schedule for days 1-5 File
  • Useful links and references Page
• 

  Day 1

  The lectures on day 1 cover general tools and practices that are useful for working in programming environments, and specifically when dealing with HPC systems.

  • Welcome and general remarks File
  • Introduction to HPC File
  • Programming Environment @ LRZ File
  • Using the IvyMUC Cluster at LRZ File
  • SSH, Modules and User Interfaces File
  • Exercise: Working with the LRZ IvyMUC system Page
  • Introduction to GNU Make and Cmake File
  • Managing software development with SVN and git File
  • Exercises File
• 

  Day 2

  Day 2 gives an introduction to the dominant HPC programming models (MPI and OpenMP) as well as to computer architecture on a general level.

  • Elementary MPI and OpenMP File
  • Intro: OpenMP & MPI on IvyMUC Page
  • Exercise: π by the Monte Carlo method Page
  • Introduction to computer architecture File
  • Affinity by OpenMP places File
  • Exercise: Measuring the FP divide throughput Page
• 

  Day 3

  On day 3 we cover the principles of serial code optimization, and in the afternoon we get an in-depth look at some features of MPI.

  • Principles of serial code optimization File
  • Exercise: General code optimization Page
  • Working with likwid-topology, likwid-pin Page
  • Advanced MPI Programming File
  • Exercise: Parallelization of a raytracer code Page
  • Spoiler alert: Solution for MPI File
• 

  Day 4

  The lectures of day 4 introduce parallelism on a theoretical level (parallel patterns, performance models) and the different variants and structures of parallel computer systems.

  We also cover some advanced correctness and performance issues with OpenMP. 

  • Parallel architectures in depth File
  • Parallel Algorithms File
  • Advanced OpenMP Programming File
  • Exercise: Parallel histogram computation Page
  • Exercise: Eliminating recursion Page
  • OpenMP performance and correctness File
  • How to pin threads and processes (HPC Wiki) URL
• 

  Day 5

  Day 5 is deals with Intel tools for performance analysis, tuning and correctness checking of parallel programs. It also introduces the Scalasca tuning framework. In addition, version control with SVN and GIT is covered.
  

  • Performance analysis tools: Intel® VTuneTM Amplifier and Advisor File
  • ITAC File
  • Scalasca File
  • Score-P File
  • Some exercises solutions as archive File
• 

  Evaluation

  Please fill out the course evaluation form before leaving the lecture room to help us to increase the quality of our trainings.

  https://survey.lrz.de/index.php/176237?lang=en
  
  

  This survey is anonymous.

  
  

  Thank you for your collaboration and thank you for joining our course!!