This workshop provides an introduction to using multiple GPUs for high-performance computing. Participants will explore essential techniques and tools, including CUDA with C++ and Python, PyTorch, and MPI, to optimize performance and handle complex computations.

CUDA with C++ and Python will be used to demonstrate the basics of GPU programming, memory management, and multi-GPU computations. PyTorch will be introduced for training deep learning models on multiple GPUs, covering both distributed and model parallelism. MPI will be used to implement multi-GPU scenarios, providing practical examples and exercises.

By the end of this workshop, participants will have the practical knowledge and skills to effectively utilize multiple GPUs, overcoming the limitations of single GPU setups. Additionally, participants will learn how to install and configure the tools discussed in the workshop.

Instructor: Shao-Ching Huang

Register Now