What is MIG (Multi-Instance GPU) on A100/H100?

MIG (Multi-Instance GPU) — is a technology introduced in NVIDIA A100 and H100 GPUs that allows dividing a single physical GPU into multiple isolated logical GPUs (instances). Each of these instances has dedicated computing resources (CUDA cores, memory, etc.) and operates as a separate GPU.

This is useful for:

• Running multiple tasks in parallel with guaranteed resource separation.
• Increasing GPU utilization efficiency in environments with different workloads.
• Improving isolation and security when sharing a GPU.

For example, on a single A100, you can create multiple MIG instances to run different machine learning models simultaneously without interfering with each other.

In CUDA programming, this allows assigning tasks to specific MIG instances, managing resources more flexibly.