How was the inference of LLM (DeepSeek R1 33B) organized? How many GPUs, and how was parallelization handled?

Inference of large language models (LLM) like DeepSeek R1 33B typically requires distributed parallelization due to the volume of parameters.

For a model with 33 billion parameters, multiple GPUs (e.g., 8 or 16) are often used, connected via technologies like NVIDIA NCCL.

Main parallelization methods:

• Tensor Parallelism — splitting individual model layers across multiple GPUs.
• Pipeline Parallelism — dividing the model into sequential parts, each on a separate GPU.
• Data Parallelism — parallel processing of different data batches.

In DeepSeek R1 33B, a combination of tensor and pipeline parallelism was likely used to efficiently utilize memory and accelerate inference.

The exact number of GPUs and details depend on the infrastructure, but such models usually require a cluster of several high-performance GPUs with large video memory (e.g., A100 with 40-80GB VRAM).