Tell about quantization of models: what it is, why it is needed, and what types exist.

Model quantization is the process of converting the weights and activations of a neural network from floating-point numbers (usually 32-bit floats) to lower-precision numbers, such as 8-bit integers. This is done to reduce the model size and speed up computations, especially on resource-constrained devices (mobile devices, embedded systems).

Why quantization is needed:

• Reduce the memory required to store the model.
• Accelerate inference through simpler integer operations.
• Decrease power consumption.

Main types of quantization:

1. Post-Training Quantization — applied after training the model, without additional training. A quick way to reduce model size, but may slightly decrease accuracy.

2. Quantization Aware Training (QAT) — during training, the model accounts for quantization, which helps maintain high accuracy.

3. Dynamic Quantization — weights are quantized beforehand, and activations are dynamically quantized during inference.

4. Fixed-point Quantization — conversion to fixed-point with a specified number of bits.

Example: When quantizing weights from float32 to int8, values are scaled and shifted to preserve the range and minimize information loss.

# Example of post-training quantization using PyTorch
import torch
model = ...  # trained model
quantized_model = torch.quantization.quantize_dynamic(
    model, {torch.nn.Linear}, dtype=torch.qint8
)