What is bagging and why does it reduce the variance of a model?

Bagging (Bootstrap Aggregating) is an ensemble method where multiple training samples are created using bootstrap sampling (random sampling with replacement from the original data). For each such sample, a separate model is trained, and the final prediction is obtained by averaging (for regression) or voting (for classification) of all models' results.

Why bagging reduces variance:

• Each model is trained on slightly different data, so the errors of the models are not fully correlated.
• Averaging the results reduces the impact of random errors of individual models.
• As a result, the ensemble becomes more resistant to overfitting and noise in the data.

Thus, bagging reduces the variance of the model, increasing its stability and generalization ability.