Sobes.tech
Middle — Senior

Why are shallower trees usually used in gradient boosting than in random forests?

sobes.tech AI

Answer from AI

Gradient boosting typically uses shallower trees for several reasons:

  1. Avoiding overfitting. Gradient boosting builds models sequentially, with each subsequent model correcting the errors of the previous ones. Deep trees can fit the residuals too closely, leading to overfitting.

  2. Stability and generalization. Small trees (usually depth 3-6) create simpler models that better generalize the data.

  3. Training efficiency. Smaller trees train faster and are easier to interpret.

Unlike random forests, where each tree is trained independently and can be deeper for maximum split of the space, boosting relies on sequential error correction, so shallow trees are preferred.