Почему деревья сильнее переобучаются по сравнению с линейными моделями?

Trees (for example, decision trees) tend to overfit more than linear models due to their high flexibility and ability to fit training data precisely.

Reasons:

• High expressive power: a tree can create complex partitions of the feature space, allowing it to fit noise and outliers perfectly.
• Lack of regularization by default: without constraints on depth or minimum number of objects in a leaf, a tree can grow very deep, memorizing details of the training set.
• Local solutions: a tree builds rules that work well on training data but may generalize poorly to new data.

Unlike linear models, which are limited to linear dependencies and usually have fewer parameters, trees can model complex nonlinear dependencies, increasing the risk of overfitting.

To combat this, regularization methods (limiting depth, minimum number of objects in a leaf) and ensembles (Random Forest, Gradient Boosting) are used, which reduce overfitting by averaging.