What are the disadvantages of the linear regression analytical method?

The analytical method of linear regression based on direct calculation of coefficients through a formula (for example, using the pseudo-inverse matrix) has several disadvantages:

• High computational complexity with large data: calculating the inverse or pseudo-inverse matrix requires O(n³) operations, which is inefficient for large datasets.

• Sensitivity to multicollinearity: if the features are highly correlated, the matrix can be poorly conditioned, leading to unstable coefficient estimates.

• Lack of regularization: the analytical method does not include mechanisms to combat overfitting, unlike regularization methods (Ridge, Lasso).

• Problems with scalability and model updating: when new data arrives, the model must be recalculated entirely, unlike iterative methods (gradient descent).

Due to these limitations, numerical and iterative optimization methods are more commonly used for large and complex tasks.