How to understand when to stop feature selection?

Feature selection should be stopped when further exclusion or addition of features no longer improves the model quality or starts to degrade it. Usually, quality metrics (e.g., accuracy, F1, ROC-AUC) on the validation set are used as a guide. If after a feature selection step:

• the metric does not improve,
• the improvement is negligible,
• or the model becomes too complex,

then it is a signal to stop.

It is also important to consider the balance between performance and model complexity: too many features can lead to overfitting and increased training/prediction time.

Example: when using Recursive Feature Elimination (RFE), you can stop when the validation metric reaches its maximum or starts to decrease.