Можно ли считать новую модель с F1-score 0.81 лучше модели с 0.8 и сразу запускать в продакшен?

It is not possible to automatically consider a model with an F1-score of 0.81 better than a model with 0.8 and immediately deploy it to production. The difference of 0.01 may be statistically insignificant and does not guarantee improvement in real-world conditions.

Before deployment, it is recommended to:

• Conduct a statistical significance test (e.g., bootstrapping) to assess the reliability of the improvement.
• Evaluate the model on an independent test dataset.
• Check the stability and performance of the model under conditions close to production.
• Assess the impact on business metrics and potential risks.

Only after a comprehensive evaluation can a decision be made regarding the deployment of the new model.