What is cross-validation and what is it used for?

Cross-validation is a method for assessing the quality of a machine learning model, which helps to verify how well the model will perform on new, unseen data. The main idea is to split the original dataset into several parts (folds), for example, 5 or 10. Then, the model is trained on all parts except one, and tested on the remaining part. This process is repeated until each part has served as the test set.

This allows for a more reliable estimate of the model's performance, reduces overfitting, and helps select the best parameters. For example, in 5-fold cross-validation, the data is divided into 5 parts, and the model is trained 5 times, each time on 4 parts, testing on 1.

Here's an example in Python using sklearn:

from sklearn.model_selection import cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.datasets import load_iris

X, y = load_iris(return_X_y=True)
model = RandomForestClassifier()
scores = cross_val_score(model, X, y, cv=5)
print(f"Average accuracy: {scores.mean():.2f}")