How to handle missing values in a dataset?

Missing values in a dataset can negatively impact the quality of machine learning models, so they need to be handled carefully. Main approaches:

• Removing rows or columns with missing values — a simple method, but it can lead to data loss.
• Imputation of values:
  • Using the mean, median, or mode for numerical features.
  • Using the most frequent value for categorical features.
  • Employing more complex methods, such as KNN imputation or models for predicting missing values.
• Using special values or indicators for missingness — adding a boolean feature indicating missing data.
• Models that are robust to missing data — some algorithms can handle missing values directly.

Example of filling missing values with the mean in Python with pandas:

import pandas as pd

df = pd.DataFrame({'age': [25, None, 30, None, 22]})
mean_age = df['age'].mean()
df['age'].fillna(mean_age, inplace=True)

The choice of method depends on the amount of missing data, the nature of the data, and the task.