What is aleatoric and epistemic uncertainty?

Aleatoric and epistemic uncertainty are two types of uncertainty in machine learning and statistics.

• Aleatoric uncertainty (stochastic, internal uncertainty) — related to noise and randomness in data. It reflects variability that cannot be eliminated even with an infinite amount of data. For example, measurement errors or unpredictable factors.

• Epistemic uncertainty (epistemic, knowledge uncertainty) — related to the lack of knowledge of the model about the data or process. It arises due to limited data or an incomplete model and can be reduced by collecting additional data or improving the model.

Example:

• In a regression task, aleatoric uncertainty manifests as scatter of values around the true function.
• Epistemic uncertainty is high in regions of the feature space where there are few training examples.

In Bayesian models and Gaussian Processes, these uncertainties are modeled separately, allowing for better assessment of prediction reliability and decision-making under uncertainty.