Mis on peeking probleem A/B-testides?

Peeking problem in A/B tests is a situation where the experimenter repeatedly checks intermediate test results and makes decisions about stopping or modifying the test based on these preliminary data.

The problem is that frequent checks increase the likelihood of false positive results (Type I error), meaning one might incorrectly conclude that one version is better, when in fact it is just a coincidence.

To avoid peeking problems, methods for adjusting the significance level (e.g., corrections for multiple comparisons) are used, or fixed data analysis points are planned in advance.