What is the novelty effect?

The novelty effect is a phenomenon where users show increased interest or improved reactions to a new product, feature, or change simply because of its novelty, rather than due to actual improvement in quality or usefulness.

In the context of A/B testing, it is important to consider this, as an initial spike in activity or positive feedback may be caused by the novelty effect, not a sustainable improvement.

For example, if a new button is added to an app, users may actively use it in the first few days, but over time, interest will decline, and metrics will return to previous levels. Therefore, when analyzing test results, it is advisable to look at a long-term period to distinguish the novelty effect from the real impact of the changes.