What are contextual bandits?

Contextual bandits are a type of multi-armed bandit algorithms where, at each step, the algorithm receives some context (for example, information about the user or environment) and must choose an action (for example, showing an advertisement or recommending a product) to maximize the expected reward. Unlike classical bandits, where the choice is made without considering the context, here additional information is taken into account, allowing for more effective adaptation to different situations.

An example of use is personalization of recommendations: the system sees the user's profile (context) and chooses the best content option (action), receiving feedback (reward) in the form of a click or purchase. This helps conduct more effective A/B tests and experiment with different options, minimizing losses from unsuccessful recommendations.