What is an experience replay buffer and why is it needed?

Experience replay buffer is a data structure used in reinforcement learning that stores past agent transitions (state, action, reward, next state).

Why needed:

• Allows reusing experience, increasing learning efficiency.
• Breaks correlation between sequential observations, making training more stable.
• Helps avoid overfitting on recent data, ensuring diversity of examples.

Example: an agent plays a game, each action and result are stored in the buffer. During training, random mini-batches from the buffer are used to update the model, improving generalization.