What is model-based RL and what are its advantages over model-free?

Model-based Reinforcement Learning (RL) is an approach where the agent builds or uses a model of the environment (transition model) to predict the consequences of its actions. This allows for planning and decision-making using simulations within the model.

Unlike model-free RL, where the agent learns directly from interaction experience with the environment without an explicit model, model-based RL uses a model for more efficient learning.

Advantages of model-based RL:

• Higher sample efficiency — the agent can learn faster as it uses the model to generate additional data and plan.
• Planning capability — it can predict future states and choose optimal actions in advance.
• Flexibility — when the environment changes, it is enough to update the model rather than retrain the entire strategy.

However, model-based RL is more complex to implement because it requires accurate construction of the environment model, which is not always possible.

Example: in a game, the agent can use a physics model to simulate moves and choose the best strategy instead of trying all actions directly in the real game.