What is MDP (Markov Decision Process) and what are its components?

MDP (Markov Decision Process) is a mathematical model for describing decision-making under uncertainty, widely used in reinforcement learning tasks.

Main components of MDP:

• S (States) — a set of states in which the agent can be.
• A (Actions) — a set of actions the agent can perform.
• P (Transition Probability) — a function of the probabilities of transitioning from one state to another when performing a certain action, usually denoted as P(s'|s,a).
• R (Reward Function) — a reward function that defines the numerical reward for transitioning from state s to s' when taking action a.
• γ (Discount Factor) — a discount coefficient that determines the importance of future rewards compared to current ones.

MDP formalizes the decision-making process where the agent chooses actions to maximize the total expected reward.