What is a gradient from a mathematical point of view?

A gradient is a vector consisting of the partial derivatives of a function with respect to all its variables. Mathematically, if there is a multivariable function (f(x_1, x_2, ..., x_n)), then the gradient (\nabla f) is a vector:

[ \nabla f = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, ..., \frac{\partial f}{\partial x_n} \right) ]

The gradient indicates the direction of the greatest increase of the function, and its magnitude shows the rate of change of the function in that direction.

In machine learning, the gradient is used for optimization — for example, in gradient descent, where we move in the direction opposite to the gradient to minimize the loss function.

Example: for the function (f(x, y) = x^2 + y^2), the gradient will be (\nabla f = (2x, 2y)). At the point (1,1), the gradient is (2,2), indicating the direction of the greatest increase of the function.