Kuidas töötab gradientlangus mudeli parameetrite otsimiseks?

Gradient descent is an iterative optimization algorithm used to find the model parameters that minimize the loss function. The idea is to step-by-step adjust the parameters in the opposite direction of the gradient of the loss function, as the gradient indicates the direction of the steepest increase.

The process looks like this:

1. Initialize the model parameters with random values.

2. Calculate the gradient of the loss function with respect to the parameters — a vector of partial derivatives.

3. Update the parameters by moving in the direction of decreasing the loss function:

   ( \theta := \theta - \alpha \nabla L(\theta) )

   where ( \alpha ) is the learning rate.

4. Repeat steps 2-3 until convergence or until reaching the maximum number of iterations.

Example in Python using numpy:

import numpy as np

def gradient_descent(x, y, theta, learning_rate, iterations):
    m = len(y)
    for _ in range(iterations):
        predictions = x.dot(theta)
        errors = predictions - y
        gradient = (1/m) * x.T.dot(errors)
        theta = theta - learning_rate * gradient
    return theta

# x — feature matrix, y — target variable, theta — model parameters

This way, gradient descent allows finding the optimal parameters by minimizing the model's error.