What is the RBF kernel and its hyperparameters?

RBF kernel (Radial Basis Function) is a popular kernel in machine learning methods, especially in SVMs and Gaussian processes. It measures the similarity between two points in space based on their Euclidean distance.

RBF kernel formula:

K(x, x') = exp(-\gamma \|x - x'\|^2)

Where:

• (x, x') are input vectors,
• (\gamma > 0) is a hyperparameter controlling the width of the kernel.

Hyperparameters:

• (\gamma) — determines the influence radius of a single training point. A small (\gamma) means influence spreads far, resulting in a smoother model. A large (\gamma) means influence is local, which can lead to overfitting.
• Model parameters using RBF kernel (e.g., in SVM — regularization parameter C) also affect the final model, but (\gamma) directly relates to the kernel.

Example of setting in SVM:

from sklearn.svm import SVC
model = SVC(kernel='rbf', gamma=0.1, C=1.0)

The RBF kernel is well-suited for nonlinear tasks where data is not linearly separable.