Mis on Bayesian närvivõrgud ja kuidas arvutada posterior?

Bayesian Neural Networks (BNN) are neural networks where weights and parameters are considered as random variables with a prior distribution. Instead of fixed weights, BNN model uncertainty in parameters, allowing probabilistic predictions and confidence assessment of the model.

The main idea is to compute the posterior distribution of the network parameters ( p(\theta | D) ), where ( \theta ) are the network parameters and ( D ) are the training data. According to Bayes' theorem:

[ p(\theta | D) = \frac{p(D | \theta) p(\theta)}{p(D)} ]

• ( p(\theta) ) is the prior distribution of the parameters.
• ( p(D | \theta) ) is the likelihood function (probability of data given parameters).
• ( p(D) ) is the normalizing constant (marginal probability of data).

Exact calculation of the posterior is usually impossible due to high dimensionality and model complexity, so approximate methods are used:

• Variational methods (Variational Inference): approximate the posterior distribution with a simpler function, minimizing the Kullback-Leibler divergence.
• MCMC (Markov Chain Monte Carlo): sampling from the posterior using stochastic methods.
• Dropout as a Bayesian approximation (MC Dropout).

Example: the variational Bayesian approach defines a parametric distribution ( q(\theta) ) and optimizes its parameters to approximate ( p(\theta | D) ).

Thus, BNNs allow accounting for uncertainty in predictions, which is useful in tasks with limited data or high error costs.