What is histogram-based gradient boosting?

Histogram-based gradient boosting is an optimization of gradient boosting algorithms, where continuous features are pre-discretized into bins, and training occurs on these histogram representations.

Main advantages:

• Significantly speeds up the computation of gradients and the search for optimal splits, as operations are performed on a limited number of bins rather than all unique feature values.
• Reduces memory consumption.

An example of usage — popular libraries such as LightGBM use histogram-based boosting for efficient training on large tabular data.

The idea is to replace the search for optimal split thresholds with a search over discrete bins, which simplifies and accelerates the tree-building process.