What is kernel fusion and where does it help?

Kernel fusion is an optimization in GPU programming where multiple consecutive kernels are combined into a single kernel. This reduces the number of kernel launches, cuts down on overhead from context switching, and decreases memory access.

Where kernel fusion helps:

• Accelerates the execution of operation chains, for example, in machine learning with sequential matrix operations.
• Reduces delays associated with synchronization between kernels.
• Improves memory utilization efficiency and bus bandwidth.

Example: instead of launching three separate kernels for addition, multiplication, and activation operations, they are combined into one kernel that performs all three operations sequentially, minimizing global memory accesses.