What is depthwise separable convolution and where is it used (MobileNet)?

Depthwise separable convolution is a type of convolution that breaks down a standard convolution into two separate operations:

1. Depthwise convolution — applies a separate filter to each input channel (depth) individually.
2. Pointwise convolution — a 1x1 convolution that combines the results from the depthwise layer to create new features.

This approach significantly reduces the number of parameters and computational complexity compared to regular convolution, while maintaining the model's quality.

In MobileNet, depthwise separable convolution is used to build lightweight and efficient models suitable for mobile and embedded devices with limited resources.

Example: if a regular 3x3 convolution with N input and M output channels requires 33NM parameters, then depthwise separable convolution requires 33N (depthwise) + 11NM (pointwise), which is much smaller.