What is a 1x1 convolution and what is it used for?

1x1 convolution is a convolution operation with a kernel size of 1 by 1 pixel. It is used in convolutional neural networks to change the number of channels (depth) of the input tensor without altering the spatial dimensions (width and height).

Main applications of 1x1 convolution:

• Reducing or increasing the dimensionality of channels, which helps decrease the number of parameters and computational load.
• Introducing non-linearity between layers using activation functions after 1x1 convolution.
• Mixing information between channels, improving feature representation.

Example in PyTorch:

import torch
import torch.nn as nn

conv1x1 = nn.Conv2d(in_channels=64, out_channels=128, kernel_size=1)
input_tensor = torch.randn(1, 64, 32, 32)  # batch_size=1, 64 channels, 32x32
output = conv1x1(input_tensor)
print(output.shape)  # torch.Size([1, 128, 32, 32])