How do mean/max pooling differ from learnable pooling?

Mean pooling and max pooling are fixed aggregation operations applied to input data (e.g., features in convolutional neural networks). Mean pooling computes the average value of elements in a window, while max pooling takes the maximum value. These methods have no learnable parameters and serve to reduce dimensionality and extract important features.

Learnable pooling (trainable pooling) is an approach where the aggregation operation is parameterized, and parameters are trained along with the rest of the network. For example, it can be a weighted average with trainable weights or more complex mechanisms like attention pooling. This allows the model to adapt to data and highlight more relevant features.

An example of simple learnable pooling — parameterized weighted pooling:

import torch
import torch.nn as nn

class LearnablePooling(nn.Module):
    def __init__(self, input_dim):
        super().__init__()
        self.weights = nn.Parameter(torch.randn(input_dim))

    def forward(self, x):
        # x shape: (batch, features)
        w = torch.softmax(self.weights, dim=0)
        return (x * w).sum(dim=1)

Thus, the main difference is that mean/max pooling is fixed and not trained, while learnable pooling has parameters that are optimized during training.