What is padding mask and why must it be considered in attention?

Padding mask is a mask that indicates to the Transformer model which positions in the sequence are fillers (padding) and which are actual data. In NLP tasks, input sequences often have different lengths, so they are padded to the same length with special tokens (padding) for batch processing.

It is essential to consider the padding mask in attention so that the model does not consider these fillers when computing attention. If padding is not masked, the model may pay attention to empty positions, leading to incorrect calculations and degraded quality.

In implementation, the attention mask is applied to the scoring matrix (usually by adding a large negative number to padding positions before applying softmax) so that attention weights for padding are close to zero.

Example (pseudocode):

scores = Q @ K.T / sqrt(d_k)
# mask is a boolean array where True indicates padding
scores = scores.masked_fill(mask, -1e9)
attention_weights = softmax(scores)