What are masked autoencoders (MAE) for CV?

Masked Autoencoders (MAE) for computer vision are a type of models trained to reconstruct the hidden (masked) part of an image from the visible part. The idea is that an image with randomly masked (hidden) patches is input, and the model's task is to reconstruct these masked patches.

This helps the model learn to understand the structure and context of the image without the need for annotations. MAE often use transformers, which work well with sequences of image patches.

Example:

• The image is divided into patches (e.g., 16x16 pixels).
• A large portion of patches (e.g., 75%) is randomly selected for masking.
• The model receives only the unmasked patches and must predict the pixels of the masked patches.

This approach is effective for learning image representations in a self-supervised mode, which improves subsequent classification or other CV tasks.