What is ColBERT and what is late interaction?

ColBERT (Contextualized Late Interaction over BERT) is a model for efficient and accurate information retrieval based on using BERT to create text embeddings.

Main idea: ColBERT employs a "late interaction" approach, where contextual embeddings of individual tokens are first computed for the query and documents using BERT, and then interaction between the query and document occurs at the level of these embeddings, rather than at the level of entire sentences or documents.

Late interaction means that instead of combining all tokens into a single vector (early interaction), the model retains embeddings of each token and compares them with the embeddings of query tokens using efficient operations (e.g., maximum scalar product). This allows for better consideration of local matches and relevance of individual parts of the text.

This approach provides a balance between accuracy (thanks to deep BERT context) and efficiency (due to separate processing and indexing of token embeddings).

The application of ColBERT is relevant in search systems where it is necessary to quickly find relevant documents based on complex queries considering context.