How does BERT capture the semantic similarity between sentences that are written differently but have the same meaning?

BERT captures the semantic similarity between different sentences that are written differently but have the same meaning thanks to the contextual representation of words and sentences. Unlike traditional models, BERT uses a transformer with an attention mechanism that considers the context of each word in a sentence, rather than just its isolated meaning.

During training, BERT predicts hidden words and their relationships, which allows the model to understand semantic connections. As a result, sentences with different sets of words but similar meanings obtain close vector representations (embeddings). To evaluate semantic similarity, embeddings of the CLS token or averaged word embeddings are typically used, and cosine similarity is calculated between them.

Example: sentences "I love reading books" and "Reading books gives me pleasure" will have close vector representations despite having different sets of words.