What are the fundamental differences between GloVe and word2vec?

The main differences between GloVe and word2vec are related to their approach to training word vector representations:

• word2vec is a predictive model that learns to predict a word's context (CBOW) or the word from its context (Skip-gram). It optimizes the probability of words appearing in context using a neural network approach.

• GloVe is a model based on matrix factorization of a global word co-occurrence matrix. It considers the statistics of word co-occurrences across the entire corpus, attempting to approximate the logarithm of the probability of co-occurrence pairs.

Thus, word2vec focuses on local context and prediction, while GloVe emphasizes global statistical relationships between words. In practice, GloVe may better capture global semantic relationships, whereas word2vec captures local contextual dependencies.

An example use case: for semantic search tasks, GloVe can be chosen if global semantics are important, or word2vec if the context of specific phrases is crucial.