What is negative sampling in training word2vec and why is it needed?

Negative sampling is an optimization technique used in training the word2vec model for efficient learning in tasks with a very large vocabulary. Instead of computing the probability for all words in the vocabulary (which is very costly), negative sampling selects a few 'negative' examples (words not in the context of the given word) and trains the model to distinguish between real word pairs (positive examples) and random pairs (negative).

This allows for significantly faster training and reduced computational costs while maintaining the quality of the embeddings. As a result, the model learns to increase the probability of neighboring words and decrease the probability of random words.

Example: for a pair (word, context) — a positive example, several random words are selected from the vocabulary as negatives, and the model is trained to distinguish them.

# Pseudocode for negative sampling
for target_word, context_word in corpus:
    positive_pair = (target_word, context_word)
    negative_samples = sample_negative_words(k)  # k is the number of negative examples
    update_model(positive_pair, negative_samples)