What features are used in classic NLP models (TF-IDF, n-grams, character n-grams)?

In classical NLP models such as TF-IDF and n-grams, the following features are used:

• TF-IDF (Term Frequency-Inverse Document Frequency):

  • Term Frequency (TF) — the frequency of a word in a document.
  • Inverse Document Frequency (IDF) — the inverse frequency of the document, indicating how unique a word is to that document relative to the entire corpus.
  • The final feature is the product of TF and IDF for each word, reflecting the importance of the word in the document.

• n-grams:

  • Sequences of n consecutive words (e.g., bigrams — pairs of words, trigrams — triplets).
  • Used to capture context and phraseology.

• Character n-grams:

  • Sequences of n consecutive characters.
  • Help account for morphology, errors, and writing features.

These features are usually encoded as vectors, where each element corresponds to the frequency or weight of a specific word or n-gram in the text.