What features are used in speech models (MFCC, mel-spectrogram)?

In speech models, features based on spectral analysis are often used to represent audio signals, such as MFCC and mel-spectrogram.

• MFCC (Mel-Frequency Cepstral Coefficients) — coefficients of the Mel-frequency cepstrum. They reflect the short-term spectral structure of sound, taking into account the characteristics of human perception. MFCCs are computed by transforming the signal into a spectrum, applying a Mel-scale filter bank, taking the logarithm, and then performing an inverse Fourier transform. These features compactly describe the timbral characteristics of speech and are widely used in speech recognition.

• Mel-spectrogram — a spectrogram where the frequency axis is scaled according to the Mel scale, which approximates human perception of sound frequencies. It shows the distribution of energy over time and frequency. Mel-spectrograms are often used as input for neural network models, especially in speech synthesis and classification tasks.

Both features account for the non-uniform perception of frequencies by humans, making them informative for speech analysis and processing.