What are concept activation vectors (TCAV)?

TCAV (Testing with Concept Activation Vectors) is a method of interpretability for deep learning models that allows understanding how certain high-level concepts influence the model's predictions.

The idea is that instead of analyzing individual neurons or weights, TCAV uses an activation vector corresponding to a specific concept (e.g., "stripes", "circles", or "a particular object") and measures how changes in activation along this vector affect the model's output. This helps to understand how much the model relies on given concepts when making decisions.

Example of application: if you have an image classification model and want to check how much it uses the concept "stripes" to recognize a zebra, TCAV can quantitatively assess this contribution.

Thus, TCAV is a tool for explaining and validating models, increasing trust and transparency of AI.