What is p-tuning?

P-tuning (prompt tuning) is a method of adapting large language models, where instead of retraining the entire model, a small set of parameters added as special "prompts" to the input data are optimized.

The idea is to find optimal vectors (prompt parameters) that, when provided along with the query, allow the model to produce the desired responses without changing the weights of the model itself.

Advantages of p-tuning:

• Significantly fewer parameters to train.
• Faster and cheaper than full retraining.
• Allows effective adaptation of the model for specific tasks.

Example: instead of training the entire model for classification, we only train a few parameters that are added to the query text and guide the model to the correct answer.