What is NSP (next sentence prediction) and why did RoBERTa abandon it?

Next Sentence Prediction (NSP) is a task in which the model is trained to determine whether the second sentence is a logical continuation of the first. In BERT, NSP was used to improve understanding of text coherence.

RoBERTa abandoned NSP because experiments showed that this task does not improve the quality of the model and can even hinder training. Instead, RoBERTa focused on more thorough training using only the masked language modeling task, which yielded better results.

Thus, NSP is an additional task that is not always necessary and can be excluded to simplify and improve model training.