How to use BERT in a SQuAD-style QA task?

To use BERT for the question-answering (QA) task in the SQuAD format, a model usually trained on this dataset is employed, which takes as input a pair: a question and a context (the text in which the answer needs to be found).

Main steps:

1. Preparing input data: combine the question and context into a single input text with separators, for example, [CLS] question [SEP] context [SEP].

2. Running through BERT: the model returns two sets of logits for each token — start and end of the answer.

3. Determining the answer: select the text segment with the highest sum of start and end probabilities.

Example using the Hugging Face Transformers library in Python:

from transformers import BertForQuestionAnswering, BertTokenizer
import torch

model_name = 'bert-large-uncased-whole-word-masking-finetuned-squad'
tokenizer = BertTokenizer.from_pretrained(model_name)
model = BertForQuestionAnswering.from_pretrained(model_name)

question = "Where is the Eiffel Tower located?"
context = "The Eiffel Tower is located in Paris, France."

inputs = tokenizer.encode_plus(question, context, return_tensors='pt')
input_ids = inputs['input_ids']

outputs = model(**inputs)
start_scores = outputs.start_logits
end_scores = outputs.end_logits

# Get start and end positions of the answer
answer_start = torch.argmax(start_scores)
answer_end = torch.argmax(end_scores) + 1

answer = tokenizer.convert_tokens_to_string(tokenizer.convert_ids_to_tokens(input_ids[0][answer_start:answer_end]))
print(f"Answer: {answer}")

Thus, BERT effectively finds the answer to a question within a given context by predicting the start and end positions of the answer in the text.