Is it possible to use BERT together with a classifier for a classification task?

Yes, BERT is widely used together with classifiers for text classification tasks. Typically, a pre-trained BERT model is taken, the input text is passed through it, embeddings are obtained (for example, from the output layer of the CLS token), and then these embeddings are fed into a classifier — for example, a fully connected layer with softmax for multi-class classification.

An example in Python using the transformers library:

from transformers import BertTokenizer, BertModel
import torch
import torch.nn as nn

class BertClassifier(nn.Module):
    def __init__(self, pretrained_model_name='bert-base-uncased', num_classes=2):
        super().__init__()
        self.bert = BertModel.from_pretrained(pretrained_model_name)
        self.classifier = nn.Linear(self.bert.config.hidden_size, num_classes)

    def forward(self, input_ids, attention_mask):
        outputs = self.bert(input_ids=input_ids, attention_mask=attention_mask)
        cls_output = outputs.last_hidden_state[:, 0, :]  # CLS token
        logits = self.classifier(cls_output)
        return logits

Thus, BERT acts as a powerful feature extractor, and the classifier is a layer that solves the classification task based on these features.