How does PR-AUC differ from ROC-AUC?

PR-AUC (Precision-Recall Area Under Curve) and ROC-AUC (Receiver Operating Characteristic Area Under Curve) are metrics for evaluating the quality of binary classifiers, but they reflect different aspects and are applicable in different situations.

• ROC-AUC is based on the dependence of True Positive Rate (sensitivity) on False Positive Rate. It shows how well the model can distinguish positive and negative classes overall and is resistant to class imbalance.

• PR-AUC is based on the dependence of Precision on Recall. This metric is more informative in cases of strong class imbalance, where the positive class is rare. PR-AUC indicates how well the model finds positive examples without many false positives.

Example: if you have a task of detecting rare diseases, PR-AUC will give a more realistic assessment of the model's quality than ROC-AUC, as ROC-AUC can be inflated due to a large number of negative examples.