Classification Metrics: Why and When to Use Them

Classification models predict categorical outcomes, and evaluating their performance requires different metrics depending on the problem. Here’s a breakdown of key classification metrics, their importance, and when to use them.

1️. Accuracy
Formula:
Accuracy=Correct Predictions /Total Predictions
Use When: Classes are balanced (equal distribution of labels).
Avoid When: There’s class imbalance (e.g., fraud detection, where most transactions are legitimate).
Example: If a spam classifier predicts 95 emails correctly out of 100, accuracy = 95%.

2️. Precision (Positive Predictive Value)
Formula:
Precision=True Positives / (True Positives+False Positives)
Use When: False positives are costly (e.g., diagnosing a disease when the patient is healthy).
Avoid When: False negatives matter more (e.g., missing fraud cases).
Example: In cancer detection, high precision ensures fewer healthy people are incorrectly diagnosed.

3️. Recall (Sensitivity or True Positive Rate)
Formula:
Recall=True Positives / (True Positives+False Negatives)
Use When: Missing positive cases is dangerous (e.g., detecting fraud, security threats, or diseases).
Avoid When: False positives matter more than false negatives.
Example: In fraud detection, recall ensures most fraud cases are caught, even at the cost of false alarms.

4️. F1 Score (Harmonic Mean of Precision & Recall)
Formula:
F1=2×(Precision×Recall) / (Precision+Recall)
Use When: You need a balance between precision and recall.
Avoid When: One metric (precision or recall) is more important than the other.
Example: In spam detection, F1 ensures spam emails are detected (recall) while minimizing false flags (precision).

5️. ROC-AUC (Receiver Operating Characteristic – Area Under Curve)
What it Measures: The model’s ability to differentiate between classes at various thresholds.
Use When: You need an overall measure of separability (e.g., credit scoring).
Avoid When: Precise probability calibration is required.
Example: A higher AUC means better distinction between fraud and non-fraud transactions.

6️. Log Loss (Cross-Entropy Loss)
What it Measures: Penalizes incorrect predictions based on confidence level.
Use When: You need probability-based evaluation (e.g., medical diagnoses).
Avoid When: Only class labels, not probabilities, matter.
Example: In weather forecasting, log loss ensures a model predicting 90% rain probability is rewarded more than one predicting 60% if it actually rains.

Choosing the Right Metric
Scenario — Best Metric
Balanced Dataset- Accuracy
Imbalanced Dataset- Precision, Recall, F1-Score
False Positives are Costly- Precision
False Negatives are Costly- Recall
Need Overall Performance- ROC-AUC
Probability-Based Prediction- Log Loss

原文链接：Classification Metrics: Why and When to Use Them