1 Per-class metrics in classification

1.1 Definitions and motivation

Per-class metrics are evaluation measures computed separately for each class in a dataset. In multi-class or multi-label tasks, a model’s behavior often varies substantially across categories, particularly when some labels are more frequent or more difficult than others. By reporting performance per class, practitioners can identify which specific categories are handled well and which exhibit systematic errors. This diagnostic granularity is especially useful when a single aggregate score conceals important weaknesses.

1.2 Relationship to confusion matrices

Most per-class metrics can be derived from a per-class view of the confusion matrix. For a given class, predictions are treated in a one-vs-rest manner: instances of the target class are considered “positive,” while all other instances are treated as “negative.” The resulting counts—true positives, false positives, false negatives, and true negatives—form the basis for precision, recall, F1-score, accuracy-like measures, and rate-based variants.

1.3 Multi-class vs multi-label settings

In multi-class classification, each instance typically has exactly one ground-truth label, and predictions select a single class. Per-class metrics are computed using the one-vs-rest decomposition of the multi-class confusion matrix.

In multi-label classification, each instance may have multiple ground-truth labels and the model may output multiple predicted labels or scores per label. Per-class metrics are still computed independently per label, but they depend on how continuous scores are converted into binary decisions. As a result, threshold choices and score calibration can significantly influence per-class results.

1.4 Class “support” and class imbalance

Support refers to the number of true instances for a class (ground-truth occurrences). Classes with high support tend to yield more stable metric estimates, whereas rare classes can show large fluctuations between evaluation runs. Per-class reporting exposes imbalance: a model may achieve strong overall performance while underperforming on low-support categories. Support also informs weighted aggregation schemes and can be included alongside metrics for interpretability.

2 Core per-class metric formulas

2.1 Precision (per class)

Per-class precision measures how reliable positive predictions are for a specific class. For a class \(c\), precision is defined as the fraction of predicted positives that are correct: \[ \text{Precision}_c = \frac{TP_c}{TP_c + FP_c}. \] High precision indicates fewer false alarms for that class.

2.2 Recall (per class)

Per-class recall quantifies how completely the model captures instances of a specific class: \[ \text{Recall}_c = \frac{TP_c}{TP_c + FN_c}. \] High recall means fewer missed detections for the class.

2.3 F1-score (per class)

The F1-score combines precision and recall into a single harmonic-mean statistic: \[ \text{F1}_c = \frac{2 \cdot \text{Precision}_c \cdot \text{Recall}_c}{\text{Precision}_c + \text{Recall}_c}. \] It penalizes large disparities between precision and recall, making it useful when either false positives or false negatives are costly.

2.4 Per-class accuracy

Per-class accuracy treats the one-vs-rest decision as a binary classification problem for each class: \[ \text{Accuracy}_c = \frac{TP_c + TN_c}{TP_c + TN_c + FP_c + FN_c}. \] Although often computed in practice, accuracy per class can be less informative under strong imbalance because true negatives can dominate the score.

Specificity (true negative rate) describes how well the model avoids predicting a class when it is absent: \[ \text{Specificity}_c = \frac{TN_c}{TN_c + FP_c}. \] Related rate measures include the false positive rate \( \text{FPR}_c = \frac{FP_c}{FP_c + TN_c} \) and the false negative rate \( \text{FNR}_c = \frac{FN_c}{FN_c + TP_c} \). Such rates complement precision/recall by highlighting error tendencies from both positive and negative perspectives.

2.6 Interpreting true/false positives and negatives

For each class, the one-vs-rest confusion components can be interpreted as follows:

  • True positives (TP): correct predictions of the class.
  • False positives (FP): predictions of the class when the instance belongs to another class (or lacks the label in multi-label settings).
  • False negatives (FN): failures to predict the class when it is present.
  • True negatives (TN): correct non-predictions for the class when it is absent.

Understanding these quantities clarifies why metrics behave as they do. For instance, precision decreases when FP grows, while recall decreases when FN grows.

3 Computation workflow

3.1 Selecting the evaluation task (single-label or multi-label)

The workflow begins by specifying whether evaluation follows single-label or multi-label conventions. In single-label multi-class tasks, each instance maps to exactly one predicted class (often the argmax of model scores). In multi-label tasks, evaluation requires converting per-label scores into binary predictions, typically using a fixed threshold or label-specific thresholds.

3.2 Extracting per-class predictions

For each instance, the model outputs either:

  • a single predicted class (multi-class), or
  • a set of predicted labels derived from scores (multi-label).

These predictions are aligned with the ground-truth label representation so that per-class comparisons are consistent.

3.3 Building per-class confusion matrix components

For each class \(c\), the evaluation constructs TP, FP, FN, and TN using the one-vs-rest rule:

  • TP: predicted positive and ground-truth positive
  • FP: predicted positive and ground-truth negative
  • FN: predicted negative and ground-truth positive
  • TN: predicted negative and ground-truth negative

In multi-class problems, “ground-truth positive” is membership in class \(c\), and “predicted positive” is the predicted label equaling \(c\). In multi-label problems, membership is label presence, and predictions are derived after thresholding.

3.4 Handling thresholding (when applicable)

Thresholding applies mainly to multi-label evaluation (and sometimes to specialized decision rules). A common approach compares a score for class \(c\) against a threshold \(t\): the label is predicted as positive if the score exceeds \(t\). Adjusting \(t\) alters TP, FP, and FN, which in turn changes precision and recall. To avoid misleading comparisons, it is important to report which threshold strategy was used.

3.5 Producing metric tables by class

After computing per-class confusion components, the evaluation produces a table listing, for each class, metrics such as precision, recall, F1-score, and support. Many implementations also include additional rate metrics or per-class accuracy. Presenting support alongside the metrics helps readers interpret variability and class-specific reliability.

4 Aggregation and comparison

4.1 Macro averaging across classes

Macro averaging computes the mean of per-class metrics over all classes, treating each class equally: \[ \text{Macro-avg} = \frac{1}{K}\sum_{c=1}^{K} \text{Metric}_c. \] This approach highlights performance on minority classes, but it can yield lower aggregate values if some rare categories are difficult.

4.2 Micro averaging across instances

Micro averaging aggregates counts globally by summing TP, FP, and FN across classes before computing the metric. This effectively weights classes by their support (number of instances). Micro-averaged F1 or precision/recall tends to reflect performance on frequent labels more strongly than macro averaging.

4.3 Weighted averaging by support

Weighted averaging computes a per-class mean where each class metric is weighted by its support: \[ \text{Weighted-avg} = \frac{\sum_{c} \text{support}_c \cdot \text{Metric}_c}{\sum_{c} \text{support}_c}. \] This balances the equal-class emphasis of macro averaging with the influence of class frequency.

4.4 Comparing per-class vs overall metrics

Overall metrics typically rely on aggregation and can mask localized failure modes. Per-class metrics enable targeted comparison, such as:

  • identifying classes with low recall (missed detections),
  • identifying classes with low precision (many false alarms),
  • checking whether errors cluster in specific categories.

Comparing aggregated and per-class results helps determine whether issues are widespread or concentrated.

4.5 Error analysis using class rankings

One practical method for error analysis is ranking classes by a chosen metric (e.g., lowest F1-score) or by error magnitude (e.g., highest FN count). This ranking can reveal systematic patterns, such as consistent confusion between specific pairs of classes or vulnerability to certain kinds of inputs that predominantly affect one category.

5 Practical considerations and pitfalls

5.1 Zero-division cases and undefined metrics

Some metrics become undefined when denominators are zero. For example, precision is undefined when \(TP_c + FP_c = 0\) (no predicted positives for class \(c\)). Recall can be undefined when \(TP_c + FN_c = 0\) (class \(c\) is absent in ground truth). Evaluation protocols typically handle this via explicit conventions (e.g., assigning zero or skipping the class), and the chosen convention should be documented to avoid inconsistent reporting.

5.2 Rare classes and unstable scores

For classes with very small support, per-class metrics can swing dramatically with small changes in predictions. A difference of a few instances can substantially alter TP/FP/FN counts. Stable evaluation often requires sufficient data size or repeated evaluation under controlled splits, along with the reporting of support.

5.3 Effects of label imbalance

Class imbalance affects both training and evaluation. In per-class reporting, imbalance often manifests as:

  • high accuracy but low recall for rare labels,
  • overly high precision paired with low recall (model predicts the rare label rarely but when it does it is often correct),
  • inconsistent F1 across classes.

Per-class metrics allow these effects to be observed directly rather than inferred from aggregate scores.

5.4 Calibration and score-to-label conversion

In multi-label settings, reported per-class metrics depend on how scores are converted to labels. Poorly calibrated scores can cause a model to overpredict or underpredict certain labels, distorting precision and recall. Calibration techniques (or threshold tuning) can improve per-class performance profiles, but comparisons require consistent evaluation settings.

5.5 Consistent class indexing and reporting

Per-class metrics require a stable mapping between class indices and class names. Misalignment—such as swapped label ordering between training and evaluation—can produce misleading results that appear plausible but refer to the wrong categories. Rigorous evaluation pipelines enforce consistent label mapping and include clear reporting of which metric row corresponds to which class.

6 Visualization and reporting

6.1 Metric-by-class tables

A per-class metric table typically lists precision, recall, F1-score, accuracy-like measures, and support for each class. Sorting can be used to emphasize worst-performing categories or to group semantically related classes. Including support in the same table improves interpretability.

6.2 Heatmaps from per-class confusion matrices

Heatmaps visualize confusion structure for multi-class problems. Although the full confusion matrix captures inter-class misclassification patterns, a per-class decomposition can also be visualized by emphasizing row/column sums corresponding to error rates for each class. Heatmaps help locate systematic confusions between particular classes.

6.3 Bar charts for precision/recall/F1

Bar charts present per-class metrics compactly and are useful for comparing multiple measures simultaneously (e.g., grouped bars for precision and recall). When many classes exist, plotting only top-N and bottom-N categories can reduce clutter while preserving the diagnostic value.

6.4 Reporting with confidence intervals (overview)

When evaluation is repeated across folds or using bootstrap resampling, confidence intervals can be reported for per-class metrics. These intervals communicate uncertainty, which is especially relevant for low-support classes. Even a simple overview—interval width alongside the mean metric—can prevent overinterpretation.

6.5 Reproducible evaluation logs

Reproducibility benefits from logging evaluation settings: dataset split, model checkpoint, thresholding rules (for multi-label), class ordering, and metric definitions. Capturing these details enables consistent recomputation of per-class metrics and supports fair comparisons across experiments.

7 Variants and extensions

7.1 Per-class metrics for hierarchical labels (overview)

Some taxonomies organize labels in hierarchies (e.g., parent-child relationships). Per-class metrics can be extended to reflect partial correctness when predictions match at a higher level. For instance, a model might be penalized less for predicting a correct ancestor label even if the exact leaf category is wrong. Such variants require definitions of how hierarchy distance maps to evaluation outcomes.

7.2 Top-k per-class evaluation (overview)

Top-k evaluation checks whether the true class appears within the k most probable predictions. A per-class adaptation tracks, for each class \(c\), how often instances of \(c\) are included in the top-k list. This is common when misranking severity matters less than whether the model proposes plausible alternatives.

7.3 Class-wise metrics for imbalanced datasets

In imbalanced datasets, class-wise metrics may be reported alongside additional diagnostics such as per-class ROC or precision-recall curves. Although ROC and precision-recall curves are not “single-number” metrics, they complement per-class summaries by showing how performance varies across decision thresholds. This is useful when selecting operating points for different classes.

7.4 Per-class metrics in anomaly detection (conceptual mapping)

Anomaly detection often uses a scoring function rather than categorical labels. Per-class concepts can still apply by treating “normal” and “each anomaly category” as separate evaluation targets, or by mapping specific failure modes into pseudo-classes. In such cases, TP/FP/FN interpretations become conceptual: the focus shifts from predicting a discrete class to identifying which type of abnormality is present, using decision thresholds on anomaly scores.