1 Definition and Motivation
1.1 What “macro” means in evaluation
Macro averaging is an evaluation technique that summarizes performance across multiple categories by first computing a metric for each category independently and then averaging those category-level values. The prefix “macro” indicates that the aggregation is carried out at the level of categories rather than weighting by the amount of data in each category.
1.2 When macro averaging is appropriate
Macro averaging is most suitable when the goal is to treat each category as equally important in the final score. This is common in research settings where investigators want performance to reflect the model’s ability across the entire label set, including less frequent classes, rather than being dominated by the majority class.
1.3 Comparison with micro averaging and weighted averaging
Macro averaging differs from other common aggregation schemes:
- Micro averaging aggregates contributions across all categories before computing the metric, which effectively emphasizes categories with more instances.
- Weighted averaging uses per-class metrics combined with weights related to class frequency (or another chosen weight), producing a compromise between micro-like dominance and strict equal-category treatment.
In practice, macro scores are often more sensitive to poor performance on rare categories, while micro and weighted scores may mask these deficiencies.
2 Core Procedure
2.1 Compute per-class metrics
The procedure begins by selecting an evaluation metric (for example, precision, recall, or F1). For each class \(c\), the metric is computed using predictions restricted to that class versus all other classes, yielding a per-class value \(M_c\).
2.2 Handle class-specific denominators and edge cases
Per-class metrics may involve denominators that can be zero for a given class. For instance, precision for a class can become undefined if there are no predicted positives for that class. Implementations address this in a consistent way, such as defining the metric as zero, skipping the class, or using a library-specific convention. The chosen convention affects the final macro score.
2.3 Aggregate by unweighted mean
Once per-class values \(M_c\) are available, macro averaging computes the unweighted mean: \[ M_{\text{macro}}=\frac{1}{K}\sum_{c=1}^{K} M_c \] where \(K\) is the number of classes included in the computation. The term “unweighted” is central: each class contributes equally to the final result.
2.4 Relationship to confusion matrix entries
Many per-class metrics can be expressed in terms of confusion matrix quantities. For a given class \(c\), typical counts include:
- True positives: instances correctly predicted as \(c\)
- False positives: instances predicted as \(c\) but belonging to another class
- False negatives: instances belonging to \(c\) but predicted as another class
Precision, recall, and F1 for class \(c\) are derived from these quantities, and macro averaging then averages the class-level outcomes.
3 Macro-Averaged Metrics
3.1 Macro-precision
Macro-precision is the unweighted mean of per-class precision values: \[ P_{\text{macro}}=\frac{1}{K}\sum_{c=1}^{K} \frac{\text{TP}_c}{\text{TP}_c+\text{FP}_c} \] It measures, for each class, how often predictions labeled as that class are correct, and then summarizes across classes with equal influence.
3.2 Macro-recall
Macro-recall averages per-class recall: \[ R_{\text{macro}}=\frac{1}{K}\sum_{c=1}^{K} \frac{\text{TP}_c}{\text{TP}_c+\text{FN}_c} \] This reflects, for each category, what fraction of true instances are successfully recovered by the model, regardless of class prevalence.
3.3 Macro-F1 score
Macro-F1 combines precision and recall at the class level using F1 for each class and then averaging: \[ F1_{\text{macro}}=\frac{1}{K}\sum_{c=1}^{K} \frac{2P_cR_c}{P_c+R_c} \] It balances false positives and false negatives for each category before applying equal weighting across categories.
3.4 Macro accuracy (clarifying common usage)
In strict formal terms, “accuracy” is typically defined per instance (correct vs. incorrect), yielding a single scalar without a direct per-class decomposition. However, in practice, some reporting uses “macro accuracy” to mean the mean of per-class recalls, which equals balanced accuracy in multiclass settings: \[ \text{Macro accuracy} \approx \frac{1}{K}\sum_{c=1}^{K} \frac{\text{TP}_c}{\text{TP}_c+\text{FN}_c} \] Because terminology varies across tools and papers, clarity about the exact definition is important.
4 Practical Considerations
4.1 Class label sets and mapping
Macro averaging assumes a well-defined label set. If labels are mapped from one representation to another (e.g., merging subclasses into broader categories), the per-class metrics must be computed after the mapping so that each final category aligns with the intended evaluation scheme.
4.2 Imbalanced datasets and interpretability
Macro scores are often reported when class imbalance is present. By giving every class equal weight, macro averaging makes the evaluation sensitive to failures on minority categories. Interpreting macro results therefore requires attention to whether the evaluation objective emphasizes equal per-class competence or reflects operational priorities.
4.3 Missing classes and zero-support scenarios
A frequent issue occurs when a class has zero support in the evaluation data (no true instances of that class). In such cases, per-class recall may be undefined or treated as zero depending on convention. If a class is never present, averaging it can distort the intended meaning of “balanced performance.” Many practitioners either exclude absent classes from the macro computation or document the convention used by their library.
4.4 Thresholding and averaging across decision scores
For probabilistic models, predicted labels may come from thresholding decision scores. When computing macro metrics, the chosen threshold can strongly influence per-class denominators (and thus precision/recall behavior). If scores are averaged across thresholds or tuned per class, the macro computation must remain consistent with the final prediction rule used to generate the class-wise decisions.
5 Variants and Extensions
5.1 Macro averaging in multilabel classification
In multilabel tasks, each instance can belong to multiple classes simultaneously. Per-class precision and recall are computed by treating each label as its own binary classification problem (label present vs. absent). Macro averaging then averages these label-level metrics equally. This yields a score that accounts for which labels are reliably predicted, independent of how common each label is.
5.2 Macro averaging for hierarchical or structured labels
When labels form a hierarchy (e.g., categories with parent-child relationships), structured evaluation may define partial credit for predicting related labels. Macro averaging can be adapted by computing metric values per node level or per structured component and then averaging across those components, enabling fair comparison across the hierarchy.
5.3 Macro averaging over folds (cross-validation)
Cross-validation introduces a second axis of averaging: performance across data splits (folds) and across classes. A common approach is to compute macro metrics per fold and then average those fold-level macro values. Alternatively, one can accumulate predictions across folds and compute macro once; these two approaches can differ slightly because the averaging occurs at different stages.
5.4 Macro averaging with probability calibration contexts
Calibration affects how decision thresholds are chosen or how confidence estimates are interpreted. In settings where models are calibrated (for example, to align predicted probabilities with observed frequencies), macro metrics are still computed from resulting class decisions. The calibration step may improve macro performance by stabilizing the precision–recall trade-off across classes, but it does not change the definition of macro averaging itself.
6 Implementation Notes
6.1 Common pitfalls in software libraries
Different libraries implement macro averaging with varying defaults regarding:
- Handling of undefined per-class metrics when denominators are zero
- Inclusion or exclusion of classes with no predictions or no true instances
- Interpretation of “accuracy” when labeled outputs require per-class decomposition
To avoid misinterpretation, users should verify the library’s documentation and explicitly set parameters controlling these edge-case behaviors.
6.2 Reproducibility checklist
A reproducible evaluation typically records:
- The metric type (macro precision, macro recall, macro F1, or macro accuracy definition)
- The exact label set used
- The treatment of zero-support and zero-prediction classes
- The prediction generation procedure (thresholding strategy or argmax rule)
- The cross-validation aggregation method, if applicable
These details ensure that macro results can be replicated and compared across experiments.
6.3 Example pseudocode for per-class aggregation
Conceptual pseudocode for macro averaging precision or recall follows the same pattern:
- Determine the set of classes \(C\).
- For each class \(c\):
- Compute TP\(_c\), FP\(_c\), FN\(_c\).
- Compute the per-class metric \(M_c\) using the chosen convention for edge cases.
- Return the unweighted mean across classes:
| - \(M_{\text{macro}} = \frac{1}{ | C | }\sum_{c \in C} M_c\) |
|---|
This structure generalizes across precision, recall, and F1.
7 Reporting and Interpretation
7.1 How to present macro results in papers
When reporting macro metrics, papers typically include:
- The metric name (e.g., macro-F1)
- The label set size \(K\)
- Any special handling of edge cases (zero divisions, absent classes)
- The evaluation setting (multiclass vs. multilabel, cross-validation details)
Providing per-class metrics or a brief summary table can further clarify where macro scores come from.
7.2 Interpreting divergences between macro and micro scores
Large gaps between macro and micro values usually indicate that performance differs substantially across classes. If macro is low but micro is high, the model may do well on frequent categories while struggling with rare ones. Conversely, macro higher than micro can indicate more uniform or minority-focused success relative to overall frequency.
7.3 Communicating limitations and assumptions
Macro averaging equalizes class influence regardless of prevalence, which is often desired but not always aligned with real-world costs. Authors typically note that macro results reflect an emphasis on per-class fairness rather than overall incidence. Additionally, they document the definition used for “macro accuracy,” since terminology can vary.
8 Worked Example (Conceptual)
8.1 Per-class metric calculation walkthrough
Consider a three-class classification problem with classes \(A\), \(B\), and \(C\). For each class, compute:
- Precision \(P_A, P_B, P_C\) from TP and FP for each class
- Recall \(R_A, R_B, R_C\) from TP and FN for each class
- If using F1, compute \(F1_A, F1_B, F1_C\) from \(P\) and \(R\)
Suppose the per-class precision values are \(P_A=0.9\), \(P_B=0.5\), and \(P_C=0.2\). Each is computed independently using only the counts relevant to that class.
8.2 Aggregation into a macro score
The macro-precision is the unweighted mean: \[ P_{\text{macro}}=\frac{0.9+0.5+0.2}{3}=\frac{1.6}{3}\approx 0.533 \] The same averaging pattern applies to macro-recall and macro-F1, substituting the class-specific metric values.
8.3 Interpreting the final macro metric
In this example, the macro score reflects a balance across classes: despite high precision on \(A\), the low precision on \(C\) substantially reduces the final macro value. This outcome is typical of macro averaging: it penalizes uneven performance and surfaces weaknesses on underperforming categories.