The F1-score is a statistical metric used to evaluate the performance of binary classification models. It is defined as the harmonic mean of precision and recall, yielding a single value between 0 and 1 that balances false positives and false negatives. The F1-score is particularly useful when dealing with imbalanced datasets, where accuracy alone can be misleading. It is widely applied in fields such as information retrieval, natural language processing, and medical diagnosis.

1 Definition and Formula

1.1 Precision and Recall

Precision (also called positive predictive value) is the fraction of correctly predicted positive instances among all instances predicted as positive: \[ \text{Precision} = \frac{TP}{TP + FP} \] Recall (sensitivity or true positive rate) is the fraction of correctly predicted positive instances among all actual positive instances: \[ \text{Recall} = \frac{TP}{TP + FN} \] where TP, FP, and FN denote true positives, false positives, and false negatives, respectively.

1.2 Harmonic Mean

The harmonic mean is a type of average that is more sensitive to small values than the arithmetic mean. The F1-score is defined as: \[ F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}} \] This formulation penalizes extreme imbalances between precision and recall, ensuring that a high F1-score requires both components to be reasonably high.

1.3 Generalization to Multiclass

For multi-class classification, the F1-score can be extended in several ways by averaging per-class scores.

1.3.1 Macro F1-score

The macro F1-score computes the F1-score independently for each class and then takes the unweighted arithmetic mean. It treats all classes equally, regardless of their frequency.

1.3.2 Micro F1-score

The micro F1-score aggregates the contributions of all classes into a global confusion matrix (summing TP, FP, FN across classes) and then calculates the F1-score from those totals. It is equivalent to accuracy when classes are perfectly balanced and reflects per-instance performance.

1.3.3 Weighted F1-score

The weighted F1-score computes the F1-score for each class and then averages them, weighted by the number of true instances per class. This variant accounts for class imbalance by giving more influence to larger classes.

2 Interpretation and Properties

2.1 Range and Ideal Values

The F1-score ranges from 0 (worst) to 1 (best). A value of 1 indicates perfect precision and recall (all positive predictions are correct and all positive instances are detected). A value of 0 occurs when either precision or recall is zero.

2.2 Sensitivity to Class Imbalance

Because the F1-score focuses on the positive class, it is not affected by a large number of true negatives. In imbalanced datasets, a model that always predicts the majority class may achieve high accuracy but a low F1-score, making F1-score a more informative metric.

2.3 Trade‑off between Precision and Recall

In many classification systems, increasing precision tends to decrease recall, and vice versa. The F1-score provides a single summary of this trade‑off. It is highest when both are high and equal, and it drops when one is much lower than the other.

2.3.1 Fβ‑score Variant

The Fβ‑score generalizes the F1-score by allowing a weighted harmonic mean: \[ F_\beta = (1 + \beta^2) \cdot \frac{\text{Precision} \cdot \text{Recall}}{\beta^2 \cdot \text{Precision} + \text{Recall}} \] A β > 1 gives more weight to recall, while β < 1 emphasizes precision. The F1-score corresponds to β = 1.

3 Computation and Usage

3.1 Calculation from Confusion Matrix

Given a binary confusion matrix with values TP, FP, TN, FN, the F1-score is computed using the formula in section 1.2. For example, if TP = 80, FP = 20, and FN = 10, then precision = 80/100 = 0.8, recall = 80/90 ≈ 0.889, and F1 ≈ 2 × (0.8 × 0.889)/(0.8 + 0.889) ≈ 0.842.

3.2 Software Implementation

3.2.1 scikit‑learn Example

In Python’s scikit‑learn library, the f1_score function can be used directly:

from sklearn.metrics import f1_score
y_true = [0, 1, 0, 1, 1]
y_pred = [0, 1, 0, 0, 1]
print(f1_score(y_true, y_pred))  # Output: 0.666...

Arguments such as average='macro' or average='weighted' allow multiclass variants.

3.2.2 Manual Calculation Steps

To compute the F1-score manually:

  1. Determine TP, FP, FN from the confusion matrix.
  2. Compute precision = TP / (TP + FP).
  3. Compute recall = TP / (TP + FN).
  4. Compute F1 = 2 × (precision × recall) / (precision + recall).

4 Applications and Limitations

4.1 Common Use Cases

4.1.1 Information Retrieval

In search engines and document retrieval, the F1-score is used to evaluate ranking and relevance. It balances the proportion of relevant documents retrieved (precision) with the proportion of relevant documents actually returned (recall).

4.1.2 Medical Diagnostics

For diagnostic tests, especially when detecting rare diseases, the F1-score provides a better assessment than accuracy. A test that correctly identifies all sick patients (high recall) may produce many false positives (low precision); the F1-score captures this trade‑off.

4.1.3 NLP and Text Classification

In tasks such as sentiment analysis, spam detection, and named entity recognition, the F1-score is the standard metric. Multiclass variants are routinely reported in evaluation benchmarks.

4.2 Limitations

4.2.1 Ignoring True Negatives

The F1-score completely disregards true negatives. For problems where correct identification of negative instances is critical (e.g., safety‑critical systems), the F1-score alone may be insufficient.

4.2.2 Asymmetric Costs

The F1-score treats false positives and false negatives equally, even when the costs of these errors differ. In practice, a false negative may be much more harmful than a false positive (e.g., failing to detect a disease). In such cases, the Fβ‑score or cost‑sensitive metrics are preferable.

5.1 Accuracy and Error Rate

Accuracy = (TP + TN) / (TP + TN + FP + FN) and its complement, error rate = 1 – accuracy, are straightforward but can be misleading for imbalanced datasets. The F1-score provides a more balanced view of positive class performance.

5.2 Precision‑Recall Curve

The precision‑recall curve plots precision against recall at various classification thresholds. The area under this curve (AUC‑PR) is another common metric for imbalanced problems. The F1-score corresponds to a single point on this curve where the harmonic mean is maximized.

5.3 Matthews Correlation Coefficient

The Matthews correlation coefficient (MCC) is a more comprehensive metric that accounts for all four confusion matrix entries (TP, TN, FP, FN). It ranges from –1 (perfect disagreement) to +1 (perfect agreement). Unlike the F1-score, MCC incorporates true negatives and is considered more informative for binary evaluation.