1 Concept and Motivation

Out-of-distribution (OOD) describes inputs that are not drawn from the same probability distribution as the data used to train and evaluate a machine learning model. When a model encounters such unfamiliar inputs, it may respond unpredictably—sometimes producing confident but incorrect predictions. This motivates research into methods that can recognize unfamiliarity, adjust confidence, or maintain useful performance when conditions differ from training.

1.1 In-distribution vs. out-of-distribution

In-distribution (ID) refers to data that match the training distribution assumptions, at least approximately. OOD inputs come from a different distribution, which can be more distant from the ID distribution or may differ in specific nuisance factors. In practice, the boundary between ID and OOD is rarely known precisely; it is often operationalized using held-out datasets or explicitly constructed shifts.

1.2 Why OOD happens in practice

Several mechanisms lead to real-world OOD behavior. Sensor conditions may change, acquisition pipelines can differ, environmental factors may vary, and user behavior can evolve. Additionally, even if the underlying task remains the same, background context or measurement noise can alter observed inputs. Models trained on one set of conditions can then encounter inputs that resemble the task superficially while differing in subtle, predictive correlations.

1.3 Common evaluation scenarios

OOD evaluation is commonly designed using controlled shifts. Typical scenarios include: testing on a dataset collected under different conditions; holding out certain classes or groups so that test samples are “unseen” during training; and generating perturbed inputs through synthetic transformations. These scenarios aim to isolate which types of unfamiliarity a method can detect or handle.

1.4 Relationship to domain shift

OOD is closely related to domain shift, though the terms are used differently across subfields. Domain shift often denotes changes in data-generating processes between training and deployment, whereas OOD emphasizes the statistical mismatch relevant to model predictions. OOD can be seen as a particular lens for domain shift, focusing on whether the model can detect or generalize beyond its assumed distribution.

2 Problem Formulations

OOD research includes multiple related tasks, each making different assumptions about what is known at test time. The main categories are OOD detection, OOD generalization, and open-set recognition, where the test stream may include previously unseen classes or individuals.

2.1 OOD detection

OOD detection aims to identify which test inputs are likely to be outside the training distribution. The approach varies depending on whether labels for OOD classes are available and how the ID/OOD separation is evaluated.

2.1.1 Binary classification framing

A standard formulation converts the problem into a binary decision: “ID” versus “OOD.” The model produces a score indicating unfamiliarity; higher scores can correspond to greater likelihood of being OOD. Ground truth for this binary task is established by labeling examples from the ID dataset as positive or negative and OOD examples from separate distributions as the opposite label.

2.1.2 Scoring functions and thresholds

Detection systems rely on a scoring function plus a decision rule. Common choices include using confidence signals derived from the model output, energy-like quantities computed from logits, or learned detectors that map internal representations to an OOD probability. Threshold selection then turns a continuous score into an accept/reject decision.

Detection performance is frequently summarized with threshold-free metrics. Receiver operating characteristic (ROC) curves plot true positive rate against false positive rate across thresholds. Related measures include detection error tradeoffs and area-under-curve summaries that provide a compact view of separability.

2.1.2 Precision-recall tradeoffs

In settings with class imbalance—such as when OOD samples are rare—precision-recall behavior can be more informative than ROC. Precision-recall curves capture the tradeoff between correctly identifying OOD and incorrectly flagging ID samples, which is often important when false alarms carry a cost.

2.2 OOD generalization

OOD generalization addresses performance on OOD samples rather than only detecting them. A model may abstain, route to a fallback, or still produce predictions. The central question is how well it can maintain accuracy or calibrated behavior when the input distribution changes.

2.2.1 Robustness under distribution changes

Robustness formulations evaluate worst-case or average-case performance under defined perturbations and domain shifts. Methods may encourage invariance to nuisance factors, reduce reliance on spurious correlations, or shape decision boundaries to behave more smoothly across distribution changes.

2.2.2 Transfer-like settings

Some OOD generalization settings resemble transfer learning, where the target distribution differs from the source but shares some structural relationship. Approaches may adapt representations, reweight samples, or use training signals that anticipate distribution mismatch.

2.3 Open-set recognition

Open-set recognition allows the possibility that the test stream contains unknown classes. The key distinction from generic OOD detection is that “unknown” refers specifically to class-level novelty rather than only distribution-level mismatch.

2.3.1 Unknown classes vs. unknown instances

Unknown classes mean that the model has never encountered those categories during training, although they may be related to known categories. Unknown instances can also arise when inputs belong to known classes but are atypical or corrupted in ways that place them away from the ID manifold. Many methods struggle when both class-level novelty and instance-level anomalies are present simultaneously.

3 Data and Benchmarking

Benchmarking OOD methods requires careful dataset construction. A major challenge is ensuring that OOD splits are neither trivial nor identical in disguised ways to the ID data.

3.1 Dataset construction for OOD

A common practice is to train on an ID dataset and evaluate on either: (i) another dataset collected under different conditions, (ii) a held-out subset created by design, or (iii) transformed versions of the ID samples. For binary detection, OOD examples must come from a distribution that differs in a controlled manner so that detection is meaningful.

3.2 Types of distribution shift

Distribution shifts can change input statistics, label relationships, or both. Understanding which component changes is essential for interpreting results.

3.2.1 Covariate shift

Covariate shift occurs when the input distribution changes but the conditional relationship between labels and inputs stays stable. Models trained under one covariate regime may generalize poorly if they rely on patterns that are not invariant across environments.

3.2.2 Label shift

Label shift means the distribution of labels changes while the conditional distribution of inputs given labels remains the same. This affects calibrated probabilities and can bias decision thresholds if a model implicitly assumes a stable class prior.

3.2.3 Concept drift

Concept drift involves changes in the underlying relationship between inputs and labels. Here, the task itself changes, making it difficult to reuse prior knowledge without adaptation.

3.2.4 Spurious feature changes

Spurious feature shifts occur when models exploit shortcuts—features correlated with labels in training but not causally relevant. If those shortcuts disappear or change in the OOD environment, performance can drop abruptly, and confidence signals may become unreliable.

3.3 Synthetic vs. real OOD sources

Synthetic OOD sources include additive noise, geometric transformations, style transfers, and perturbations guided by augmentations. Real OOD sources come from different data collection pipelines, sensors, or populations. Synthetic benchmarks often offer repeatability, while real benchmarks better capture complex mismatch but are harder to control.

3.4 Protocols and split strategies

Protocols specify how to form ID training data, ID test data, and OOD test data, along with whether hyperparameters can be tuned using OOD examples. Split strategies range from simple “train on A, test on B” to multi-domain settings where multiple domains exist and some are reserved for evaluation. Rigorous reporting typically details preprocessing steps, model selection criteria, and how thresholds are chosen.

4 Uncertainty and Confidence

Uncertainty estimation provides tools for determining whether a model’s predictions are trustworthy. OOD detection is often framed as a particular kind of uncertainty: the model is less certain when the input is unfamiliar.

4.1 Calibration

Calibration refers to whether predicted confidence aligns with empirical correctness frequency.

4.1.1 Confidence calibration concepts

A perfectly calibrated classifier produces confidence values that match observed accuracy within bins. Under OOD, confidence can become miscalibrated, with the model assigning high probabilities to incorrect outputs due to learned correlations that do not hold in the new distribution.

4.1.2 Calibration metrics and diagnostics

Calibration is commonly assessed with reliability diagrams and scalar metrics that quantify deviations between predicted and observed outcomes. Diagnostics often include evaluating calibration separately on ID and OOD subsets to reveal where the mismatch emerges.

4.2 Predictive uncertainty types

Uncertainty is typically decomposed into sources that are different in nature and respond differently to modeling choices.

4.2.1 Aleatoric vs. epistemic uncertainty

Aleatoric uncertainty captures noise inherent in the data-generating process, such as sensor noise or label ambiguity. Epistemic uncertainty reflects uncertainty about the model parameters due to limited data. OOD situations often increase epistemic uncertainty, though not always—especially for models that do not represent uncertainty explicitly.

4.2.2 Ensemble- and sampling-based uncertainty

Ensembles, Monte Carlo dropout, and other sampling methods estimate variability in predictions across model instances or training samples. This variability can correlate with unfamiliarity, enabling uncertainty-driven OOD scores. The computational cost and calibration quality depend on the specific method and training regime.

4.3 Confidence-based OOD scoring

Confidence-based detectors rely on the model’s own output to construct an OOD score.

4.3.1 Softmax probability approaches

One baseline uses maximum softmax probability: if the model’s top class probability is low, the input may be OOD. However, softmax confidence is not guaranteed to reflect correctness or familiarity, and it can remain high for certain OOD types, especially when the model is overconfident on outliers.

4.3.2 Logit-based and energy-based scores

Logit-based methods use raw outputs before normalization, while energy-based approaches compute a scalar derived from logits intended to correlate with likelihood-like quantities. These scores can behave differently from maximum softmax and may offer improved separation in certain benchmarks.

5 Representation Learning for OOD

Representation learning aims to structure internal features so that ID and OOD inputs become easier to separate. This can improve both detection and downstream robustness.

5.1 Feature space separation

A practical goal is to map ID inputs to regions of feature space with consistent structure and to push OOD samples away from those regions. Separation can be measured indirectly through detector performance, or directly via embedding clustering and distance statistics.

5.2 Domain-invariant representations

Domain-invariant representations try to preserve task-relevant information while discarding variations caused by environment-specific factors. Achieving invariance often involves training objectives that reduce sensitivity to domain labels or perturbations that are not causally linked to the target.

5.3 Self-supervised pretraining and OOD

Self-supervised learning can provide general features learned from large-scale unlabeled data. Such features may improve OOD robustness by capturing broad visual, textual, or structural regularities. The degree of benefit depends on whether pretraining data resembles the deployment shifts.

5.4 Augmentation strategies to reduce shift

Data augmentation modifies inputs during training to encourage invariance and broaden coverage of possible test variations.

5.4.1 Data augmentation and invariance targets

Common augmentations include geometric transformations, color or style perturbations, and masking. Methods differ in what invariance they enforce—some aim for robustness to nuisance transformations while others also attempt to reduce reliance on fragile cues.

6 Model-Based OOD Methods

Beyond detection-by-confidence, many approaches incorporate OOD awareness into training using regularization, learned objectives, or generative assumptions.

6.1 Regularization and loss functions

Regularization modifies the learning process so that confidence and embeddings behave better under mismatch.

6.1.1 Auxiliary OOD objectives

Auxiliary objectives introduce additional training signals that explicitly penalize OOD-like behavior. Depending on method design, OOD samples might be drawn from auxiliary datasets, from synthetic perturbations, or from deliberately excluded regions of feature space.

6.1.2 Confidence penalties and margins

Confidence penalties reduce overconfident predictions by discouraging extreme probability assignments. Margin-based approaches can enforce separation between class scores in a way intended to avoid sharp decision boundaries that collapse under shift.

6.2 Generative modeling approaches

Generative modeling can provide an alternative view of unfamiliarity by focusing on how plausible an input is under a learned data model.

6.2.1 Likelihood vs. OOD detection assumptions

A common assumption is that OOD inputs have lower likelihood under a model trained on ID data. In practice, likelihood estimation quality and calibration depend on model capacity and training dynamics, so likelihood may not always correlate with OODness.

6.2.2 Reconstruction error as a signal

Reconstruction-based methods train an autoencoder-like model and use reconstruction quality as an indicator of unfamiliarity. If an OOD input lies outside the learned manifold, it may be harder to reconstruct, increasing the error. The signal can be confounded by model overfitting or by OOD types that still resemble ID at a low-level.

6.3 Discriminative alternatives

Discriminative methods treat OOD as deviation from learned class boundaries or as novelty relative to the training set.

6.3.1 One-class and novelty detection

One-class novelty detection trains using only ID data to learn the “normal” region. At test time, inputs far from this region are flagged as novel. Such methods can be effective when OOD forms a broad anomaly set, but they may struggle when OOD is structured or close to ID.

6.3.2 Margin-based classifiers for unfamiliarity

Margin-based classifiers incorporate an explicit notion of rejection by adding constraints that define how far inputs must be from known class regions to be considered unfamiliar. The tuning of margins and the choice of feature metric strongly affect behavior under realistic shift.

7 Evaluation Practices

Evaluation for OOD should include both threshold-free detection metrics and task-level performance assessments, along with diagnostics that reveal where methods fail.

7.1 Metrics beyond accuracy

Accuracy alone does not capture whether the model can abstain appropriately or avoid false alarms.

7.1.1 AUROC, AUPR, and detection error

AUROC summarizes detection separability across thresholds for ID-versus-OOD classification. AUPR emphasizes performance under skewed class ratios. Detection error measures often aggregate false positive and false negative rates in a single statistic, facilitating comparisons across methods.

7.1.2 False positive/false negative analysis

False positive analysis quantifies how often ID inputs are incorrectly flagged as OOD, which is crucial for user-facing systems. False negative analysis shows missed OOD cases that may lead to unsafe or incorrect predictions. Examining these failure modes can guide threshold selection and model improvements.

7.2 Robustness curves and sensitivity tests

Robustness curves vary the intensity of perturbations or shift magnitude to observe how detection quality degrades. Sensitivity tests can also evaluate whether performance depends strongly on particular corruption types or augmentation choices.

7.3 Cross-domain and cross-dataset generalization

Cross-domain evaluations train on one domain set and test on another, sometimes varying both within-task content and acquisition conditions. This probes whether the method captures general principles of unfamiliarity rather than exploiting dataset-specific artifacts.

7.4 Reproducibility and reporting standards

Reproducibility requires reporting preprocessing, model architecture, training schedules, tuning procedures, and detection threshold selection. Benchmark protocols should clearly state which data are available during training and validation for fair comparison.

8 Failure Modes and Pitfalls

OOD methods frequently fail in predictable ways. Understanding these pitfalls is essential for interpreting benchmark results and deploying models responsibly.

8.1 Confusing OOD with low-confidence in-distribution

Some detectors conflate unfamiliarity with uncertainty or difficulty within the ID distribution. A model may label hard-but-ID cases as OOD, increasing false positives. This can occur when the detector relies on confidence measures that correlate with error rather than with distribution mismatch.

8.2 Overfitting to benchmark OOD

Methods can unintentionally tailor decisions to a specific benchmark shift, learning patterns that do not generalize. Overfitting may be amplified when OOD data is repeatedly used for hyperparameter tuning or when the benchmark shift type is too narrow.

8.3 Distribution similarity effects

If an OOD distribution is close to ID in feature space, detection becomes inherently difficult. Conversely, if OOD is very different, many methods can perform well, masking deficiencies that appear under moderate shift. Similarity assumptions should therefore be considered when interpreting results.

8.4 “Threshold hacking” and metric leakage

Threshold hacking occurs when thresholds are tuned using test-set information or by selecting the operating point that maximizes a metric on the evaluation set. Metric leakage also arises when the evaluation pipeline inadvertently encodes test statistics. Both issues can inflate reported performance without reflecting real-world deployment.

8.5 Calibration issues under shift

Even well-calibrated models on ID data may become miscalibrated on OOD samples. Calibration drift can cause detectors based on confidence to underperform, requiring calibration-aware techniques or retraining under anticipated shifts.

9 Practical Deployment Considerations

Deployment changes the objective from controlled evaluation to managing uncertainty in an operational pipeline.

9.1 When and how to trigger abstention

Abstention policies determine whether the model returns a prediction, refuses, or routes to another system. Triggering abstention typically depends on a detection score and a cost model specifying the relative burden of false alarms versus missed OOD cases.

9.2 Fallback policies for uncertain inputs

Fallback systems can include requesting human review, using a specialized model, or returning a conservative response. The design should reflect the expected OOD frequency and the downstream cost of incorrect predictions.

9.3 Monitoring for dataset drift

Once deployed, the system should monitor for drift using feature statistics, confidence trends, or embedding distances. Monitoring helps distinguish gradual changes from sudden shifts and can inform when retraining or recalibration is needed.

9.4 Human-in-the-loop workflows

Human-in-the-loop workflows can correct for residual OOD cases by labeling flagged samples or resolving ambiguous outputs. Effective workflows include mechanisms to ensure that human feedback is integrated back into the training or calibration pipeline.

9.5 System-level safety and logging

System-level safety requires auditability: logging input metadata, model scores, and decisions. Logging supports investigations into OOD events, helps diagnose calibration problems, and enables compliance with organizational requirements for risk management.

10 Future Directions

Research continues to address limitations of current OOD methods, including reliance on assumptions about shift type and the difficulty of uncertainty calibration under complex mismatches.

10.1 Better uncertainty estimation under shift

Improving uncertainty estimation aims to produce reliable uncertainty signals when the model faces distributions that differ substantially from training. This includes exploring more robust uncertainty formulations and better calibrating predictive distributions under mismatch.

10.2 OOD with fewer assumptions

Many methods assume access to OOD-like auxiliary data or particular shift structures. Future work seeks techniques that require fewer assumptions, such as approaches that rely on general properties of representations or that can adapt online without extensive OOD supervision.

10.3 Continual and online OOD handling

Continual learning frameworks may integrate new evidence as distributions evolve, updating both decision boundaries and OOD detectors. Online handling also raises questions about preventing catastrophic forgetting while maintaining reliable rejection behavior.

10.4 Benchmarks that reflect real-world shifts

Realistic benchmarks increasingly attempt to model the diversity of real deployment changes, including correlated nuisance factors and multi-stage pipelines. Improved benchmark design helps ensure that gains on detection metrics translate into practical robustness and safer behavior.