1 Concept and Motivation

1.1 What “discrepancy” means in modeling

Model discrepancy is the systematic mismatch between a model’s predicted outputs and the real-world responses it is intended to describe. Unlike errors that average out or vary unpredictably, discrepancy reflects persistent departures caused by limitations in the model form, missing mechanisms, or simplified assumptions. In practice, discrepancy is treated as an explicit component of the data-generating process so that inference and forecasting remain honest about what the model can and cannot capture.

1.2 Why discrepancy arises despite good calibration

Discrepancies can remain even after a model is calibrated using available observations. Calibration can align parameters and scaling factors with data, but it cannot repair structural omissions—for example, an omitted physical effect, an overly simplified constitutive relationship, or a numerical approximation that fails in certain regimes. Additionally, model inputs used in the field may differ from those assumed during modeling (e.g., unrecorded latent drivers, sensor processing differences, or boundary-condition variations), leading to persistent gaps that look like “bias” under the chosen model form.

1.3 Discrepancy vs. random error

Random error is typically modeled as noise that is approximately independent of the model’s mean prediction, with properties such as constant variance (or a specified variance pattern) across inputs. Discrepancy, by contrast, is structured: it tends to follow systematic trends with covariates, regimes, or operating conditions. In many applications, the two sources can be separated only partially; nevertheless, treating discrepancy as structured leads to more credible uncertainty estimates and often improves predictive performance.

1.4 Discrepancy vs. parameter uncertainty

Parameter uncertainty captures uncertainty in the values of model parameters given limited data and imperfect knowledge. Discrepancy, however, concerns uncertainty about the adequacy of the model form itself. A model can have sharply estimated parameters yet still generate incorrect functional behavior if key relationships are missing. Inference procedures therefore often distinguish between “learning the parameters” and “learning how reality deviates from the model,” because these two uncertainties play different roles in both prediction and diagnostics.

2 Mathematical Formulations

2.1 Additive discrepancy models

An additive discrepancy model represents the observation as the sum of a deterministic model prediction and an extra term accounting for mismatch. This formulation is common because it is intuitive and integrates naturally with regression-like likelihoods.

2.1.1 Constant and linear discrepancy components

The simplest additive discrepancy terms are constant offsets or linear adjustments in one or more predictors. A constant discrepancy captures systematic shifts across the entire input range, while linear components allow for gradual trends not represented by the base model. Though limited, these forms can be useful as baseline corrections and as a diagnostic starting point.

2.1.2 Heteroscedastic discrepancy formulations

In some settings, mismatch interacts with the operating regime, producing larger gaps where the model is less reliable. Heteroscedastic discrepancy allows the discrepancy variance (or discrepancy scale) to vary with inputs. This can represent different levels of model inadequacy across conditions without forcing the noise model to absorb all structure.

2.2 Multiplicative and transformed discrepancy models

Multiplicative models represent mismatch as a scaling of the model output. This is often appropriate when errors grow proportionally with the magnitude of the prediction or when the underlying quantity is naturally positive.

2.2.1 Log-scale discrepancy representations

A common transformed approach is to model discrepancy on a logarithmic scale. For positive outputs, transforming observations and predictions into log space can turn multiplicative deviations into approximately additive ones. The result is a likelihood that is more aligned with proportional error patterns and can stabilize variance.

2.3 Latent-function discrepancy

Latent-function discrepancy treats discrepancy as an unknown function of the inputs, learned from data. Instead of fixing a parametric adjustment, the method lets discrepancy vary flexibly while remaining constrained by assumptions such as smoothness or boundedness.

2.3.1 Functional discrepancy in regression frameworks

In regression settings, the discrepancy function can be introduced as an extra latent term added to the mean function. The base model captures the mechanistic or previously known part, while the discrepancy function absorbs deviations. This separation is useful when the base model represents known structure, and the discrepancy represents residual inadequacy.

2.4 Discrepancy in Bayesian modeling

Bayesian formulations incorporate discrepancy through prior distributions on the discrepancy terms and the remaining model parameters, enabling joint inference and coherent uncertainty propagation.

2.4.1 Prior specification for discrepancy terms

Priors encode assumptions about discrepancy scale, smoothness, and correlation across inputs. Overly permissive priors may allow discrepancy to mimic parameter effects, while overly restrictive priors can prevent the model from correcting systematic shortcomings. Good practice typically includes sensitivity checks or principled prior elicitation tied to physical intuition or empirical residual patterns.

2.4.2 Identifiability considerations

Discrepancy terms can be non-identifiable when they can trade off with parameters in the base model. For example, a slowly varying discrepancy function may replicate the effect of adjusting a calibration parameter, leaving posterior uncertainty concentrated in a combined direction. Identifiability depends on data coverage, model structure, and the strength and form of discrepancy priors.

3 Statistical Inference Implications

3.1 Effects on parameter estimation

Including discrepancy can change parameter estimates because some structure previously attributed to parameters may instead be explained by the mismatch term. This can reduce misleading parameter learning from data where the base model is structurally inadequate. In return, parameters may become less tightly constrained, reflecting the reality that part of the signal is not governed by the calibrated model alone.

3.2 Effects on prediction and uncertainty quantification

Discrepancy-aware prediction typically yields uncertainty that is more realistic, especially when the model’s structural limitations matter for extrapolation or for regions not well covered by calibration data. Predictive intervals broaden when discrepancy is plausibly large, preventing overconfident forecasts. In-sample fit can also improve because the model is allowed to match systematic patterns it previously forced into parameter adjustments.

3.3 Regularization and overfitting trade-offs

Flexible discrepancy models can overfit noise if allowed too much freedom. Regularization arises naturally through priors, smoothness penalties, kernel structures, or constraint choices that limit discrepancy complexity. Balancing fidelity and parsimony is central: discrepancy should capture persistent mismatch, not random fluctuation.

3.4 Learning discrepancy from limited data

When observations are sparse, discrepancy learning is constrained by prior assumptions and by how informative the data are about input-to-output structure. Limited data can make discrepancy estimates uncertain or overly influenced by the chosen functional form. Practical approaches often emphasize conservative discrepancy representations, careful prior calibration, and validation on held-out sets to ensure that learned mismatch generalizes.

4.1 Using Gaussian processes for discrepancy

Gaussian process (GP) discrepancy models represent discrepancy as a random function with a specified covariance structure. GPs provide flexible yet controlled functional behavior, typically governed by assumptions about smoothness, length scales, and amplitude.

4.1.1 Kernel choices and smoothness assumptions

Kernel selection determines what patterns discrepancy can express. Smooth kernels allow gradual variation across inputs, while kernels with different length-scale parameters permit sharper changes. Composite kernels can encode structured behavior such as periodic components or varying smoothness across regimes. Kernel design therefore directly shapes how discrepancy is expected to behave physically or empirically.

4.1.2 Hyperparameter learning strategies

GP hyperparameters such as length scales and variance can be learned by maximizing marginal likelihood, sampling in a fully Bayesian scheme, or using empirical Bayes approximations. Each approach trades off computational cost against uncertainty fidelity. Poorly estimated hyperparameters can lead to either underfitting (discrepancy too rigid) or overfitting (discrepancy too flexible), so diagnostics and sensitivity analysis are commonly used.

4.2 Emulators and discrepancy separation

In computer experiments, a fast surrogate (emulator) can approximate expensive simulator outputs. Discrepancy-aware frameworks often combine an emulator for the simulator and a separate discrepancy term for the gap between simulator and reality.

4.2.1 Calibration with surrogate models

When direct simulator runs are costly, the base simulator response is emulated from a limited set of runs. Calibration then proceeds by fitting emulator uncertainty alongside discrepancy. This separation clarifies sources of uncertainty: one from limited surrogate accuracy and another from structural mismatch between the simulator and the physical system.

4.3 Non-Gaussian extensions (overview)

Standard GP models rely on Gaussian assumptions for discrepancy distributions and sometimes for observation noise.

4.3.1 Heavy-tailed discrepancy terms

Heavy-tailed formulations can increase robustness when discrepancies occasionally produce outliers or when residuals show larger-than-Gaussian variability. These approaches can reduce the influence of anomalous points on learned discrepancy structure and lead to more stable predictions in the presence of atypical behavior.

5 Model Calibration and Validation

5.1 Calibration with discrepancy-aware likelihoods

Calibration often replaces a naive likelihood based solely on model outputs with a likelihood that accounts for discrepancy terms. The result is a joint fit that simultaneously estimates model parameters, discrepancy structure (or its hyperparameters), and noise characteristics. This helps prevent the optimizer or posterior sampler from forcing parameters to compensate for structural misfit.

5.2 Posterior predictive checks including discrepancy

Posterior predictive checks assess whether replicated data drawn from the fitted model resemble observed data. When discrepancy is included, checks can target both overall fit and regime-specific behavior, such as whether residual patterns persist or whether predictive distributions capture systematic deviations.

5.3 Cross-validation for discrepancy modeling

Cross-validation evaluates generalization by fitting the discrepancy model on training data and testing predictive accuracy on held-out inputs or observations. This is particularly important for flexible discrepancy representations, where fit quality might be high in-sample but degrade outside the training region. Metrics can be based on predictive means, interval coverage, or proper scoring rules.

5.4 Assessing model inadequacy without controversy

Model inadequacy assessment focuses on statistical evidence rather than interpretive debates. In discrepancy-aware workflows, the emphasis is on empirical mismatch patterns—such as consistent residual structure, poor coverage under naive uncertainty, or systematic errors varying across inputs. These diagnostics support transparent reporting of limitations and motivate the chosen discrepancy structure.

6 Identifiability and Practical Diagnostics

6.1 When discrepancy and bias are confounded

Confounding occurs when the discrepancy term can mimic bias-like effects within the base model. For example, a discrepancy function that includes a constant or near-linear component may absorb what would otherwise be interpreted as a calibration parameter offset or measurement bias. Identifying the split requires informative data patterns, carefully constrained discrepancy priors, or supplemental calibration information.

6.2 Sensitivity analysis for discrepancy priors

Because priors influence discrepancy behavior, sensitivity analysis checks how posterior conclusions change under alternative prior assumptions. Practical variations may include modifying correlation length scales, amplitude bounds, or kernel smoothness. Large changes indicate weak identifiability or insufficient data support for learning discrepancy; small changes suggest robust inference.

6.3 Influence of data coverage

Data coverage strongly affects learnability of discrepancy. Dense observations in a narrow input region may yield accurate discrepancy estimates there but remain uncertain elsewhere, leading to broad predictive intervals in extrapolation. Incorporating design-of-experiments principles and ensuring representative sampling can reduce uncertainty in discrepancy structure.

6.4 Diagnostic plots and residual structure

Residual analysis remains valuable even in sophisticated Bayesian settings. Diagnostic plots can reveal whether residuals exhibit systematic trends, heteroscedasticity, or correlated structure. When discrepancy is correctly modeled, residuals should behave more like noise under the chosen likelihood, with diminished structure across inputs and improved calibration of predictive uncertainty.

7 Engineering Workflow and Best Practices

7.1 Choosing a discrepancy representation

A discrepancy representation should match the expected nature of mismatch. If errors are mostly constant across conditions, low-dimensional adjustments may suffice. If mismatch varies smoothly with inputs, latent-function approaches like GPs are more appropriate. For proportional errors, multiplicative or log-scale representations can be more stable than additive forms.

7.2 Designing experiments to learn discrepancy

Learning discrepancy benefits from input diversity that excites the parts of the system where the base model is likely to fail. Carefully designed experiments can increase identifiability by providing data across regimes, boundaries, and operating ranges. In simulator settings, selecting training points to cover the design space can reduce the tendency for discrepancy to remain underdetermined.

7.3 Reporting uncertainty sources transparently

Good reporting separates uncertainty into components such as parameter uncertainty, discrepancy uncertainty, and measurement noise. Transparency helps readers understand whether predictive uncertainty is dominated by limited calibration data or by structural mismatch. It also clarifies which assumptions are driving the width and shape of predictive intervals.

7.4 Computational considerations and scalability

Discrepancy models, especially GP-based ones, can be computationally demanding due to covariance matrix operations. Scalability can be improved using sparse GP approximations, inducing point methods, or reduced-rank representations. For large data sets, careful choice of inference algorithm and approximation strategy is often necessary to make discrepancy modeling practical.

8 Applications and Use Cases

8.1 Computer experiments and scientific simulation

In scientific computing, discrepancy accounts for the gap between a simplified simulator and a real physical system. It is particularly relevant when simulators capture broad behavior but miss fine-scale effects, numerical artifacts, or unmodeled boundary conditions. Discrepancy-aware calibration supports more credible scientific conclusions by acknowledging that simulation is an imperfect representation of nature.

8.2 Data integration across sensors or instruments

When measurements come from multiple instruments, systematic differences can appear as discrepancy, especially if instruments have different biases, resolution limits, or processing pipelines. Discrepancy modeling can help harmonize these sources by separating model-based predictions from instrument- or context-dependent deviations.

8.3 Forecasting with structurally simplified models

Forecasting often relies on models that simplify complex dynamics for tractability. Discrepancy-aware formulations can compensate for those simplifications by learning systematic gaps from historical data. This can improve uncertainty calibration, particularly when forecasts extend into conditions not heavily represented in training.

8.4 Surrogate modeling in high-cost systems

In high-cost settings, such as engineering design or expensive laboratory experiments, surrogate models approximate expensive evaluations. Discrepancy terms can then capture both the mismatch between the surrogate’s underlying simulator and the real system, and the remaining structural departures not explained by the surrogate alone. This combination enables efficient calibration while maintaining uncertainty that reflects both emulation and model form limitations.