1 Foundations of variance decomposition

1.1 Concept of partitioning total variance

Variance decomposition refers to methods that rewrite the variability of a random variable (or a forecast error) as the sum of distinct components. Rather than reporting a single variance number, the approach allocates portions of that total variability to interpretable sources—such as group differences, systematic versus random effects, or contributions from specific model terms. In practical terms, decomposition is often used to answer questions like “How much of the variability is explained by factor A versus the remainder?” or “Which shocks contribute most to uncertainty?”

A central idea is that multiple explanatory influences may act simultaneously. Decomposition frameworks therefore define a rule for dividing the total variance into additive pieces that correspond to chosen mechanisms (model terms, latent factors, or perturbations). The interpretability of the results depends strongly on the modeling and the partitioning conventions used.

1.2 Key assumptions and notation

Most variance decompositions rely on a probability model that specifies how a target variable relates to predictors, latent states, or shocks. A standard starting point is a scalar random variable \(Y\) with finite second moment. The total variance is \[ \mathrm{Var}(Y)=\mathbb{E}\big[(Y-\mathbb{E}Y)^2\big]. \] When a decomposition is conditional on predictors or model structure, notation frequently includes conditional expectations, such as \(\mathbb{E}[Y\mid X]\) and \(\mathrm{Var}(Y\mid X)\).

To create additive components, many formulations assume linearity and/or orthogonality among the pieces being separated (as in projection methods). Others rely on independence or uncorrelatedness among sources (as in certain shock-based models). In sensitivity-analysis settings, assumptions are often phrased in terms of the joint distribution of inputs (e.g., independence among model inputs) to support clean attribution to main and interaction effects.

1.3 Additivity and “explained vs. unexplained” variation

A common decomposition separates variability into predictable and unpredictable parts: \[ \mathrm{Var}(Y)=\mathrm{Var}(\mathbb{E}[Y\mid X])+\mathbb{E}[\mathrm{Var}(Y\mid X)]. \] The first term represents the variation in the conditional mean as \(X\) changes (often interpreted as “explained”), while the second term represents the average conditional variance left after accounting for \(X\) (often interpreted as “unexplained” or irreducible uncertainty under the assumed model).

In many applied contexts, the “explained” portion is further subdivided. For example, in a linear model with multiple predictors, explained variance can be broken into contributions from individual predictors and their interactions, often using orthogonal projections or sequential regressions. For time-series or state-space models, analogous ideas are applied to forecast error, treating parts of uncertainty as attributable to known shocks or to latent-state uncertainty.

1.4 Relationship to covariance and correlation

Variance decomposition is tightly linked to covariance structure. When two components contribute to \(Y\), their covariance determines whether the total variance is simply the sum of marginal variances or whether cross-terms appear: \[ \mathrm{Var}(A+B)=\mathrm{Var}(A)+\mathrm{Var}(B)+2\,\mathrm{Cov}(A,B). \] This is why many decomposition frameworks enforce orthogonality: if components are uncorrelated (or orthogonal in a projection sense), covariance terms vanish, enabling clean additivity. Correlation also influences interpretation: a component may appear important in isolation but become less decisive when combined with another component that covaries with it.

Understanding this relationship clarifies why decomposition results can be model-dependent. Changing basis functions, ordering, or constraints can alter whether components are uncorrelated, which in turn affects the share attributed to each source.

2 Variance decomposition in classical statistics

2.1 ANOVA decomposition

2.1.1 Between-group vs. within-group variance

ANOVA-style variance decomposition partitions total variability into differences among group means and dispersion within groups. Suppose observations are grouped by a categorical factor with group means \(\bar{Y}_g\) and overall mean \(\bar{Y}\). The total sum of squares is split into between-group and within-group components:

  • Between-group variance reflects how far each group mean deviates from the overall mean.
  • Within-group variance reflects variability of observations around their respective group mean.

In this context, the decomposition quantifies how much of the observed spread is associated with the grouping factor rather than random fluctuation inside groups. It is particularly natural when the factor defines meaningful categories and when groups are treated as exchangeable replicates.

2.1.2 Mean squares, sums of squares, and degrees of freedom

The ANOVA decomposition is often expressed through sums of squares (SS) and mean squares (MS). The total sum of squares can be written as \[ SS_T=\sum_{i}(Y_i-\bar{Y})^2, \] and then split into \(SS_B\) (between) and \(SS_W\) (within). Mean squares divide each sum by its corresponding degrees of freedom, yielding \(MS_B\) and \(MS_W\). These quantities enter classical F-tests via the ratio \(MS_B/MS_W\).

Degrees of freedom matter because the decomposition components have different sample support: between-group effects are estimated using fewer independent comparisons than within-group effects. Consequently, scaling by degrees of freedom is essential for both inference and for comparing variance contributions across factors.

2.2 Coefficients of determination and variance explained

2.2.1 R-squared and interpretation via variance terms

In regression, \(R^2\) measures the proportion of variance explained by the model, usually defined as \[ R^2 = 1 - \frac{SS_{\text{res}}}{SS_T}, \] where \(SS_{\text{res}}\) is the residual sum of squares. Under standard least squares assumptions, \(SS_T\) is decomposed into explained and residual components, making \(R^2\) interpretable as a variance share at the fitted level.

Variance-based interpretations of \(R^2\) connect to decomposition directly: explained variance corresponds to variability in the fitted values relative to the overall mean, while residual variance corresponds to variability around fitted values. This framing helps distinguish model fit quality from predictive performance, since \(R^2\) is computed on the same data used to estimate parameters unless cross-validation is used.

2.2.2 Extensions to adjusted and alternative fit metrics

When models include different numbers of predictors or different design structures, raw \(R^2\) can be difficult to compare. Adjusted metrics correct for degrees of freedom, effectively rescaling the explained versus residual variance shares to account for model complexity. Other criteria (such as information-theoretic measures or likelihood-based metrics) do not always map to variance proportions, but they can still be related to decompositions when expressed through variance components under specific distributional assumptions.

In many applications, the variance decomposition perspective encourages reporting not only a single fit statistic but also the underlying variance shares—especially when the goal is interpretability rather than purely predictive accuracy.

3 Decomposition in linear models

3.1 Decomposing fitted values and residuals

In linear least squares, the fitted response can be decomposed from the observed response into a systematic part and a residual part. Denote the fitted values by \(\hat{Y}\) and residuals by \(e = Y - \hat{Y}\). Variance can then be viewed through the lens of the sums of squares: \[ SS_T = SS_{\text{fit}} + SS_{\text{res}}. \] Here \(SS_{\text{fit}}\) captures variability accounted for by the model subspace, whereas \(SS_{\text{res}}\) captures leftover variability orthogonal to that subspace. This provides a concrete variance decomposition tied to the geometry of least squares.

The interpretability is enhanced because the fitted component corresponds to the model-implied conditional mean, while residuals correspond to discrepancies not captured by the chosen linear structure.

3.2 Orthogonality and projection viewpoint

Least squares has a projection interpretation: the fitted values are the orthogonal projection of \(Y\) onto the column space of the design matrix. Because the residuals are orthogonal to the fitted component, covariance terms disappear in the variance decomposition. This orthogonality yields clean additive variance shares: \[

\|Y\|^2 = \|\hat{Y}\|^2 + \|e\|^2

\] (up to scaling depending on conventions). The same idea extends to decompositions involving subspaces for subsets of predictors.

This projection view also clarifies why variance decomposition in linear models depends on how predictors are encoded. Changing the basis (while spanning the same subspace) preserves the total explained variance but can redistribute contributions among components depending on the chosen decomposition scheme.

3.3 Partial and sequential (type I/II/III) contributions

When multiple predictors are present, there are multiple legitimate ways to attribute explained variance to specific terms. One common approach compares models with and without a given predictor block:

  • Partial contributions quantify incremental explained variance given that other predictors are already included.
  • Sequential contributions depend on the order in which predictors are entered, reflecting incremental fitting along a particular path.

Classical ANOVA terminology describes different “types” of sums of squares for factor effects (often associated with type I, II, or III conventions). While they agree in some special designs, they can differ in the presence of multicollinearity or unbalanced data. The key point is that each type corresponds to a specific comparison model and therefore reflects a different notion of “unique” versus “shared” variance among correlated predictors.

3.4 Interactions and variance attributable to cross-terms

When models include interaction terms, variance attribution must account for cross-structure. An interaction term represents variability in how the effect of one predictor changes with another. In decomposition terms, interaction contributions can be substantial even when main effects are modest, especially when the systematic part of the response is nonlinear in predictors.

In orthogonalized bases, interaction and main-effect components may be separated more cleanly. In raw factor coding, however, main and interaction contributions can be entangled; different decomposition conventions redistribute shared variance. As a result, careful coding (e.g., centering and orthogonal contrasts) can improve interpretability of variance shares for interaction terms.

4 Decomposition in time-series and state-space models

4.1 Innovations and forecast error variance

In time-series models, a natural target for variance decomposition is the forecast error variance. Innovations (also called shocks or one-step-ahead prediction errors) are treated as fundamental unpredictable components generated over time. Forecast uncertainty at horizon \(h\) can be expressed in terms of how present and past innovations propagate through the model.

This leads to variance shares across time lags or across innovation sources. For example, uncertainty at a given forecast horizon can be decomposed into contributions from shocks occurring at different times, often enabling attribution such as “recent shocks explain most of the \(h\)-step uncertainty.”

4.2 Role of latent states

State-space models introduce latent states to describe dynamics not directly observed. Forecast error variance then depends both on process noise driving the states and on observation noise affecting what is measured. Variance decomposition must therefore include uncertainty in the latent state estimates, typically tracked through filtering and prediction recursions.

Latent-state uncertainty can dominate long-horizon forecast variance, particularly when the observation signal is weak. Decompositions in this setting clarify whether forecast uncertainty primarily stems from intrinsic system randomness or from inability to infer the hidden state accurately.

4.3 Error decomposition across horizons

Forecast error variance grows (or evolves) with horizon. A decomposition framework often allocates uncertainty at each horizon to distinct sources such as:

  • the initial condition uncertainty carried into the forecast,
  • innovations at intermediate times,
  • measurement noise affecting updates.

Because propagation of shocks is horizon-dependent, the importance ranking of sources may change as \(h\) increases. This makes horizon-specific variance decomposition valuable for understanding when certain shocks matter most for longer-term predictions.

4.4 Practical computation and simulation strategies

In many models, closed-form expressions for variance decomposition are available only for limited classes (e.g., linear Gaussian systems). Otherwise, computation uses:

  • analytic recursions for Kalman-filter-based settings,
  • Monte Carlo simulation of shocks and propagation through the model,
  • numerical evaluation of covariance matrices of forecasts and errors.

Simulation-based methods produce estimates of variance shares but require attention to sampling variability. Ensuring convergence often involves increasing the number of simulated trajectories, checking stability of estimated components, and verifying that numerical schemes preserve key properties like positive semidefiniteness of covariance estimates.

5 Frequency-domain and dynamic contributions

5.1 Spectral decomposition concepts

Some variance decomposition methods operate in the frequency domain by linking variance to spectral density. For stationary processes, variance can be expressed as an integral (or sum) of the power across frequencies. In this view, variability is not only attributed to time-domain shocks but also to cyclical or oscillatory components at particular frequencies.

This is useful when the signal of interest has distinct periodicities or when different mechanisms produce variability concentrated at different spectral bands.

5.2 Variance allocation across cycles and frequencies

Frequency-based decomposition partitions total variance into contributions from ranges of frequencies. For example, variability attributable to “slow movements” can be separated from variability attributable to “rapid fluctuations.” Such allocations can be computed via band-pass filtering, spectral integration, or frequency-domain representations of dynamic models.

The interpretability depends on the spectral assumptions (e.g., stationarity) and on how frequency bands are chosen. When bands are well motivated by the data-generating process or scientific context, frequency allocation becomes a natural complement to time-domain decomposition.

5.3 Impulse-response interpretations (general framework)

In many dynamic models, impulse-response functions describe how a shock affects the system over time. Variance decomposition connects to impulse responses by relating the magnitude of propagated responses to contributions in forecast error variance. Although impulse-response analysis is often presented in terms of trajectories, it can be summarized in terms of variance explained by shocks by aggregating impulse-response effects appropriately over time or across horizons.

The general framework emphasizes that impulse-response and variance decomposition are two perspectives on shock propagation: one shows paths of effects, the other quantifies their contribution to uncertainty.

5.4 Connecting time- and frequency-domain variance views

Time- and frequency-domain decompositions can be linked through transforms and through the equivalence between autocovariance structure and spectral density. Under suitable conditions, a time-domain variance decomposition based on shocks can correspond to a frequency allocation based on spectral content, even though the computations and interpretive metaphors differ.

Connecting these views helps analysts choose the most appropriate representation for their questions—whether they seek attribution by event timing (time) or by oscillatory behavior (frequency).

6 Structural and causal-style variance decomposition (model-based)

6.1 Sensitivity and contribution under model structure

Model-based variance decomposition can be used to quantify how changes in assumed mechanisms alter the variability of outcomes. In structural models, different structural components (parameters, functional forms, or blocks of equations) can be varied to see how much of output variance they generate or suppress.

This style of decomposition is closely tied to sensitivity analysis: contributions are not purely statistical properties of observed data alone, but depend on the structural hypotheses encoded by the model.

6.2 Counterfactual-style variance partitioning (general)

Counterfactual approaches partition uncertainty by considering alternative hypothetical scenarios for parts of the data-generating system. The variance of an outcome under the model can be attributed to variability in components such as:

  • one subset of exogenous disturbances,
  • one equation block,
  • one structural submechanism.

General counterfactual variance partitioning relies on defining which components are held fixed or replaced, and then computing how the outcome distribution changes across those counterfactual scenarios. The resulting decomposition shares the interpretive flavor of “source attribution,” but it remains conditional on the structural assumptions and on the counterfactual intervention definitions.

6.3 Identifiability and interpretation limits

A key limitation is that different structural models can reproduce similar observational patterns, making it difficult to uniquely identify the “true” contributions of structural components. Even when variance decomposition is computed precisely for a fitted model, its causal interpretation may be fragile if the model is not identifiable.

This limitation is not specific to variance decomposition; it reflects a broader issue in structural modeling. Nonetheless, decomposition amplifies interpretive stakes because it produces rankings and shares that may be taken as substantive “drivers.” Readers should therefore distinguish descriptive variance attribution from structurally identified causal contributions.

6.4 Model misspecification effects on contributions

If the model is misspecified—through incorrect functional forms, omitted variables, or wrong noise structure—the estimated variance shares can be biased. Because decomposition expresses variance through model-implied quantities (conditional means, propagated shocks, or counterfactual distributions), misspecification can distort both the total explained variability and how it is apportioned.

Robust practice involves checking diagnostics for model adequacy, assessing sensitivity to alternative specifications, and comparing decompositions across plausible modeling choices.

7 Global sensitivity analysis via variance-based methods

7.1 Variance-based sensitivity indices

Variance-based global sensitivity analysis quantifies how uncertainty in inputs contributes to uncertainty in outputs. Let \(Y=f(X_1,\dots,X_k)\) where the inputs \(X\) are random according to some distribution. The total output variance is decomposed into components associated with subsets of inputs.

The canonical indices measure:

  • how much of \(\mathrm{Var}(Y)\) is attributable to varying a specific input (main effects),
  • how much arises from interactions among inputs (interaction effects),
  • how much remains when an input and all its interactions are considered (total effects).

These indices provide a unified framework for ranking drivers under a specified probabilistic input model.

7.2 Main effects and interaction effects

A main-effect index isolates the portion of output variance due to one input alone, averaged over the uncertainty in all other inputs. Interaction effects capture the additional variance that cannot be explained by main effects alone, reflecting scenarios where the impact of one input depends on the value of another.

In practical workflows, interaction indices are often computed indirectly (e.g., via complementary totals) because the number of subset combinations can grow quickly with input dimension.

7.3 Total-effect indices and interpretation

Total-effect indices measure the contribution of an input including all interactions with other inputs. Conceptually, they answer: “How much of the output uncertainty disappears if we fix this input (and leave the rest random)?” A large total-effect index indicates that the input is important either through its standalone effect or through higher-order interactions.

These indices are especially useful for screening when many inputs are candidate drivers, because they summarize complex interaction structure with a single number per input.

7.4 Sampling, estimation, and convergence considerations

Variance-based sensitivity indices are typically estimated using Monte Carlo designs. Common approaches use structured sampling plans (such as difference-based estimators) that reuse model evaluations efficiently to reduce variance of the estimates.

Convergence depends on:

  • the number of samples,
  • the complexity of the function \(f\),
  • the presence of strong nonlinearities or rare-event behavior.

Uncertainty quantification can be obtained via bootstrap resampling or repeated experiment designs. Analysts also need to ensure that the chosen input distribution matches the intended interpretation of uncertainty.

8 Estimation, computation, and diagnostics

8.1 Numerical methods and Monte Carlo estimation

Many decomposition quantities can be expressed through expectations involving conditional variances, covariance matrices, or model-implied propagation. When analytic computation is infeasible, Monte Carlo methods approximate these expectations by simulating input draws and re-evaluating the model.

A key computational concern is how to compute not only total variance but also the additive components. Estimators should preserve the intended decomposition identity (up to numerical tolerance), otherwise shares can fail to sum properly due to estimation error.

8.2 Bootstrap and uncertainty quantification

Because decomposition shares are estimated quantities, they inherit sampling variability from the underlying data. Bootstrap methods resample the dataset (or residuals, in regression contexts) to build an empirical distribution for each variance component.

Reporting confidence intervals for variance shares helps prevent overinterpretation of small differences between components. When bootstrap intervals are wide, the decomposition may be informative only qualitatively, not quantitatively.

8.3 Diagnostics for stability and robustness

Stability diagnostics address whether variance shares persist under:

  • alternative resampling splits (cross-validation),
  • different random seeds in simulation-based computations,
  • small perturbations to model specification (e.g., alternative feature encoding).

Robustness checks also involve verifying assumptions such as independence (in sensitivity analysis), stationarity (in spectral/time-series decompositions), and adequate coverage (in model validation). When diagnostics reveal instability, variance decomposition should be treated as suggestive rather than definitive.

8.4 Common pitfalls (leakage, dependence, and scaling)

Several pitfalls recur across applications:

  • Leakage: using information improperly during estimation (e.g., target leakage in predictive models) can inflate explained variance shares.
  • Dependence: when inputs or predictors are dependent, attribution to “unique” sources becomes ambiguous unless the decomposition framework accounts for dependence structure.
  • Scaling: variance shares are sensitive to the scaling of variables; inconsistent units or transformations can change computed contributions. Transformations that alter variance can also alter the interpretability of “explained” proportions.

Careful preprocessing, consistent model evaluation protocols, and clear description of assumptions are essential for credible variance decomposition results.

9 Applications and interpretation

9.1 Factor importance in predictive modeling

In predictive modeling, variance decomposition can be used to assess which predictors drive variability in the target. For linear models, this aligns with explained variance by term or by sequential addition. For more general models, variance-based sensitivity indices can identify inputs that dominate output uncertainty under an assumed input distribution.

Interpretation should connect variance shares to the modeling goal: a factor may explain variance in-sample but not generalize; conversely, a factor with a small variance share in-sample might still improve out-of-sample calibration.

9.2 Explaining uncertainty in forecasts

Forecasting applications benefit from decomposing forecast error variance into contributions from innovations, latent-state uncertainty, and measurement noise. This provides a structured explanation for why predictions become uncertain over time, and which mechanisms (shocks at certain periods, state uncertainty, or observation error) are most responsible.

Such analyses are useful for deciding where additional data collection or model refinement would be most effective—for example, improving measurement noise characteristics or refining state estimation.

9.3 Comparing competing models by variance shares

Variance decomposition can support model comparison by examining whether different models attribute uncertainty to similar sources. Two models can yield comparable total forecast error but disagree on which components explain it. Comparing variance shares helps diagnose whether the model’s internal mechanism resembles an expected driver structure.

Model comparison should also incorporate uncertainty around estimated shares, since differences may reflect estimation error rather than substantive divergence.

9.4 Communicating results to non-technical audiences

Communicating variance decomposition effectively often requires translating variance shares into plain language. A typical strategy is to describe the decomposition as “where the uncertainty comes from,” using interpretable categories like systematic effects, noise, or shock sources.

Visualizations such as stacked bars (additive components), horizon profiles (how shares evolve with time), or frequency bands (slow versus fast variation) can help audiences understand contributions without needing to follow the mathematics.

10.1 Other decomposition frameworks (e.g., PCA/latent-factor viewpoints)

Variance decomposition ideas appear in principal component analysis and latent-factor modeling. PCA decomposes total variance into orthogonal components ordered by explained variance, producing interpretable variance shares across principal directions. In latent-factor models, variance allocation can be tied to factor loadings and idiosyncratic components.

These methods share the theme of replacing a single variance measure with a structured set of components, though the interpretive mapping differs: PCA components are basis directions, while regression or sensitivity components correspond more directly to model terms or input subsets.

Although bias-variance decomposition in machine learning is distinct from variance decomposition of a random variable’s variability, both are often conceptually related. The bias-variance tradeoff describes expected prediction error as contributions from systematic error and variance of the estimator across datasets. Variance decomposition of outcomes describes variability arising from stochasticity and uncertainty sources. In practice, analysts sometimes combine these perspectives when explaining why a model is unstable, overfits noise, or fails to capture systematic structure.

10.3 Decomposition under constraints and regularization

Regularization introduces constraints that can change projection geometry and therefore redistribute variance across components. In constrained regression or models with shrinkage, variance shares may correspond to posterior or penalized notions of explained variability rather than the classical unpenalized variance explained.

When decompositions are performed after regularization, it is often important to clarify whether components reflect:

  • explained variance in the fitted model,
  • effective degrees of freedom,
  • or contributions in a probabilistic (e.g., Bayesian) posterior predictive sense.

10.4 Hierarchical and multilevel variance structures

In multilevel models, variance decomposition often separates variability across grouping levels (e.g., between groups versus within groups) and can also include multiple random-effect components. This generalizes the ANOVA intuition to models where effects are nested and correlated.

Hierarchical decompositions support interpreting where variability resides in structured data, such as schools within regions or sensors within devices. Estimation typically uses likelihood methods or approximate Bayesian inference, and the resulting variance components can be summarized as shares of total variability at each level.