Long-run variance

1 Definition and Intuition

Long-run variance is a measure of how variability accumulates over long horizons for a time-indexed random process. While an ordinary variance quantifies spread at a single time point, long-run variance accounts for serial dependence: if outcomes tend to remain above or below their mean for extended periods, then averaging or summing over many observations retains uncertainty that is larger than what would be predicted by treating observations as independent.

1.1 Long-run variance as variance growth of partial sums

Consider a stationary process \(\{X_t\}\) with \(\mathbb{E}[X_t]=\mu\). Define the partial sum \[ S_n=\sum_{t=1}^n (X_t-\mu). \] If \(\operatorname{Var}(S_n)\) grows approximately linearly with \(n\) for large \(n\), then the coefficient of that linear growth is the long-run variance, commonly written as \(\Omega\). Intuitively, persistent correlations inflate the variance of aggregated quantities, so \(\Omega\) summarizes the “effective” variability per time period once dependence is taken into account.

1.2 Relationship to limiting distribution of averages

The same quantity governs the asymptotic uncertainty of sample averages. Under suitable conditions, \[ \sqrt{n}\,(\bar X_n-\mu) \;=\; \frac{1}{\sqrt{n}} \sum_{t=1}^n (X_t-\mu) \] often converges in distribution to a normal law with variance \(\Omega\). This means that \(\Omega\) plays the role that \(\operatorname{Var}(X_t)\) would have under independence, but adjusted for temporal correlation.

1.3 Contrast with instantaneous (short-run) variance

Instantaneous variance is \(\gamma(0)=\operatorname{Var}(X_t)\). Long-run variance equals \(\gamma(0)\) plus the cumulative contribution of autocovariances across all lags. When correlations decay quickly, \(\Omega\) is close to \(\gamma(0)\). When correlations persist, \(\Omega\) can be substantially larger (or, with negative dependence patterns, smaller than \(\gamma(0)\)).

2 Formal Definitions

2.1 Covariance-series representation

For a weakly stationary process, define the autocovariance function \[ \gamma(k)=\operatorname{Cov}(X_t, X_{t+k}). \] A standard representation of long-run variance is \[ \Omega \;=\; \sum_{k=-\infty}^{\infty}\gamma(k), \] which can also be written as \[ \Omega \;=\; \gamma(0) + 2\sum_{k=1}^{\infty}\gamma(k) \] when stationarity implies \(\gamma(-k)=\gamma(k)\).

2.1.1 Autocovariance function and summability conditions

The series expression is meaningful when the autocovariances are absolutely summable (or summable in an appropriate weaker sense): \[

\sum_{k=-\infty}^{\infty}	\gamma(k)	< \infty

\] is a common sufficient condition for existence and finiteness. Under such conditions, the variance of partial sums satisfies \[ \operatorname{Var}(S_n) \;=\; n\Omega + o(n), \] so \(\Omega\) is the limiting per-period growth rate.

2.1.2 Equivalence under stationarity assumptions

When strict or weak stationarity holds, covariance-stationary assumptions ensure that the long-run variance computed from the covariance series matches the limiting variance growth rate derived from partial sums. If stationarity fails, the notion of “one” long-run variance may need modification, for example by considering locally stationary approximations or time-varying analogues.

2.2 Spectral density interpretation

Long-run variance can also be expressed via the spectral density \(f(\omega)\) of a stationary process: \[ \Omega = 2\pi f(0). \] This interprets \(\Omega\) as the mass of the process at frequency zero (the “lowest frequency” component), which reflects how much low-frequency, slowly varying structure contributes to long-horizon variability.

2.2.1 Value of spectral density at frequency zero

In spectral terms, \(f(0)\) quantifies long-run, near-deterministic or slowly changing movement. If the process exhibits substantial low-frequency dependence, \(f(0)\) becomes large, and so does \(\Omega\). Conversely, rapid oscillations and short-lived correlations concentrate less energy at zero frequency, producing a smaller long-run variance.

2.3 Alternative formulations (e.g., using kernels)

Kernel-based approaches estimate long-run variance by weighting sample autocovariances up to a lag that depends on the sample size. Such estimators can be viewed as smoothed versions of the spectral density near \(\omega=0\). Formally, if \(\widehat{\gamma}(k)\) denotes an estimate of \(\gamma(k)\), a common form is \[ \widehat{\Omega}=\widehat{\gamma}(0) + 2\sum_{k=1}^{L} w(k/L)\,\widehat{\gamma}(k), \] where \(w(\cdot)\) is a kernel weight function and \(L\) is a truncation lag.

2.4 Conditions for existence and finiteness

Long-run variance exists and is finite under a range of assumptions, often involving:

• summability of autocovariances (absolute or conditional),
• decay of dependence (e.g., mixing or near-epoch dependence in probabilistic terms),
• absence of too-strong long memory that causes variance growth to be non-linear in \(n\).

When dependence decays slowly enough to prevent summability, the variance of partial sums can grow faster than linearly, and the standard long-run variance may either diverge or be replaced by a different scaling regime.

3 Examples and Special Cases

3.1 Independent and identically distributed (i.i.d.) processes

If \(\{X_t\}\) are i.i.d., then \(\gamma(k)=0\) for all \(k\neq 0\). The long-run variance reduces to \[ \Omega=\gamma(0)=\operatorname{Var}(X_t), \] so long-horizon uncertainty matches the short-run variance per period.

3.2 White noise and uncorrelated processes

For white noise, \(\mathbb{E}[X_t]=\mu\) and \(\operatorname{Cov}(X_t,X_{t+k})=0\) for \(k\neq 0\). If only second-order properties are considered, this yields \(\Omega=\gamma(0)\). In practice, “white noise” often implies additional distributional assumptions, but the long-run variance concept depends primarily on the autocovariance structure.

3.3 Moving-average processes

For a finite moving-average process of order \(q\), dependence dies out after \(q\) lags. As a result, the infinite covariance series collapses to a finite sum, making \(\Omega\) computable directly from the nonzero autocovariances. This provides a clear example where long-run variance is larger or smaller than \(\gamma(0)\) depending on whether lagged correlations contribute positively or negatively.

3.4 Autoregressive (AR) and general linear processes

In autoregressive models with sufficiently fast decay, autocovariances typically decrease geometrically. Then the autocovariance series is summable, and \(\Omega\) is finite. For linear processes driven by innovations, \(\Omega\) can often be expressed in closed form by summing the impulse response coefficients, yielding a transparent link between persistence and long-run uncertainty.

3.5 Deterministic trend removal and stationarity adjustments

Long-run variance is defined for stationary processes. If a series has a deterministic trend, then subtracting an estimate of the trend (or differencing) is often used to create a stationary residual component. Long-run variance then characterizes the remaining stochastic fluctuations. The adjustment matters: using an insufficiently removed trend can create spurious low-frequency dependence and distort estimates of \(\Omega\).

4 Long-Run Variance in Limit Theorems

4.1 Central limit theorem for dependent sequences

For dependent data, standard central limit theorems typically require that dependence is not too strong and that correlations are controlled. When conditions hold and \(\Omega\) is finite, a central limit theorem often takes the form: \[ \frac{1}{\sqrt{n}}\sum_{t=1}^n (X_t-\mu) \;\Rightarrow\; \mathcal{N}(0,\Omega). \] Here \(\Omega\) is the long-run variance, replacing \(\operatorname{Var}(X_t)\) from the i.i.d. case.

4.2 Weak dependence assumptions (overview)

“Weak dependence” is a collective term for conditions ensuring that long-range correlations either decay or are sufficiently limited. Examples include:

• mixing conditions that bound dependence between distant observations,
• functional dependence measures that quantify how a present value depends on past shocks,
• near-epoch dependence assumptions where only a limited window of past observations matters effectively.

These frameworks support the conclusion that partial sums behave like a normal random walk with variance parameter \(\Omega\).

4.3 Mixing and near-orthogonality intuition

A common heuristic is that distant observations become approximately uncorrelated, so the sum of dependent terms resembles a sum of nearly independent increments. Even when dependence persists at short lags, the cumulative effect of those correlations is summarized through \(\Omega\). In this view, long-run variance is what remains after dependence “between blocks” becomes negligible.

4.4 Implications for confidence intervals of sample means

If \(\sqrt{n}(\bar X_n-\mu)\) is asymptotically normal with variance \(\Omega\), then approximate \((1-\alpha)\)-level confidence intervals use standard errors proportional to \(\sqrt{\Omega/n}\). When \(\Omega\) is larger than the naive variance estimate, confidence intervals widen to reflect serial correlation; when it is smaller, the intervals tighten.

5 Estimation of Long-Run Variance

Estimating \(\Omega\) is central in practice because \(\Omega\) depends on the entire autocovariance structure, which is unknown. Estimators typically trade off bias from truncating or smoothing dependence against variance from estimating many autocovariances.

5.1 Plug-in covariance estimators

A basic strategy replaces theoretical autocovariances with their sample counterparts and then sums them with truncation.

5.1.1 Sample autocovariances and truncation

Let \(\widehat{\gamma}(k)\) be computed from data and consider \[ \widehat{\Omega}=\widehat{\gamma}(0)+2\sum_{k=1}^{L}\widehat{\gamma}(k), \] or a weighted version that uses fewer or differently weighted lags. Truncation at lag \(L\) limits the influence of far-lag estimates, which are noisy in finite samples.

5.2 Spectral (frequency-domain) estimators

Because \(\Omega=2\pi f(0)\), one can estimate the spectral density at low frequencies and multiply accordingly.

5.2.1 Periodogram smoothing near zero frequency

A common approach smooths the periodogram around frequency zero using a bandwidth. Smoothing reduces variance of the spectral estimate, while the bandwidth determines how much bias is introduced by oversmoothing or undersmoothing.

5.3 Kernel and bandwidth-based methods

Kernel estimators combine covariance and smoothing ideas: they apply weights that taper toward higher lags, mirroring smoothing in frequency space.

5.3.1 Choice of kernel

Different kernels (e.g., tapering or second-order kernels) affect how weights decline as lag increases. Kernels that taper smoothly can reduce unwanted artifacts such as negative weights or excessive bias, depending on the dependence pattern.

5.3.2 Bandwidth selection and bias–variance trade-off

Bandwidth (or truncation lag) controls the effective window length over which dependence is considered. Larger bandwidths include more autocovariances, decreasing truncation bias but increasing estimation variance. Smaller bandwidths do the opposite. Data-driven bandwidth selection methods aim to balance these effects asymptotically or via resampling criteria.

5.4 HAC (heteroskedasticity- and autocorrelation-consistent) estimators

In regression settings, HAC estimators provide robust standard errors when residuals are both heteroskedastic and autocorrelated. They compute an estimate of a long-run covariance matrix associated with the score or residual process and use it to form robust variance estimates for regression coefficients. The underlying idea is the same: dependence across time alters uncertainty, so standard variance formulas must be adjusted using a long-run variance-type estimator.

6 Properties of the Long-Run Variance

6.1 Nonnegativity and when it can be zero

Long-run variance is nonnegative because it corresponds to the limiting variance of partial sums (or to a spectral quantity). It becomes zero only in degenerate cases where the cumulative fluctuation of the process vanishes asymptotically, such as when \(X_t\) is almost surely constant after centering.

6.2 Behavior under linear transformations

If \(Y_t=a+bX_t\), then centering removes the intercept and scaling affects variance: \[ \Omega_Y = b^2\,\Omega_X. \] For multivariate linear combinations, the long-run covariance matrix transforms analogously through the corresponding linear operators.

6.3 Additivity across independent components

For independent processes \(X_t\) and \(Z_t\), the long-run variance of their sum equals the sum of their long-run variances: \[ \Omega_{X+Z}=\Omega_X+\Omega_Z. \] Independence ensures cross-covariances vanish, preventing additional long-horizon terms from appearing in the covariance series.

6.4 Sensitivity to long memory versus short memory

Long-run variance is well-behaved for short-memory processes where autocovariances are summable. For long-memory processes, autocovariances decay too slowly, and the variance of partial sums may grow faster than linearly. In such cases, the classical long-run variance either diverges or must be replaced by a different normalization tied to the strength of the memory.

7 Practical Considerations

7.1 Diagnostics for dependence persistence

Before estimating \(\Omega\), analysts often examine autocorrelation plots, decay rates of sample autocovariances, and residual-based diagnostics after detrending or fitting preliminary models. Evidence of slow decay suggests that long-run variance may be sensitive to bandwidth choices and that larger effective windows may be required.

7.2 Robustness to model misspecification

Long-run variance estimators are typically robust to certain forms of misspecification, particularly in semiparametric settings. For example, HAC procedures in regression focus on residual dependence without requiring correct specification of the conditional variance dynamics. Still, severe structural misspecification can distort residuals in ways that affect dependence patterns and therefore the estimated long-run uncertainty.

7.3 Finite-sample issues and edge effects

With finite \(n\), sample autocovariances at higher lags are less precise, motivating truncation or smoothing. Additionally, differencing or moving-window transformations can create edge effects that change the effective dependence structure near the boundaries. Careful implementation and consideration of degrees of freedom can improve stability.

7.4 Interpreting estimated long-run variance

An estimate of \(\Omega\) should be interpreted as an uncertainty per unit horizon for aggregated statistics. Large values indicate strong cumulative dependence, implying that averages converge slowly. Comparing \(\widehat{\Omega}\) to the naive sample variance offers a practical gauge of how much serial correlation is inflating uncertainty.

8 Connections and Applications

8.1 Heteroskedasticity and autocorrelation in regression

In ordinary least squares, conventional standard errors assume independent, homoskedastic errors. When residuals display serial correlation or changing variance over time, coefficient estimates remain valid under certain conditions, but their uncertainty measures must be updated. HAC methods incorporate a long-run covariance estimate of residual-related quantities.

8.2 Time series inference for policy/decision metrics (general)

For metrics computed from repeated observational units over time—such as performance measures, demand indicators, or experimental outcomes—sample averages are often used for decision making. If observations are temporally dependent, inference requires an uncertainty estimate consistent with dependence. Long-run variance provides the asymptotic variance input for confidence intervals and hypothesis tests for such averages.

8.3 Role in stochastic approximation and averaging

Many algorithms and estimators rely on averaging noisy updates across time. When updates are autocorrelated (due to recursions, adaptive steps, or feedback), the long-run variance determines the dispersion of the averaged iterate around its target. This helps characterize convergence rates and variability in stochastic approximation frameworks.

8.4 Robust standard errors in empirical analysis

Outside time-series-only contexts, long-run variance ideas appear wherever repeated observations yield correlated moment conditions. Robust variance estimation methods—often implemented through HAC-type formulas—provide standard errors that remain reliable under autocorrelation and mild heteroskedasticity, facilitating more trustworthy empirical conclusions.

9 References and Further Reading

9.1 Foundational texts on time series CLTs

For central limit theorems under dependence, standard references include monographs on limit theory for dependent sequences and time series. These works typically cover covariance summability, mixing conditions, and the asymptotic normality of partial sums and averages.

9.2 Spectral methods and asymptotic theory resources

Spectral interpretations connect long-run variance to the behavior of the spectral density near zero frequency. Background resources on time series spectra and asymptotic analysis of estimators provide the mathematical underpinning for frequency-domain approaches and periodogram smoothing.

9.3 HAC estimation references

HAC estimators are widely discussed in econometrics and applied statistics literature. References focusing on robust variance estimation, kernel-based covariance estimators, and bandwidth selection provide practical guidance on implementation and theoretical properties.