The Wiener–Khinchin theorem (also called the Wiener–Khinchin–Einstein theorem) is a fundamental result in the theory of stationary stochastic processes. It states that for a wide-sense stationary (WSS) random process, the power spectral density (PSD) is the Fourier transform of the autocorrelation function. The theorem bridges time-domain statistical properties and frequency-domain spectral content, making it essential in signal processing, time series analysis, and statistical physics. It was independently developed by Norbert Wiener (1930) and Alexander Khinchin (1934), with earlier contributions by Albert Einstein (1914).

1 Statement

1.1 Continuous-time formulation

Let \( x(t) \) be a continuous-time wide-sense stationary random process with autocorrelation function \( R_{xx}(\tau) = \mathbb{E}[x(t+\tau)x^*(t)] \). The power spectral density \( S_{xx}(f) \) is defined as the Fourier transform of \( R_{xx}(\tau) \):

\[ S_{xx}(f) = \int_{-\infty}^{\infty} R_{xx}(\tau) e^{-j2\pi f\tau} \, d\tau. \]

Conversely, the autocorrelation function can be recovered via the inverse Fourier transform:

\[ R_{xx}(\tau) = \int_{-\infty}^{\infty} S_{xx}(f) e^{j2\pi f\tau} \, df. \]

1.2 Discrete-time formulation

For a discrete-time WSS process \( x[n] \) with autocorrelation \( R_{xx}[k] = \mathbb{E}[x[n+k]x^*[n]] \), the power spectral density is given by the discrete-time Fourier transform:

\[ S_{xx}(e^{j\omega}) = \sum_{k=-\infty}^{\infty} R_{xx}[k] e^{-j\omega k}, \]

where \( \omega \in [-\pi, \pi] \). The inverse relation is:

\[ R_{xx}[k] = \frac{1}{2\pi} \int_{-\pi}^{\pi} S_{xx}(e^{j\omega}) e^{j\omega k} \, d\omega. \]

1.3 Conditions and assumptions

1.3.1 Wide-sense stationarity

The process must be wide-sense stationary: its mean is constant and its autocorrelation depends only on the time shift \( \tau \). This ensures that the Fourier transform relationship is meaningful.

1.3.2 Absolute integrability of the autocorrelation function

The autocorrelation function must be absolutely integrable: \( \int_{-\infty}^{\infty}R_{xx}(\tau)\, d\tau < \infty \) (continuous-time) or \( \sum_{k=-\infty}^{\infty}R_{xx}[k]&lt; \infty \) (discrete-time). This guarantees the existence of the Fourier transform.

1.3.3 Existence of the Fourier transform

The Fourier transform of \( R_{xx} \) must exist in the conventional sense. In practice, the theorem holds for a broad class of processes, including those with spectra that are not purely continuous (e.g., line spectra can be handled using generalized functions).

2 Proof

2.1 Heuristic derivation using the Fourier transform of the autocorrelation

Consider a finite-duration realization \( x_T(t) \) of a WSS process, equal to \( x(t) \) for \(t\leq T/2 \) and zero elsewhere. Its Fourier transform \( X_T(f) \) exists. The expected power per unit frequency is:

\[

\mathbb{E}\left[ \frac{X_T(f)^2}{T} \right] = \frac{1}{T} \int_{-T/2}^{T/2} \int_{-T/2}^{T/2} \mathbb{E}[x(t)x^*(s)] e^{-j2\pi f(t-s)} \, dt\, ds.

\]

Let \( \tau = t-s \). As \( T \to \infty \), the double integral converges to the Fourier transform of \( R_{xx}(\tau) \), yielding the PSD.

2.2 Rigorous proof via Bochner's theorem

2.2.1 Bochner's theorem statement

Bochner&#039;s theorem states that a continuous function \( R(\tau) \) is positive definite if and only if it is the Fourier transform of a finite non-negative measure. Positive definiteness means that for any finite set of times \( t_i \) and complex coefficients \( a_i \), \( \sum_{i,j} a_i \overline{a_j} R(t_i - t_j) \geq 0 \).

2.2.2 Application to positive definite autocorrelation functions

For a WSS process, the autocorrelation function \( R_{xx}(\tau) \) is positive definite. By Bochner's theorem, there exists a finite non-negative spectral measure \( \mu(f) \) such that:

\[ R_{xx}(\tau) = \int_{-\infty}^{\infty} e^{j2\pi f\tau} \, d\mu(f). \]

If \( \mu(f) \) is absolutely continuous, its derivative \( S_{xx}(f) = d\mu/df \) is the PSD, and the Fourier transform relation holds.

2.3 Alternative proof using the spectral representation theorem

The spectral representation theorem states that any WSS process can be expressed as \( x(t) = \int_{-\infty}^{\infty} e^{j2\pi ft} \, dZ(f) \), where \( Z(f) \) is an orthogonal increments process. The autocorrelation becomes:

\[

R_{xx}(\tau) = \mathbb{E}[x(t+\tau)x^*(t)] = \int_{-\infty}^{\infty} e^{j2\pi f\tau} \, \mathbb{E}[dZ(f)^2].

\]

Since \( \mathbb{E}[dZ(f)^2] = S_{xx}(f) \, df \), the Fourier relation follows directly.

3 Applications

3.1 Signal processing

3.1.1 Spectral density estimation

The theorem provides the theoretical basis for estimating the PSD from a finite-length sample of a WSS process.

3.1.1.1 Periodogram method
The periodogram estimates the PSD by computing \( \hat{S}_{xx}(f) = \frac{1}{N}\sum_{n=0}^{N-1} x[n] e^{-j2\pi fn}^2 \). It is asymptotically unbiased but inconsistent due to high variance.
3.1.1.2 Welch's method

Welch's method reduces variance by averaging modified periodograms: the signal is divided into overlapping segments, each windowed, and the squared magnitude of the DFT is averaged.

3.1.2 Random noise analysis

The theorem allows characterization of noise sources (e.g., white noise, pink noise) by their PSD. White noise has a flat PSD, while the autocorrelation is a delta function.

3.1.3 Filter design and system identification

The input-output relationship of a linear time-invariant (LTI) system with transfer function \( H(f) \) yields \( S_{yy}(f) =H(f)^2 S_{xx}(f) \). This is used to design filters and identify system parameters.

3.2 Physics and engineering

3.2.1 Fluctuation-dissipation theorem

In statistical physics, the theorem relates the spectral density of thermal fluctuations to the dissipative response of a system. The Wiener–Khinchin theorem provides the mathematical link.

3.2.2 Thermal noise (Nyquist formula)

Nyquist's formula for the voltage noise across a resistor at temperature \( T \) is given by \( S_V(f) = 4k_B T R \), where \( R \) is resistance. This follows from the autocorrelation of the random thermal motion of electrons.

3.2.3 Optics and photon statistics

In optics, the theorem relates the intensity autocorrelation of a light source to its power spectrum, used in interferometry and photon counting statistics.

3.3 Economics and finance

3.3.1 Spectral analysis of economic time series

Economic data (e.g., GDP, unemployment) are often modeled as WSS processes. The theorem enables frequency-domain analysis of business cycles and seasonal patterns.

3.3.2 Volatility modeling

In finance, the PSD of asset returns reveals periodicities and memory effects, aiding in volatility forecasting and risk management.

4.1 Generalized Wiener–Khinchin theorem for non-stationary processes

Extensions exist for locally stationary processes, where the PSD is replaced by a time-varying spectrum (e.g., the Wigner–Ville distribution or the evolutionary spectrum).

4.2 Wiener–Khinchin theorem for random fields

The theorem generalizes to multidimensional random fields, where the autocorrelation function’s Fourier transform gives the spatial power spectral density, used in image processing and turbulence studies.

4.3 Relationship to the cross-spectral density

For two jointly WSS processes \( x(t) \) and \( y(t) \), the cross-power spectral density \( S_{xy}(f) \) is the Fourier transform of the cross-correlation function \( R_{xy}(\tau) \). This is the cross-spectral analog of the theorem.

4.4 Discretization effects and aliasing

When sampling a continuous-time WSS process at rate \( f_s \), the discrete-time PSD is a periodic sum of the continuous PSD, leading to aliasing if the signal is not band-limited. The theorem clarifies how to choose \( f_s \) to avoid distortion.