1 Definition and intuition
1.1 Stationarity in distribution
A stationary distribution is a probability distribution over the states of a stochastic process such that, once the process is initialized with this distribution, the distribution of the state at later times is unchanged. It captures a notion of “time invariance” at the level of distributions rather than sample paths. In practical terms, it represents a stable regime where the process does not exhibit systematic drift in its state probabilities.
1.2 Markov processes and the evolution operator
For Markov processes, the evolution of distributions is governed by an operator determined by the dynamics. In discrete time, the next distribution is obtained by applying a transition rule; in continuous time, distributions evolve under a rate-based mechanism. A stationary distribution is exactly a fixed point of this distribution-evolution operator: applying the dynamics does not move the distribution.
1.3 Relation to equilibrium concepts
Although “equilibrium” is often used informally, stationary distributions formalize equilibrium-like behavior for Markovian models. They align with the idea of a long-run balance of probability flow across states. In many settings, stationary distributions also correspond to equilibrium measures for systems described by stochastic dynamics, and they underpin long-term averages of observables.
2 Mathematical formulation
2.1 Discrete-time Markov chains
2.1.1 Transition matrix form (πP = π)
Consider a discrete-time Markov chain with state space \(S=\{1,\dots,n\}\) (or countable) and transition matrix \(P\), where \(P_{ij}\) is the probability of moving from state \(i\) to state \(j\) in one step. A distribution \(\pi\) over \(S\) is stationary if it satisfies \[ \pi P = \pi, \] meaning that the distribution after one transition equals the original distribution.
2.1.2 Balance equations
In component form, \(\pi P=\pi\) yields a set of linear equations: \[ \pi_j=\sum_i \pi_i P_{ij}\quad \text{for each state } j, \] together with the normalization constraint \(\sum_j \pi_j=1\). These equations are often called global balance equations: they enforce that, in aggregate, probability mass arriving at each state matches the mass leaving under stationarity.
2.2 Continuous-time Markov chains
2.2.1 Generator form (πQ = 0)
For a continuous-time Markov chain with generator matrix \(Q\), where off-diagonal entries \(Q_{ij}\ge 0\) represent jump rates and diagonal entries satisfy \(Q_{ii}=-\sum_{j\ne i}Q_{ij}\), a stationary distribution \(\pi\) satisfies \[ \pi Q = 0. \] This expresses that the net instantaneous rate of change of the state probabilities is zero when the system is distributed according to \(\pi\).
2.2.2 Kolmogorov forward perspective
The forward (Kolmogorov) equation describes time evolution of the distribution \(\mu(t)\): \[ \frac{d}{dt}\mu(t)=\mu(t)Q. \] A stationary distribution is characterized by \(\mu(t)=\pi\) as a constant solution. Equivalently, the generator annihilates \(\pi\) on the right under the convention \(\pi Q=0\).
3 Existence of stationary distributions
3.1 Finite state spaces
3.1.1 Construction via linear algebra
If the Markov chain has finitely many states, stationary distributions exist under very broad conditions. One can typically compute them by solving the linear system \(\pi P=\pi\) (or \(\pi Q=0\) in continuous time) plus normalization. Existence is tied to the presence of at least one invariant probability measure. In finite settings, standard linear-algebraic arguments guarantee that an invariant probability vector exists for any stochastic transition structure.
3.2 Countable state spaces
3.2.1 Recurrence vs. transience overview
For countably infinite state spaces, existence of a stationary distribution is subtler. The distinction between recurrence and transience strongly influences whether a normalizable invariant distribution can be formed. Heuristically, if the chain repeatedly returns to certain regions with sufficient frequency, one can often construct a stationary distribution; if it “escapes” to infinity with positive probability in a way that prevents normalization, a stationary distribution may fail to exist.
3.3 General state spaces (high-level)
3.3.1 Invariant measures vs. stationary distributions
In general (possibly uncountable) state spaces, one works with invariant measures. A stationary distribution in the probabilistic sense corresponds to an invariant probability measure, while more general invariant measures need not be normalizable. Existence results depend on properties such as tightness, recurrence-like behavior, and regularity conditions on the transition mechanism. High-level approaches include fixed-point theorems and measure-theoretic criteria.
4 Uniqueness and ergodicity
4.1 Uniqueness criteria (overview)
Stationary distributions may not be unique. Uniqueness is often linked to whether the chain can reach all states from all others (in some probabilistic sense) and whether the chain avoids periodic oscillations. When uniqueness holds, the stationary distribution typically becomes the limiting distribution as time grows, for a large class of initial states.
4.2 Irreducibility
A chain is irreducible if every state can reach every other state with positive probability (possibly over multiple steps). In irreducible systems, the structure is less fragmented, which tends to reduce the possibility of multiple stationary distributions. In many classical results, irreducibility is a key ingredient in determining uniqueness and ergodic behavior.
4.3 Aperiodicity
Periodicity refers to the existence of cyclic behavior in return times. Aperiodicity means the chain does not get trapped into strict step-count patterns. Even with a unique stationary distribution, a periodic chain may fail to converge to it from arbitrary starting points; instead, distributions can oscillate among subsets of states over time.
4.4 Ergodic theorems and long-run averages
Ergodic theorems connect stationary distributions to long-run time averages of functions of the state. Under appropriate conditions, averages computed along a single long trajectory converge to expectations under the stationary distribution. This provides a bridge between distributional equilibrium and empirical observables, turning asymptotic probability statements into practical estimators.
5 Computation methods
5.1 Solving linear systems
5.1.1 Normalization constraints (sum to 1)
For a finite chain, computing \(\pi\) often reduces to solving \(\pi P=\pi\) plus \(\sum_i \pi_i=1\). Because the equations are not all independent, the system can be arranged so that the normalization constraint selects the correct probability scale. For continuous-time chains, the analogous task is solving \(\pi Q=0\) with \(\sum_i \pi_i=1\).
5.2 Iterative methods
5.2.1 Power iteration / repeated application of transitions
When directly solving linear systems is inconvenient, iterative methods apply the transition operator repeatedly to an initial distribution \(\mu_0\): \(\mu_{k+1}=\mu_k P\) (or \(\mu_{k+1}=\mu_k e^{Q\Delta t}\) in a time-discretized setting). Under conditions ensuring convergence, \(\mu_k\) approaches the stationary distribution, effectively performing a power-iteration-like procedure on the operator.
5.2.2 Convergence diagnostics
In practice, one monitors convergence using criteria such as distance between successive iterates, residual norms for \(\pi P-\pi\), or behavior of tracked statistics. Since convergence speed may vary widely across chains, diagnostics help determine whether the computed distribution is close enough to the invariant one for the intended application.
5.3 Special structures
5.3.1 Birth–death chains
Birth–death chains (nearest-neighbor chains on a line) admit tractable formulas for stationary distributions. Their structure turns balance equations into simple recursions, allowing \(\pi\) to be expressed via products of ratios of forward and backward transition probabilities. After forming the unnormalized weights, normalization yields the stationary distribution when the total mass is finite.
5.3.2 Reversible chains and simplifications
Reversibility often simplifies computation. When detailed balance holds (a strong form of stationarity), one can find \(\pi\) by solving pairwise balance relations rather than global ones. This can reduce the computational burden and can also provide explicit normalizing constants in many examples.
6 Convergence to stationarity
6.1 Types of convergence
6.1.1 Total variation distance
A common metric for assessing convergence is total variation distance between distributions \(\mu\) and \(\nu\): \[
| \|\mu-\nu\|_{\text{TV}}=\frac12\sum_{i} | \mu_i-\nu_i |
|---|
\] (in discrete finite settings). Small total variation indicates that probabilities of all events are close under the two distributions, making it a strong and practically meaningful notion of convergence.
6.1.2 Pointwise distribution convergence
Sometimes each state’s probability converges individually: \(\mu_k(i)\to \pi(i)\) as \(k\to\infty\). Pointwise convergence is weaker than total variation in general, but it can be easier to analyze in certain structured chains. Whether pointwise convergence implies stronger forms depends on additional assumptions, such as finiteness and uniform integrability.
6.2 Mixing time (conceptual connection)
6.2.1 Rapid vs. slow mixing
Mixing time quantifies how quickly the distribution approaches stationarity from the worst starting distribution. Chains with rapid mixing reach near-stationary behavior in relatively few steps, while slowly mixing chains retain memory of the initial state for a long time. Mixing time is especially relevant for randomized algorithms and simulation-based estimation.
6.3 Coupling and bounding techniques (overview)
Coupling methods construct two copies of the chain on a shared probability space so that they eventually meet, after which they move together. This provides upper bounds on convergence rates and can yield insights into how dependencies in the chain slow or accelerate mixing. Alternative bounding tools include spectral methods and conductance-based comparisons in various settings.
7 Reversible and detailed balance
7.1 Detailed balance condition
A Markov chain is reversible with respect to a distribution \(\pi\) if it satisfies the detailed balance condition. In discrete time, for all states \(i,j\), \[ \pi_i P_{ij}=\pi_j P_{ji}. \] This implies that, in equilibrium, the rate of transitions from \(i\) to \(j\) matches the reverse rate.
7.2 Using detailed balance to find π
Detailed balance turns the stationary problem into pairwise equations. One can often compute \(\pi\) up to a multiplicative constant by relating \(\pi_i\) and \(\pi_j\) along edges where transitions occur. After obtaining unnormalized weights, normalization ensures the result sums to one.
7.3 Common reversible examples (generic)
Reversible chains appear in many classic constructions where transition rules reflect symmetric mechanisms. In practice, many structured models—especially those defined through symmetric “local moves” with state-dependent acceptance—can be expressed in reversible form with respect to a computable \(\pi\). Reversibility is valued because it frequently guarantees nicer convergence behavior and enables efficient stationary distribution computation.
8 Long-run behavior and applications
8.1 Asymptotic state probabilities
When a chain has a unique stationary distribution and satisfies conditions ensuring ergodicity, the probability of being in each state approaches \(\pi\). Thus \(\pi(i)\) can be interpreted as the long-run fraction of time the process spends at state \(i\) in probabilistic terms, rather than merely as a formal fixed point.
8.2 Time averages and empirical measures
If one considers the empirical distribution of visited states over time, many ergodic results imply convergence toward the stationary distribution. More generally, for any function \(f\) of the state, the average \( \frac1n\sum_{t=0}^{n-1} f(X_t)\) can converge to \(\sum_i \pi_i f(i)\) under suitable hypotheses. This provides an operational method for estimating expectations without directly computing \(\pi\).
8.3 Performance metrics in models (abstract)
Stationary distributions serve as baseline inputs to compute steady-state performance metrics in abstract stochastic models. Examples include long-run frequencies of configurations, equilibrium costs, expected waiting times under equilibrium assumptions, or probabilities of being in particular regimes. These metrics frequently depend on expectations under \(\pi\), making stationary analysis a central step.
9 Common pitfalls
9.1 Confusing stationary distributions with transient behavior
A stationary distribution describes invariance of distributions over time; it does not imply that individual trajectories are constant or that the chain has already reached its long-run regime. Starting from a different initial state can produce a time-dependent distribution even if a stationary distribution exists.
9.2 Multiple stationary distributions in reducible chains
When the chain decomposes into communicating classes that do not communicate with each other, each class may have its own stationary distribution. In such reducible cases, the long-run behavior may depend on which class the process starts in, and there may be no single distribution that attracts all initial distributions.
9.3 Periodic chains and non-convergent distributions
Periodic chains can have a stationary distribution yet fail to converge to it in the sense of \(\mu_k\to \pi\). Instead, the distribution can oscillate, with subsequences approaching different limits or approaching the stationary distribution only at certain time parities.
10 Worked examples and exercises
10.1 Two-state Markov chain example
Let the states be \(\{0,1\}\) with transition matrix \[ P=\begin{pmatrix} 1-a & a\\ b & 1-b \end{pmatrix}, \quad a,b\in(0,1). \] A stationary distribution \(\pi=(\pi_0,\pi_1)\) satisfies \(\pi=\pi P\) and \(\pi_0+\pi_1=1\). From the \(0\to 1\) and \(1\to 0\) balancing, \[ \pi_1=\pi_0 a/\!b \quad \Rightarrow\quad \pi_1=\frac{a}{a+b},\ \pi_0=\frac{b}{a+b}. \] If \(a,b>0\), the chain is irreducible and aperiodic, so iterating from any initial distribution converges to this \(\pi\).
10.2 Random walk on a finite graph example
Consider a random walk on a finite undirected graph where, from a vertex \(i\), the chain moves to a uniformly chosen neighbor. The transition probabilities satisfy \[ P_{ij}=\frac{1}{\deg(i)}\quad \text{if } j\text{ is adjacent to }i,\ \text{and }0\text{ otherwise}. \] A standard reversible candidate is \[ \pi_i=\frac{\deg(i)}{\sum_k \deg(k)}. \] One checks detailed balance: \[
| \pi_i P_{ij}=\frac{\deg(i)}{2 | E | }\cdot \frac{1}{\deg(i)}=\frac{1}{2 | E | } |
|---|
=\pi_j P_{ji}. \] Thus \(\pi\) is stationary; under connectivity and aperiodicity conditions, it is also the limiting distribution.
10.3 Small birth–death chain example
Take states \(\{0,1,2\}\) with birth–death transitions:
- from \(i\) to \(i+1\) with probability \(p_i\),
- from \(i\) to \(i-1\) with probability \(q_i\),
- with remaining probability staying put when appropriate (so rows sum to one).
Assume \(p_0>0\), \(q_1>0\), \(q_2>0\), and define the interior ratio approach. Stationarity yields recursions of the form \[ \pi_1=\pi_0\frac{p_0}{q_1},\qquad \pi_2=\pi_1\frac{p_1}{q_2}. \] Hence \(\pi\) is proportional to \(\left(1,\frac{p_0}{q_1},\frac{p_0p_1}{q_1q_2}\right)\), and normalization provides \[ \pi_i=\frac{w_i}{w_0+w_1+w_2}, \] where \(w_i\) are the unnormalized weights from the ratios.
10.4 Practice problems (with answer outlines)
10.4.1 Practice problem: compute π for a given 3×3 transition matrix
Given a transition matrix \(P\) for states \(\{1,2,3\}\), find all stationary distributions by solving \(\pi P=\pi\) with \(\sum_i\pi_i=1\). Answer outline: Set up the linear equations from \(\pi=\pi P\); reduce to a minimal independent system; include normalization; solve for \(\pi\); verify nonnegativity.
10.4.2 Practice problem: detect whether convergence to π occurs
For a specified transition matrix, determine whether \(\mu_k\) converges to the stationary distribution for any initial \(\mu_0\). Answer outline: Check irreducibility and periodicity (e.g., via return-time parity or gcd of cycle lengths in finite graphs). If irreducible and aperiodic with a unique stationary distribution, convergence holds; periodicity can prevent convergence.
10.4.3 Practice problem: reversible chain verification
A chain is proposed to be reversible with respect to a candidate \(\pi\). Answer outline: Verify detailed balance on each permitted transition pair \(i,j\) where \(P_{ij}>0\) or \(P_{ji}>0\). Confirm \(\pi_iP_{ij}=\pi_jP_{ji}\) and normalize \(\pi\) if needed.
10.4.4 Practice problem: birth–death chain stationary weights
For a birth–death chain on \(\{0,\dots,N\}\) with given upward/downward probabilities, find \(\pi\) and determine when it is normalizable (if the state space is infinite). Answer outline: Use ratio relations \(\pi_{i+1}=\pi_i\cdot p_i/q_{i+1}\); express \(\pi_i\) via products of ratios; compute the normalization constant (finite sum for finite \(N\), or check divergence for infinite).