1 Introduction
1.1 Definition and basic concepts
A stochastic process is a mathematical object defined as a collection of random variables \(\{X_t\}_{t \in T}\) indexed by a parameter set \(T\), often representing time or space. The values taken by these random variables describe the state of a system that evolves in an uncertain manner. The probability law of the process is determined by the joint distributions of all finite subsets of the index set.
1.2 Index sets and state spaces
The index set \(T\) can be discrete (e.g., integers \(\mathbb{Z}\) or natural numbers \(\mathbb{N}\)) or continuous (e.g., intervals of \(\mathbb{R}\)). The state space is the set of all possible values that each \(X_t\) can assume, such as integers (discrete state space) or real numbers (continuous state space). The combination of index set and state space shapes the process’s classification and analytical tractability.
1.3 Sample paths and distributions
A sample path (or realization) of a stochastic process is a function \(t \mapsto X_t(\omega)\) for a fixed outcome \(\omega\) in the underlying probability space. The finite-dimensional distributions (f.d.d.s) – the joint distributions of any finite collection of variables – completely characterize the probabilistic behavior of the process. Convergence and regularity properties of sample paths are key topics in advanced theory.
2 Classification of stochastic processes
2.1 Discrete-time vs. continuous-time
Discrete-time processes have an index set that is countable, such as \(\{0,1,2,\ldots\}\). Continuous-time processes have an index set that is an interval of real numbers. This distinction affects the definition of transition dynamics, sample path regularity, and the tools used for analysis (e.g., difference equations vs. differential equations).
2.2 Discrete-state vs. continuous-state
A process has a discrete state space if its random variables take values in a countable set (e.g., integers, finite set). A continuous-state process takes values in an uncountable set, typically a subset of \(\mathbb{R}^d\). Many processes combine a discrete index set with a continuous state space or vice versa.
2.3 Stationarity and ergodicity
2.3.1 Strict-sense stationary
A stochastic process is strictly stationary if for any finite set of indices \(t_1,\dots,t_n\) and any shift \(h\), the joint distribution of \((X_{t_1+h},\dots,X_{t_n+h})\) is identical to that of \((X_{t_1},\dots,X_{t_n})\). This implies that all statistical properties are invariant under time shifts.
2.3.2 Wide-sense stationary
A process is wide-sense stationary (or weakly stationary) if its mean function is constant and its autocovariance function depends only on the lag, not on absolute time. This weaker condition is sufficient for many applications, especially in signal processing and time series analysis.
2.3.3 Ergodicity
Ergodicity concerns the equivalence of ensemble averages and time averages. For an ergodic stationary process, the average over a single infinitely long sample path converges (in probability) to the ensemble mean. This property is crucial for statistical inference from a single realization.
3 Key properties and characteristics
3.1 Markov property
A process satisfies the Markov property if the future evolution depends only on the present state, not on the past. Formally, for any \(t>s\), the conditional distribution of \(X_t\) given the entire past up to time \(s\) equals that given only \(X_s\). This memoryless property simplifies analysis and modeling.
3.1.1 Markov chains (discrete-time)
A discrete-time Markov chain is a process with a discrete state space and the Markov property. It is defined by a transition probability matrix \(P = (p_{ij})\), where \(p_{ij} = \Pr(X_{n+1}=j \mid X_n=i)\). Chains can be classified by their transient and recurrent states, periodicity, and stationary distributions.
3.1.2 Continuous-time Markov processes
Continuous-time Markov processes (often called Markov jump processes) evolve through jumps at random times. The dynamics are described by a rate matrix \(Q\), where off-diagonal entries give instantaneous transition rates between states. The sojourn time in each state is exponentially distributed, making the process memoryless.
3.2 Martingales
A martingale is a stochastic process that models a fair game: the conditional expectation of a future value given the present is equal to the present value. Formally, \(\mathbb{E}[X_{t} \mid \mathcal{F}_s] = X_s\) for \(t>s\), where \(\mathcal{F}_s\) is the history up to time \(s\). Martingales are central to stochastic calculus, option pricing, and sequential analysis.
3.3 Gaussian processes
A Gaussian process is a collection of random variables such that every finite linear combination is normally distributed. Its distribution is completely determined by its mean function and covariance function. Gaussian processes are widely used in machine learning (e.g., regression) and spatial statistics.
3.4 Self-similarity and long-range dependence
A process is self-similar if scaling in time yields a process with the same distribution up to a normalizing factor: \(\{X_{at}\} \overset{d}{=} \{a^H X_t\}\) for some Hurst exponent \(H\). Long-range dependence (LRD) means that the autocovariance decays hyperbolically, so past values have a persistent influence. Fractional Brownian motion is a classic example.
4 Major types of stochastic processes
4.1 Random walks
A random walk is a discrete-time process where each step is a random increment, typically independent and identically distributed. It serves as a foundational model for many phenomena in physics and finance.
4.1.1 Simple symmetric random walk
In the simple symmetric random walk on \(\mathbb{Z}\), the step distribution is \(+1\) with probability \(1/2\) and \(-1\) with probability \(1/2\). The process is recurrent in one dimension (returns to zero infinitely often with probability 1) but transient in higher dimensions.
4.1.2 Random walk with drift
A random walk with drift has steps with a non-zero mean \(\mu\). The process tends to move in the direction of the drift over time, and its position asymptotically behaves like a linear function plus a Brownian motion term. It models asset prices under a constant expected return.
4.2 Poisson processes
The Poisson process is a continuous-time counting process that models the occurrence of rare events. It is characterized by independent increments and a Poisson distribution for the number of events in any interval.
4.2.1 Homogeneous Poisson process
A homogeneous Poisson process has a constant rate \(\lambda > 0\). The number of events in an interval of length \(t\) follows a Poisson distribution with mean \(\lambda t\), and interarrival times are independent and exponentially distributed with rate \(\lambda\).
4.2.2 Non-homogeneous and compound Poisson processes
A non-homogeneous Poisson process has a time-varying intensity function \(\lambda(t)\). The number of events in an interval follows a Poisson distribution with mean \(\int \lambda(t)\,dt\). A compound Poisson process adds random jump sizes to each event, producing a process with jumps of varying magnitudes.
4.3 Brownian motion (Wiener process)
Brownian motion is a continuous-time, continuous-state process with independent Gaussian increments. It is the scaling limit of random walks and is central to stochastic calculus and diffusion theory.
4.3.1 Properties and sample paths
Standard Brownian motion \(B_t\) satisfies: \(B_0 = 0\), increments are independent and normally distributed with variance equal to the time difference, and sample paths are continuous but nowhere differentiable with probability 1. Its quadratic variation over \([0,t]\) is \(t\).
4.3.2 Relation to random walk
By the functional central limit theorem (Donsker’s theorem), a properly scaled simple random walk converges in distribution to a Brownian motion as the step size goes to zero and the number of steps goes to infinity. This connection underpins many approximation methods.
4.4 Renewal processes
A renewal process is a generalization of the Poisson process where interarrival times are independent and identically distributed but not necessarily exponential. It models events that occur at random intervals with a general distribution. Key results include the renewal equation, renewal theorem, and the elementary renewal theorem.
5 Applications
5.1 Finance (stock prices, option pricing)
Stochastic processes are indispensable in mathematical finance. Geometric Brownian motion is the classic model for stock prices under the Black–Scholes framework. Poisson jumps model sudden market movements, and stochastic volatility processes capture changing risk. Martingale theory underpins derivative pricing and hedging.
5.2 Physics (diffusion, noise)
Brownian motion models the random movement of particles suspended in a fluid (diffusion). Stochastic processes describe thermal noise in electrical circuits, the random motion of molecules, and the evolution of systems in statistical mechanics. The Langevin equation combines deterministic drift with stochastic noise.
5.3 Queueing theory
Queues are modeled using arrival processes (often Poisson) and service time distributions. Markov chains describe the number of customers in a system, and renewal processes model customer interarrivals. Results such as Little’s law and the Pollaczek–Khinchine formula depend on stochastic process theory.
5.4 Biology (population dynamics, genetics)
Stochastic birth–death processes model population growth with random births and deaths. The Wright–Fisher process and the Moran process describe genetic drift in finite populations. Branching processes model the spread of a mutant gene, and continuous-time Markov chains are used in epidemiology.