1 Introduction
1.1 Motivation and historical context
State-space models arose from the need to represent dynamical systems in a way that cleanly distinguishes internal evolution from what an instrument reports. In control engineering, this separation supported systematic design of controllers and observers. In statistics and signal processing, it enabled principled estimation when the key quantities of interest are not directly measurable. Over time, the same framework also became a unifying language for many time-series models and for probabilistic inference in sequential data.
1.2 Key idea: hidden state vs. observations
A central premise is that a system’s relevant condition at time \(t\) is summarized by a latent (unobserved) state vector. The state changes over time according to a transition rule, while the measured output depends on the current state. This structure allows analysts to infer the hidden state from observations, rather than forcing all dynamics into the observed series alone.
1.3 Basic components and notation
Most formulations use:
- A state vector \(x_t\) describing the system at time \(t\).
- An observation (measurement) vector \(y_t\) recorded at time \(t\).
- Inputs \(u_t\) that may drive the dynamics.
- Model functions that map past state (and inputs) to the next state, and map the current state to observations.
- Noise terms capturing uncertainty in both evolution and measurement.
Time can be discrete or continuous, and the state dimension may vary depending on the modeling goal.
2 Mathematical Formulation
2.1 General state-space structure
2.1.1 State transition equation
A general discrete-time transition equation has the form \[ x_{t} = f(x_{t-1}, u_t, w_t), \] where \(f\) is a (possibly nonlinear) evolution function, \(u_t\) is an optional input, and \(w_t\) is process noise. The noise term reflects randomness in how the system moves between time points.
2.1.2 Observation (measurement) equation
Observations are linked to the hidden state by a measurement equation \[ y_{t} = g(x_{t}, v_t), \] where \(g\) can be nonlinear and \(v_t\) denotes measurement noise. This equation models how sensors or data-generating mechanisms transform the latent state into observed quantities.
2.1.3 Inputs, outputs, and noise terms
Inputs \(u_t\) represent external drivers that influence state evolution (e.g., control actions in engineering or known covariates in data modeling). Outputs are the recorded series \(y_t\). Process noise \(w_t\) and measurement noise \(v_t\) are often taken to be random, potentially independent across time and between each other, subject to modeling assumptions. Their distributions play a key role in inference methods.
2.2 Linear time-invariant (LTI) models
2.2.1 Matrix representation
A common linear, discrete-time LTI model uses matrices: \[ x_{t} = A x_{t-1} + B u_t + w_t,\quad y_{t} = C x_{t} + D u_t + v_t. \] Here, \(A\) determines state propagation, \(B\) maps inputs to state changes, \(C\) maps states to measurements, and \(D\) captures direct input effects on observations. Linear-Gaussian assumptions (such as \(w_t\) and \(v_t\) being Gaussian) yield closed-form filtering and smoothing.
2.2.2 Discrete-time vs. continuous-time
Discrete-time models update states at sampled times \(t=1,2,\dots\). Continuous-time models describe evolution with differential equations, often expressed via \[ \mathrm{d}x(t) = F x(t)\,\mathrm{d}t + L u(t)\,\mathrm{d}t + \mathrm{d}w(t), \] with an associated measurement equation for \(y(t)\). Continuous-time formulations are useful when sensing is effectively continuous or when sampling is derived from continuous dynamics.
2.3 Nonlinear and time-varying models
2.3.1 Nonlinear transition functions
Nonlinear dynamics replace the linear transition matrix with an arbitrary function: \[ x_{t} = f_t(x_{t-1}, u_t) + w_t, \] where \(f_t\) may depend on time index \(t\) and can embody nonlinear physical laws or nonlinear data relationships.
2.3.2 Nonlinear observation functions
Nonlinear observation models allow complex sensor behavior: \[ y_t = g_t(x_t) + v_t, \] enabling outputs such as bounded measurements, nonlinear distortions, or models that depend on higher-order interactions between latent components.
2.3.3 Time-varying parameters
Time-varying models allow matrices or functions to evolve, for example: \[ x_t = A_t x_{t-1} + B_t u_t + w_t,\quad y_t = C_t x_t + v_t. \] Such parameter changes may represent changing conditions, gradual system drift, or seasonal effects in a time-series context.
2.4 Stochastic vs. deterministic models
2.4.1 Process noise modeling
In stochastic models, \(w_t\) accounts for unmodeled dynamics, variability in forcing, and uncertainty in evolution. Its magnitude and distribution influence how quickly the filter adapts to new measurements and how much it trusts the model versus the data.
2.4.2 Measurement noise modeling
Measurement noise \(v_t\) reflects sensor errors, random fluctuations, or quantization effects. Accurate noise characterization improves credibility of uncertainty estimates and affects likelihood calculations used in parameter learning.
3 System Properties and Assumptions
3.1 Observability and identifiability
3.1.1 Intuition for observability
Observability asks whether the latent state can be inferred from observation sequences. If two different initial states yield indistinguishable output behavior, then the system cannot reliably recover the internal state from measurements. Intuitively, observability means measurements contain enough information to “see” the state through time.
3.1.2 Practical identifiability considerations
Even when a model is observable in principle, parameter estimates may remain uncertain without sufficient excitation in the data. Practical identifiability depends on data length, noise levels, model structure, and how persistently the system explores states that generate informative outputs. In nonlinear settings, identifiability can be especially delicate, since different parameter sets can produce similar trajectories.
3.2 Controllability (for input-driven systems)
3.2.1 Intuition for controllability
Controllability concerns whether inputs can steer the system state to desired regions. If the system cannot be influenced in directions relevant to performance, then some state components remain effectively inaccessible for control or estimation under practical input constraints.
3.2.2 Implications for design and estimation
In design workflows, controllability guides feasible control objectives, while observability supports reliable state reconstruction. Together they underpin many observer-controller pairings. In estimation, controllability-like considerations influence whether data generated under certain inputs will contain enough information to learn the hidden state and parameters.
3.3 Stability and well-posedness
3.3.1 Stability in discrete-time systems
Stability in discrete-time settings often relates to whether state trajectories remain bounded under the assumed dynamics and noise. For linear systems, stability typically depends on eigenvalues of the transition matrix \(A\), though noise and filtering behavior can introduce additional considerations.
3.3.2 Stability in continuous-time systems
In continuous-time models, stability is frequently tied to system eigenvalues (in a differential equation sense) and to whether uncertainty remains bounded over time. For inference algorithms, numerical stability and robustness to parameter mismatch also matter, especially for long runs.
3.4 Markov property and dependence structure
3.4.1 Conditional independence assumptions
A common assumption is that, given the present state (and input), the next state depends only on the previous state and not on earlier history. Similarly, given the current state, observations at time \(t\) are conditionally independent of past observations. These conditional independence properties yield tractable factorizations of the joint distribution.
3.4.2 Hidden Markov models as a special case
When the state is discrete, the transition and emission probabilities define a hidden Markov model (HMM). In that special case, the state evolves as a Markov chain and observations are generated by probabilistic emissions conditioned on the current hidden state.
4 Inference: State Estimation and Prediction
4.1 Filtering (online estimation)
4.1.1 Bayesian filtering viewpoint
Filtering computes the posterior distribution of the state given measurements up to the current time: \[ p(x_t \mid y_{1:t}). \] The update combines a prior prediction from the model with information contained in the newest observation.
4.1.2 Recursive estimation concept
The defining feature of filtering is recursion: the computation at time \(t\) reuses the previous posterior at time \(t-1\). This avoids reprocessing the entire history and enables real-time operation.
4.2 The Kalman filter (linear-Gaussian)
4.2.1 Predict step
For linear-Gaussian models, the predict step produces the predicted state mean and covariance: \[
| \hat{x}_{t | t-1} = A\hat{x}_{t-1 | t-1} + Bu_t,\quad |
|---|---|---|
| P_{t | t-1} = A P_{t-1 | t-1} A^\top + Q, |
\] where \(Q\) denotes process noise covariance.
4.2.2 Update step
The update step incorporates the measurement \(y_t\) using a gain matrix derived from predicted uncertainty and measurement noise: \[
| K_t = P_{t | t-1} C^\top (C P_{t | t-1} C^\top + R)^{-1}, |
|---|
\] \[
| \hat{x}_{t | t} = \hat{x}_{t | t-1} + K_t (y_t - C\hat{x}_{t | t-1}), |
|---|
\] with \(R\) as measurement noise covariance. The posterior covariance is reduced relative to the prediction step, reflecting gained information.
4.2.3 Interpretation via belief states
In this framework, the posterior distribution is often summarized by a Gaussian “belief” characterized by mean and covariance. The filter repeatedly refines this belief as new evidence arrives.
4.3 Smoothing (offline estimation)
4.3.1 Fixed-interval smoothing intuition
Smoothing estimates the state at earlier times using both past and future observations. Instead of \[ p(x_t \mid y_{1:t}), \] it targets \[ p(x_t \mid y_{1:T}). \] This typically yields more accurate estimates than filtering because it exploits information that becomes available later.
4.3.2 Fixed-lag smoothing use cases
Fixed-lag smoothing provides a compromise: it uses future data only up to a delay window. Such methods can improve estimate quality while keeping latency bounded for near-real-time applications.
4.4 Forecasting outputs
4.4.1 Multi-step prediction
Forecasting produces predicted observations several steps ahead, often using iterative propagation of the state distribution. In linear-Gaussian settings, means and covariances can be propagated forward to obtain prediction intervals.
4.4.2 Uncertainty quantification
Because the model includes explicit noise terms, forecasts naturally come with uncertainty. Credible intervals reflect both inherent process variability and uncertainty in the current latent state estimate.
5 Parameter Learning and Model Fitting
5.1 Estimation objectives
5.1.1 Maximum likelihood estimation (MLE)
MLE chooses parameters that maximize the likelihood of observed data under the model. For state-space models, the likelihood involves integrating out latent states, typically computed with filtering-like procedures.
5.1.2 Bayesian parameter inference
Bayesian approaches treat parameters as random variables with prior distributions. The posterior over parameters is obtained by combining priors with the data likelihood, often producing uncertainty quantification for learned parameters as well.
5.2 EM algorithm for state-space models
5.2.1 E-step using smoothing
The EM algorithm alternates between computing expected latent-state sufficient statistics (E-step) and updating parameters (M-step). In state-space contexts, the E-step usually relies on smoothing to evaluate quantities under the current parameter estimates.
5.2.2 M-step for parameter updates
The M-step updates parameters to maximize the expected complete-data log-likelihood. For linear-Gaussian models, closed-form updates may exist for noise covariances and certain matrix blocks; for more complex nonlinear cases, numerical optimization may be required.
5.3 Identifying linear Gaussian models
5.3.1 Relating state-space and ARMA forms
Linear state-space models have close relationships to ARMA representations. Under suitable conditions and appropriate choices of observation matrices, the input-output behavior can match an ARMA process, making it possible to translate between forms for both interpretation and estimation.
5.3.2 Practical estimation workflow
A typical workflow includes selecting a state dimension, fitting parameters (via MLE/EM), then checking whether the resulting innovations and residual behavior match model assumptions. Model reduction techniques may be applied if the chosen dimension is larger than necessary.
5.4 Model selection and diagnostics
5.4.1 Choosing model order/dimensions
Model order governs state dimension and the complexity of dynamics. Selection can be guided by information criteria, cross-validation on prediction performance, or stability constraints ensuring reasonable behavior.
5.4.2 Residual and innovation checks
In linear-Gaussian setups, innovation sequences (measurement residuals after accounting for predictions) can be analyzed. Diagnostic plots and statistical tests assess whether innovations resemble assumed white noise, indicating a good fit.
6 Connections to Other Formalisms
6.1 Hidden Markov models (HMMs)
6.1 Hidden Markov models (HMMs)
HMMs model the hidden state as belonging to a finite set, with observations generated by emissions conditioned on the hidden state. This yields a clear correspondence between state transitions and observed categories or discrete-valued measurements.
6.1.2 Relation to filtering
Inference in HMMs uses filtering over discrete belief states. Many conceptual steps align with continuous state filtering, though computations differ in how distributions are represented.
6.2 Control theory representations
6.2.1 State-feedback control perspective
In control, state-space models support feedback laws of the form \(u_t = K x_t\) for full-state control. When the state is not fully measurable, the structure naturally motivates observer-based control.
6.2.2 Observer design concept
An observer is an algorithm that estimates hidden states from measurements. Kalman filtering is a prominent example in the linear-Gaussian case, effectively acting as an optimal observer under assumptions about noise.
6.3 Time-series models
6.3.1 ARMA and ARIMA relationships
Many ARMA processes can be represented as linear state-space systems by augmenting states to capture past values. ARIMA models similarly connect through state-space formulations that include differencing and trend components.
6.3.2 Local level and trend models (conceptual)
State-space views are well suited for components like level and slope that change gradually. These models separate signals into evolving latent pieces plus observation noise, making them convenient for time-series decomposition.
7 Extensions and Modern Variants
7.1 Nonlinear filtering approaches
7.1.1 Extended Kalman filter (EKF)
The EKF applies local linearization by approximating nonlinear functions with first-order Taylor expansions around the current estimate. This yields a Gaussian approximation that can be effective when nonlinearities are mild.
7.1.2 Unscented Kalman filter (UKF)
The UKF avoids explicit linearization by propagating a set of deterministic sample points through nonlinear functions, then reconstructing mean and covariance. This can improve accuracy in mildly to moderately nonlinear regimes.
7.1.3 Particle filters
Particle filters represent the posterior distribution with weighted samples. They can handle strong nonlinearities and non-Gaussian noise, though they may require many particles and careful resampling strategies to reduce degeneracy.
7.2 Switching and regime-based models
7.2.1 Hidden Markov switching dynamics
Switching models allow the system to change behavior between regimes, such as different linear dynamics depending on a latent regime variable. Inference combines state estimation with regime classification through probabilistic transitions.
7.2.2 Piecewise linear modeling
Piecewise linear approaches approximate nonlinear systems with multiple linear segments active under different conditions. This can simplify estimation and control design while retaining nonlinear expressive power.
7.3 Continuous-discrete hybrid models
7.3.1 Stochastic differential equation (SDE) view
Hybrid models treat the latent state as evolving continuously under an SDE, with random perturbations. This is common in physical systems where dynamics follow differential equations but measurements occur at discrete times.
7.3.2 Discrete observations of continuous dynamics
In such settings, the continuous evolution between observation times is integrated (analytically or numerically), then the discrete measurement update refines the posterior. The resulting inference process combines continuous-time prediction with discrete-time correction.
7.4 High-dimensional state spaces
7.4.1 Dimension reduction ideas
High-dimensional models can be challenging computationally. Techniques such as reduced-rank approximations, principal component structures, or latent-factor parameterizations aim to lower the effective state dimension.
7.4.2 Regularization strategies
Regularization helps prevent overfitting and stabilizes parameter estimates in complex models. Examples include priors on noise covariances, constraints that enforce smoothness, and penalty terms in optimization objectives.
8 Applications (Non-controversial, General)
8.1 Signal processing
8.1.1 Tracking and denoising
In tracking applications, state-space models separate a latent signal trajectory from measurement noise. Recursive filtering can follow changing signal levels, while smoothing improves historical estimates once data become available.
8.2 Robotics and navigation
8.2.1 Localization concepts
Robotics frequently uses state-space formulations to estimate position and motion from sensor readings such as odometry and range measurements. The hidden state may include velocity, pose, or map-related variables, depending on the modeling scope.
8.3 Economics and finance (general modeling)
8.3.1 Latent-factor interpretations
In general economic and finance modeling contexts, latent factors can drive observed indicators. State-space models provide a mechanism to represent factor dynamics and to infer unobserved components from noisy measurements.
8.4 Biology and biomedical time series
8.4.1 Latent dynamical system modeling
Biological processes often evolve over time with hidden internal dynamics. State-space models support inference about latent trajectories behind observed signals, including physiological measurements contaminated by noise.
9 Practical Considerations
9.1 Choosing state dimension and model structure
Selecting the state dimension determines model expressiveness and computational cost. Too small a state may underfit dynamics, while overly large states can complicate learning and introduce identifiability issues. Structure choices, such as which variables are included in the state and whether inputs affect the observation directly, should align with domain knowledge.
9.2 Numerical stability and implementation
9.2.1 Covariance handling
Filtering algorithms rely on covariance matrices. Numerical implementations must maintain symmetry and positive semidefiniteness, often by using stable linear algebra routines and careful computation of matrix inverses.
9.2.2 Avoiding filter divergence
Filter divergence occurs when the algorithm’s uncertainty estimates become inconsistent, leading to poor updates. Common mitigations include tuning noise covariances, ensuring consistent initialization, using robust square-root forms of filtering, and verifying model assumptions.
9.3 Handling missing or irregular observations
9.3.1 Gaps in measurement streams
When measurements are missing, prediction can proceed without updates until new data arrive. This requires correctly managing time indices and using the model to propagate the state distribution across the gap.
9.3.2 Irregular sampling strategies
Irregular time steps occur when observations arrive asynchronously or with variable intervals. Approaches include modifying the transition model for each time gap, interpolating to a regular grid, or using continuous-time formulations that naturally accommodate variable sampling.
10 Summary
10.1 Core takeaways
State-space models describe dynamical systems by linking hidden state evolution to observed measurements. They provide a common structure for estimation, prediction, and learning, ranging from linear-Gaussian Kalman filtering to nonlinear and high-dimensional variants.
10.2 Typical workflow from modeling to inference
A typical workflow starts with specifying the state and measurement equations and choosing noise assumptions. Next, practitioners assess whether the model is suitable for inference, then apply filtering or smoothing to estimate states. Finally, parameters are learned using maximum likelihood, Bayesian methods, or EM, followed by diagnostics and model refinement based on predictive performance and residual behavior.