1 Fundamentals
Time series analysis studies observations indexed in time order. Unlike cross-sectional data, each value may depend on earlier values, so the sequence itself is part of the information. The field combines statistical inference, data preparation, and forecasting methods to understand how temporal patterns arise and how they can be used in practice.
1.1 Definition and scope
A time series is a collection of measurements taken at successive times, such as hourly temperatures, daily sales, or annual population counts. The scope of time series analysis includes describing the series, modeling dependence over time, detecting changes, and predicting future values. It is used in both scientific research and operational settings where time order matters.
1.2 Time-ordered data and temporal dependence
Time-ordered data are arranged so that earlier observations precede later ones. Temporal dependence means that the current value may be related to past values, sometimes strongly and sometimes weakly. This dependence distinguishes time series from independent observations and motivates specialized methods for estimation and prediction.
1.3 Goals of time series analysis
The main goals are to summarize observed behavior, infer underlying mechanisms, and make useful predictions. Depending on the context, analysts may also seek to detect anomalies, monitor systems, or evaluate how a variable responds to interventions or external inputs.
1.3.1 Description
Description focuses on identifying recurring structure in the data, such as upward drift, periodic variation, or abrupt changes. Visual summaries and decompositions are often used to present these features clearly.
1.3.2 Forecasting
Forecasting aims to estimate future values from past observations. Good forecasts often rely on stable patterns in trend, seasonality, and dependence, along with measures of uncertainty.
1.3.3 Control and monitoring
In monitoring applications, time series methods help detect when a process deviates from expected behavior. This is common in manufacturing, infrastructure surveillance, and automated diagnostics.
1.3.4 Causal interpretation
Some analyses attempt to assess whether changes in one series are associated with changes in another, or whether an intervention altered the process. Such interpretation requires careful design because temporal association alone does not prove causation.
1.4 Types of time series data
Time series can differ by how time is represented, how many variables are observed, and how evenly the measurements are spaced. These distinctions affect modeling choices and the interpretation of results.
1.4.1 Discrete-time and continuous-time series
Discrete-time series are observed at distinct time points, such as daily closing prices. Continuous-time series are defined for all times, although in practice they are usually measured discretely. Many real datasets are analyzed as discrete approximations to continuous processes.
1.4.2 Univariate and multivariate series
A univariate series records one variable over time. A multivariate series includes several variables observed simultaneously, such as temperature, pressure, and humidity. Multivariate methods can capture interactions among series.
1.4.3 Regular and irregular sampling
Regular sampling uses equal intervals between observations, such as every minute or every month. Irregular sampling occurs when measurement times vary or are missing. Irregular data often require additional preprocessing or specialized models.
2 Key characteristics
Time series often contain several recurring features that help describe their structure. Understanding these characteristics is central to both exploratory analysis and model building.
2.1 Trend
A trend is a long-term upward or downward movement in the series. It may reflect gradual growth, decline, or a persistent directional shift. Trends can be smooth or piecewise, and they often obscure shorter-term patterns.
2.2 Seasonality
Seasonality refers to regular variation that repeats over a known period, such as daily, weekly, or yearly cycles. Seasonal patterns are common in retail, weather, and transportation data. They are often modeled separately from longer-term changes.
2.3 Cyclic variation
Cyclic variation consists of recurrent ups and downs that do not have a fixed period. Cycles are often longer and less regular than seasonal effects. They may arise in economic or biological systems influenced by complex feedback.
2.4 Noise and randomness
Noise is the unpredictable component of a series, reflecting measurement error, unobserved influences, or inherent variability. Random fluctuations make exact prediction impossible, so models usually aim to capture the systematic part of the signal while leaving residual noise.
2.5 Stationarity
Stationarity describes whether the statistical properties of a series remain stable over time. It is a central concept because many methods assume at least approximate stability in mean, variance, or dependence structure.
2.5.1 Weak stationarity
Weak stationarity means that the mean and variance are constant over time and that autocovariance depends only on lag, not on calendar time. Many classical models rely on this form.
2.5.2 Strict stationarity
Strict stationarity requires the full joint distribution to be unchanged by time shifts. This is a stronger condition than weak stationarity and is less commonly checked directly in practice.
2.5.3 Nonstationarity and structural change
Nonstationarity occurs when properties evolve over time. Structural change refers to a more abrupt shift in level, variability, or dependence. Such changes may require transformations, differencing, or models that allow parameters to vary.
2.6 Autocorrelation and lag dependence
Autocorrelation measures the relationship between values separated by a lag. Strong lag dependence indicates that past observations help explain current ones. Patterns in autocorrelation are useful for diagnosing structure and selecting models.
3 Data preparation
Before analysis, time series data often need cleaning and adjustment. Careful preparation improves interpretability and reduces the risk of misleading results.
3.1 Data collection and sampling
Data collection methods influence quality, resolution, and bias. Sampling frequency should be chosen to match the process being studied; too coarse a schedule may miss important dynamics, while overly fine sampling may add noise and burden.
3.2 Missing data handling
Missing values can arise from sensor failure, reporting delays, or incomplete records. Common responses include interpolation, imputation, model-based estimation, or omission, depending on the amount and pattern of missingness.
3.3 Outlier detection and treatment
Outliers are unusual observations that may result from error, rare events, or genuine extremes. Detection methods compare points to neighboring values, fitted models, or robust summaries. Treatment can include verification, correction, transformation, or separate modeling.
3.4 Transformation and scaling
Transformations are used to stabilize variance, make patterns easier to see, or improve model fit. Scaling helps place variables on comparable ranges, especially in multivariate settings.
3.4.1 Logarithmic transformation
A logarithmic transformation compresses large values and can reduce right-skewness. It is often useful when variability grows with the level of the series.
3.4.2 Differencing
Differencing replaces each observation with the change from a previous one. This can remove trend or reduce nonstationarity and is especially important in autoregressive integrated models.
3.4.3 Detrending and deseasonalizing
Detrending removes long-term movement, while deseasonalizing removes repeated periodic effects. These adjustments help isolate shorter-term variation or residual dynamics.
3.5 Smoothing methods
Smoothing reduces short-term irregularity so that broader patterns are easier to detect. It is commonly used in exploratory analysis and as a preliminary step in forecasting.
3.5.1 Moving averages
Moving averages replace each point with the average of nearby observations. They are simple and intuitive, but they may blur sharp changes.
3.5.2 Exponential smoothing
Exponential smoothing gives greater weight to recent observations and gradually less to older ones. It is widely used for monitoring and forecasting because of its simplicity and flexibility.
3.5.3 Kernel smoothing
Kernel smoothing averages nearby observations using a weighting function. The choice of bandwidth controls the balance between detail and smoothness.
4 Exploratory analysis
Exploratory analysis helps reveal structure before formal modeling. It combines visual inspection with summary measures to identify patterns worth studying further.
4.1 Visualization techniques
Plots are often the first tool for understanding a time series. They can show direction, seasonality, irregular variation, and potential anomalies.
4.1.1 Time plots
Time plots display values against time in chronological order. They are useful for identifying trend, level shifts, and sudden disturbances.
4.1.2 Seasonal plots
Seasonal plots compare observations within repeated periods, such as months within years. They highlight recurring seasonal shapes and changes in seasonal amplitude.
4.1.3 Lag plots
Lag plots compare a series with delayed versions of itself. They can reveal dependence, nonlinear structure, and outlying points.
4.2 Summary statistics
Common summaries include mean, variance, median, range, and quantiles. For time series, such measures are often computed over selected windows to detect changing behavior.
4.3 Correlation structure
Correlation structure describes how observations relate across time lags. It is a major clue to the underlying dynamics of the series.
4.3.1 Autocorrelation function
The autocorrelation function shows correlation at different lags. Its shape can suggest seasonal repetition, persistence, or rapid decay.
4.3.2 Partial autocorrelation function
The partial autocorrelation function measures the direct relationship at a given lag after accounting for shorter lags. It is especially helpful in selecting autoregressive models.
4.4 Spectral exploration
Spectral exploration examines how variation is distributed across frequencies. It is useful when periodic or quasi-periodic behavior is present and may be less obvious in the time domain.
5 Time series decomposition
Decomposition separates a series into interpretable components such as trend, seasonal effects, and residual noise. This can clarify the structure of complex data and support forecasting.
5.1 Classical decomposition
Classical decomposition estimates trend and seasonal components using moving averages or related procedures. It provides a straightforward way to break the series into simpler parts.
5.2 Additive and multiplicative models
In additive models, components combine by summation. In multiplicative models, seasonal variation or noise scales with the level of the series. The choice depends on how variability behaves across the range of the data.
5.3 Trend-cycle estimation
Trend-cycle estimation aims to isolate the smooth long-run movement from shorter-term fluctuations. The resulting component is often easier to interpret than the raw series.
5.4 Seasonal adjustment
Seasonal adjustment removes systematic periodic effects to better reveal underlying movements. It is common in economic and administrative statistics.
5.5 Residual analysis
Residuals are the remaining unexplained part after decomposition or modeling. They are examined to assess whether major patterns have been captured and whether additional structure remains.
6 Statistical modeling
Statistical models formalize assumptions about dependence, variability, and evolution over time. They are used for explanation, prediction, and uncertainty quantification.
6.1 Stochastic process models
A stochastic process is a collection of random variables indexed by time. This framework provides the theoretical basis for many time series methods and describes how observations may be generated.
6.2 Autoregressive models
Autoregressive methods represent a series in terms of its own past values and random shocks. They are among the most widely used classes of time series models.
6.2.1 AR models
Autoregressive models express the current value as a combination of previous values plus noise. They are useful when recent history strongly influences the present.
6.2.2 MA models
Moving average models represent the series as a function of current and past error terms. They are well suited to short-lived dependence patterns.
6.2.3 ARMA models
Autoregressive moving average models combine AR and MA components. They can represent a broad range of stationary series with relatively compact parameterizations.
6.2.4 ARIMA models
Autoregressive integrated moving average models add differencing to handle nonstationarity. They are a standard choice for many practical forecasting tasks.
6.3 Seasonal models
Seasonal models incorporate periodic structure directly into the forecasting framework. They are especially useful when repeating patterns are strong and stable.
6.3.1 SARIMA
Seasonal ARIMA extends ARIMA by including seasonal lags and seasonal differencing. It is suited to data with both short-term dependence and regular periodicity.
6.3.2 Seasonal differencing
Seasonal differencing subtracts an observation from one at the same seasonal position in a previous cycle. This can remove repeating annual or monthly effects.
6.4 State-space models
State-space models describe observations as noisy measurements of an underlying evolving state. They are flexible and can accommodate changing levels, trends, and missing data.
6.4.1 Kalman filtering
Kalman filtering estimates hidden states recursively as new data arrive. It is efficient for linear Gaussian models and is widely used in tracking and forecasting.
6.4.2 Hidden Markov models
Hidden Markov models assume the process switches among unobserved states that follow a Markov chain. They are useful for regime-like behavior and sequence classification.
6.5 Regression with time series errors
This approach combines explanatory variables with autocorrelated errors. It allows analysts to study relationships with covariates while still accounting for temporal dependence.
6.6 Multivariate models
Multivariate models analyze several time series together. They can capture interactions, shared trends, and lead-lag relationships among variables.
6.6.1 Vector autoregression
Vector autoregression models each series as a function of past values of all series in the system. It is a common framework for interconnected dynamic data.
6.6.2 Cointegration
Cointegration describes a stable long-run relationship among nonstationary series. Even if individual series drift, some linear combinations may remain stable.
6.6.3 Granger causality
Granger causality tests whether past values of one series help predict another. It is a predictive notion of influence, not proof of direct causal mechanism.
7 Forecasting
Forecasting uses historical data to estimate future outcomes. Its quality depends on model fit, data regularity, and the stability of the underlying process.
7.1 Forecast horizons
The forecast horizon is the distance into the future being predicted. Short-term forecasts often rely on recent dependence, while long-range forecasts are more sensitive to structural uncertainty.
7.2 Point forecasts
A point forecast is a single best estimate of the future value. It is usually accompanied by a method for quantifying uncertainty.
7.3 Prediction intervals
Prediction intervals give a range in which future observations are expected to fall with a specified probability. Wider intervals reflect greater uncertainty.
7.4 Forecast evaluation
Forecast evaluation compares predicted values with observed outcomes. This helps identify which methods work best for a given dataset and objective.
7.4.1 Train-test splitting
Train-test splitting separates historical data used to fit a model from later data used to assess performance. In time series, the split must preserve chronology.
7.4.2 Cross-validation for time series
Cross-validation for time series uses rolling or expanding windows rather than random resampling. This respects temporal order and provides a more realistic assessment.
7.4.3 Error metrics
Error metrics such as mean absolute error, root mean squared error, and mean absolute percentage error summarize forecast accuracy. Different metrics emphasize different aspects of performance.
7.5 Combining forecasts
Combining forecasts averages or otherwise aggregates predictions from multiple models. This can improve robustness when no single model is consistently best.
7.6 Forecast uncertainty
Forecast uncertainty reflects unknown future shocks, parameter estimation error, and possible model misspecification. Communicating this uncertainty is essential for practical decision-making.
8 Frequency-domain methods
Frequency-domain methods analyze time series by decomposing them into oscillatory components. They are especially useful for periodic, seasonal, or quasi-periodic patterns.
8.1 Spectral density
Spectral density describes how variance is distributed across frequencies. Peaks in the spectrum indicate dominant periodic behavior.
8.2 Fourier analysis
Fourier analysis represents a series as a sum of sine and cosine waves. It provides a mathematical basis for many spectral techniques.
8.3 Periodogram methods
A periodogram is an estimate of spectral density derived from observed data. It can reveal candidate frequencies, though smoothing is often needed for stability.
8.4 Harmonic regression
Harmonic regression models periodic patterns using sine and cosine terms. It is useful when seasonality has a stable and interpretable form.
8.5 Wavelet analysis
Wavelet analysis studies variation across both time and scale. It is helpful for signals whose frequency content changes over time.
9 Inference and hypothesis testing
Inference methods assess whether observed patterns are likely to reflect genuine structure rather than chance. They support model selection, diagnostic checking, and scientific interpretation.
9.1 Testing for stationarity
Stationarity tests evaluate whether a series appears stable over time. Results should be interpreted cautiously because finite samples and structural changes can complicate conclusions.
9.2 Tests for unit roots
Unit root tests examine whether a series contains a persistent stochastic trend. Such tests are commonly used to decide whether differencing is needed.
9.3 Model comparison
Model comparison weighs goodness of fit against complexity. Criteria such as information-based scores and out-of-sample performance are often used together.
9.4 Parameter estimation
Parameter estimation determines the values of model coefficients from data. Methods include least squares, maximum likelihood, and Bayesian approaches.
9.5 Confidence intervals and standard errors
Confidence intervals summarize uncertainty about estimated quantities, while standard errors measure sampling variability. In time series, these quantities must account for dependence among observations.
10 Applications
Time series analysis appears in many disciplines because time-indexed data are common in natural and engineered systems. The choice of method depends on the goals and the structure of the data.
10.1 Finance and economics
Financial and economic series include prices, returns, interest rates, and indicators of activity. Analysts use time series methods to study volatility, cycles, and risk.
10.2 Meteorology and climate science
Weather and climate records often contain trend, seasonality, and long-term variability. Time series tools help summarize changes and compare patterns across intervals.
10.3 Engineering and signal processing
Engineering applications include vibration monitoring, sensor streams, communications, and control systems. Here, detecting anomalies and filtering noise are frequent objectives.
10.4 Medicine and public health
Medical and health-related series may track vital signs, disease incidence, or treatment responses. Time series methods support monitoring, pattern detection, and outcome analysis.
10.5 Biology and neuroscience
Biological time series include growth curves, population counts, and neural signals. These data can exhibit strong dependence, cycles, or abrupt state changes.
10.6 Industrial quality control
In industrial settings, time series analysis helps monitor production processes and detect deviations from expected operation. Early detection can reduce waste and prevent failures.
11 Software and computation
Time series analysis is supported by a wide range of software tools and computational techniques. Efficient computation is especially important for large datasets and real-time systems.
11.1 Statistical software packages
Common statistical environments provide functions for smoothing, decomposition, forecasting, and model fitting. These tools make standard workflows accessible to analysts and researchers.
11.2 Numerical optimization
Many models require parameter estimation through optimization. Stable algorithms and good starting values are important because objective functions may be nonlinear or multi-modal.
11.3 Simulation and bootstrapping
Simulation helps study model behavior under controlled conditions. Bootstrapping is used to estimate uncertainty when analytic formulas are difficult to derive.
11.4 Real-time and online analysis
Real-time analysis updates estimates as new data arrive. It is used in monitoring, control, and streaming applications where timely decisions are needed.
12 Limitations and challenges
Time series analysis is powerful, but its results depend heavily on the quality of the data and the stability of the process. Real-world series often violate simplifying assumptions.
12.1 Nonstationary and evolving systems
Many systems change over time, so models fitted to past data may become less accurate later. Adapting to evolving structure remains a central challenge.
12.2 Small sample issues
Short series provide limited information about dependence and periodicity. Estimation can therefore be unstable, and forecasts may have wide uncertainty.
12.3 Measurement error
Imprecise instruments or reporting processes can distort patterns. Measurement error may hide true structure or create misleading fluctuations.
12.4 High-dimensional time series
When many series are observed together, the number of parameters can grow quickly. This increases computational burden and the risk of overfitting.
12.5 Interpretability versus predictive accuracy
Some methods offer clear interpretation but modest predictive power, while others forecast well but are harder to explain. Choosing between them depends on the purpose of the analysis.