1 Definition and basic concepts
A random vector is a collection of random variables treated as a single multivariate quantity. Instead of assigning one numerical outcome to each trial of an experiment, it assigns a tuple of outcomes, often written as an ordered vector. This framework allows several related measurements to be modeled together, capturing both their individual behavior and their joint variation.
Random vectors appear naturally whenever outcomes are recorded in multiple dimensions, such as height and weight, location coordinates, or counts across several categories. They provide the basic language for multivariate probability and are central to the study of dependence among random components.
1.1 Random variables and vector-valued mappings
Each component of a random vector is itself a random variable. Formally, a vector-valued random variable is a measurable mapping from a probability space into a Euclidean space such as R^n. The components may be of the same type or of different types, depending on the model, though classical treatments usually consider real-valued components.
This viewpoint extends the familiar one-dimensional theory of random variables. Instead of analyzing a single distribution, one studies a joint law that describes how the components vary together.
1.2 Notation and dimensionality
A random vector is commonly denoted by capital letters such as X = (X1, X2, ..., Xn) or by a boldface symbol. The dimension n indicates the number of components. The order of the components matters, since each coordinate may represent a distinct quantity.
The dimensionality determines the geometric setting for the distribution. In low dimensions, results can often be visualized directly, while in higher dimensions, the structure is typically described through algebraic and probabilistic summaries.
1.3 Examples of random vectors
Random vectors arise in many basic settings. A repeated coin toss can be summarized by counts of heads and tails, a point chosen at random in a plane can be represented by its coordinates, and a household survey may record income, age, and family size together. In each case, the outcome is naturally multicomponent.
1.3.1 Discrete random vectors
A discrete random vector takes values in a countable set. Examples include vectors of category counts, numbers of successes in several trials, or the pair of outcomes from two dice. Its distribution is described by assigning probabilities to each possible vector value.
1.3.2 Continuous random vectors
A continuous random vector ranges over regions of Euclidean space and is described by a density when one exists. Examples include a point selected uniformly from a region, measured error components, or the coordinates of a noisy signal. Probabilities are then obtained by integrating over sets of interest.
1.4 Measurability and probability space formulation
Random vectors are defined on a probability space that supplies the underlying sample space, sigma-algebra, and probability measure. Measurability ensures that events such as the vector falling in a given region have well-defined probabilities. This formulation makes it possible to treat discrete, continuous, and mixed models within the same general framework.
2 Joint distributions
The joint distribution of a random vector describes the probabilities of combinations of component values. It is the main object used to analyze how variables interact. From it, one can derive marginal behavior, conditional behavior, and dependence measures.
2.1 Joint cumulative distribution function
The joint cumulative distribution function gives the probability that each component is less than or equal to a specified threshold. For a vector X = (X1, ..., Xn), it is defined by evaluating P(X1 ≤ x1, ..., Xn ≤ xn). This function uniquely determines the distribution of the vector.
2.2 Joint probability mass function
For discrete random vectors, the joint probability mass function assigns a probability to each possible vector outcome. It gives the likelihood that the vector takes an exact value. Summing the joint mass over all possible outcomes yields one.
2.3 Joint probability density function
For continuous random vectors, the joint probability density function describes probability through integration over regions of space. It does not give probabilities at single points, but rather a local density that can be integrated to obtain probabilities for intervals or sets. The density must be nonnegative and integrate to one over the support.
2.4 Marginal distributions
Marginal distributions describe the behavior of individual components or subsets of components taken alone. They are obtained from the joint distribution by summing or integrating out the remaining variables. Marginals often simplify analysis and are useful when only one part of a multivariate system is of interest.
2.5 Conditional distributions
Conditional distributions describe the behavior of some components once others are fixed or observed. They are essential for prediction, inference, and modeling dependence. In a multivariate setting, conditioning often changes the effective center and spread of the remaining variables.
2.5.1 Conditional probability mass functions
For discrete vectors, a conditional probability mass function gives probabilities after restricting attention to a particular value of another component or group of components. It is computed by dividing the joint probability by the appropriate marginal probability, when that marginal is positive. This is used in discrete Bayesian models and categorical dependence analysis.
2.5.2 Conditional probability density functions
For continuous vectors, a conditional probability density function describes the distribution of one set of variables given fixed values of another set. It is derived from the joint density and the relevant marginal density. Conditional densities are important in regression, filtering, and sequential updating.
3 Dependence structure
Dependence structure captures how the components of a random vector vary together. Some vectors have nearly independent coordinates, while others show strong association or more subtle forms of dependence. Understanding this structure is often as important as knowing the marginal distributions.
3.1 Independence of components
Components are independent when the distribution of any subcollection factors into the product of the individual distributions. Independence is a strong and useful property, but it is not common in many real data sets. Even when exact independence fails, models may use approximate or conditional independence to simplify analysis.
3.2 Correlation and covariance
Covariance measures the tendency of two components to vary together, while correlation rescales covariance to provide a standardized measure. Positive values indicate joint movement in the same direction, and negative values indicate opposite movement. Zero covariance does not always imply independence, although it does in certain special families, such as jointly normal vectors.
3.3 Covariance matrix
The covariance matrix collects all pairwise covariances among the components of a random vector. Its diagonal entries are variances, and its off-diagonal entries record cross-covariation. This matrix is symmetric and nonnegative definite, making it a central summary in multivariate analysis.
3.4 Copulas
Copulas separate dependence structure from marginal distributions. They encode how component distributions are linked while allowing each marginal to be modeled independently. This idea is especially useful when variables have different shapes or scales, or when dependence in the tails matters.
3.5 Joint moments and mixed moments
Joint moments involve expectations of products of component powers, such as E[X1^a X2^b]. Mixed moments summarize the shape of a multivariate distribution beyond means and covariances. They are used in theoretical characterizations, approximation methods, and moment-based estimation.
4 Transformations of random vectors
Transformations study what happens when a random vector is mapped into another vector or reduced to a new variable. These operations include linear maps, coordinate changes, and nonlinear functions. They are fundamental in deriving distributions of derived quantities.
4.1 Linear transformations
A linear transformation multiplies a random vector by a matrix and possibly adds a constant vector. Such transformations preserve many structural properties and are especially convenient in theory and computation. They are widely used in dimension reduction, decorrelation, and modeling of linear combinations.
4.2 Nonlinear transformations
Nonlinear transformations include products, ratios, norms, and other functions that do not preserve linear structure. They often produce more complicated distributions, but they can represent physically meaningful quantities such as speed, distance, or intensity. Analyzing these transformations typically requires specialized techniques.
4.3 Change of variables
The change-of-variables method is used to derive the distribution of a transformed random vector. It replaces the original coordinates with new ones and adjusts for the local distortion introduced by the transformation. This is a standard tool in multivariate integration and density calculation.
4.3.1 Jacobian determinant
The Jacobian determinant measures how a transformation scales volume locally. When converting densities under a smooth one-to-one map, the density is multiplied by the reciprocal of the absolute value of the Jacobian determinant. This correction ensures that total probability is preserved.
4.3.2 Support transformation
The support of a transformed vector is the set of values it can actually take. A transformation may expand, compress, or restrict the region of possible outcomes. Correctly identifying the transformed support is necessary before writing the new density or mass function.
4.4 Functions of random vectors
A function of a random vector is any random quantity obtained by applying a rule to the vector, such as a sum, maximum, norm, or indicator. These functions appear throughout probability theory and statistics. Many key results concern the distributions of such derived quantities.
5 Common multivariate distributions
Several multivariate distributions recur frequently in theory and applications. They serve as standard models for vectors with particular dependence patterns, count structures, or matrix-valued summaries. Familiarity with them provides a foundation for practical multivariate work.
5.1 Multivariate normal distribution
The multivariate normal distribution generalizes the normal distribution to several dimensions. It is one of the most widely used models because of its mathematical tractability and its role as an approximation in many settings. Its contours are ellipsoidal, and many operations on it remain analytically manageable.
5.1.1 Mean vector and covariance matrix
The mean vector specifies the center of the distribution, while the covariance matrix determines spread and dependence. Together, they fully characterize a multivariate normal law. Changing the covariance matrix alters the orientation and shape of the density level sets.
5.1.2 Properties and characterization
The multivariate normal distribution is stable under linear transformations and marginalization. Any linear combination of its components is again normally distributed. It is also characterized by the fact that uncorrelated components are independent when the joint distribution is normal.
5.2 Multinomial distribution
The multinomial distribution models counts in several categories over a fixed number of trials. It generalizes the binomial distribution to more than two outcomes. The components are constrained to sum to the total number of trials, which creates a built-in dependence among the counts.
5.3 Multivariate t-distribution
The multivariate t-distribution is a heavy-tailed analogue of the multivariate normal distribution. It is often used when observations exhibit greater variability or occasional extreme values. Like the normal family, it has a location vector and a scatter matrix, but with thicker tails.
5.4 Dirichlet distribution
The Dirichlet distribution is defined on vectors of nonnegative components that sum to one. It is commonly used as a model for proportions and probabilities. In Bayesian statistics, it often serves as a prior distribution for categorical probability vectors.
5.5 Wishart distribution
The Wishart distribution is a distribution on positive semidefinite matrices and arises naturally from sample covariance matrices. It plays a major role in multivariate inference, especially in estimation and hypothesis testing. It is closely connected to the multivariate normal distribution.
6 Estimation and inference
Estimation and inference concern learning about an unknown random vector distribution from data. Observed samples are used to estimate parameters, assess uncertainty, and test hypotheses. Multivariate settings often require matrix-valued summaries and joint procedures rather than one-variable methods.
6.1 Sample random vectors
A sample random vector is an observed realization of a random vector, and a sample of such vectors forms the basic data set for multivariate analysis. These observations are often assumed to be independent and identically distributed. Their joint structure provides the empirical basis for estimation.
6.2 Estimators based on vector observations
Estimators based on vector observations include sample means, covariance matrices, and more specialized summary statistics. These quantities are used to estimate distributional parameters and dependence patterns. In multivariate problems, an estimator often targets several quantities at once.
6.3 Maximum likelihood methods
Maximum likelihood methods choose parameter values that make the observed vector data most probable under a chosen model. In multivariate settings, likelihoods may involve densities, covariance matrices, and constraints on parameters. The method is valued for its generality and asymptotic efficiency under suitable conditions.
6.4 Method of moments
The method of moments matches theoretical moments to sample moments to estimate unknown parameters. For random vectors, this may involve means, variances, covariances, or higher-order mixed moments. It is often simpler than likelihood methods, though sometimes less efficient.
6.5 Hypothesis testing in multivariate settings
Multivariate hypothesis testing evaluates claims about means, covariance structures, independence, or distributional form. Common tests compare observed vector samples against a null model using statistics based on matrices or joint distances. Such procedures are used in quality control, genetics, economics, and other fields with multi-attribute data.
7 Applications
Random vectors are used wherever multiple related measurements are analyzed together. Their role is especially important when the relationships among variables matter as much as the variables themselves. This makes them a core concept across applied statistics and probability-based modeling.
7.1 Multivariate data analysis
Multivariate data analysis studies data sets with several variables per observation. Random vectors provide the theoretical model for such data and support methods like principal component analysis, clustering, and classification. The main goal is often to reduce complexity while preserving meaningful structure.
7.2 Regression and predictive modeling
In regression and predictive modeling, random vectors describe groups of predictors, responses, and error terms. They help formalize the uncertainty in multiple outputs or multiple inputs. Multivariate regression extends ordinary regression to settings where several dependent variables are modeled simultaneously.
7.3 Signal processing
Signal processing often treats sampled signals as random vectors, especially when analyzing finite blocks of data. This allows noise, filtering, and correlation to be studied using matrix methods. Random vector models are useful in detection, estimation, and adaptive algorithms.
7.4 Finance and risk modeling
In finance, random vectors are used to model collections of asset returns, losses, or risk factors. Dependence among components is crucial because joint fluctuations can affect portfolio behavior. Multivariate distributions are therefore central to diversification analysis and risk assessment.
7.5 Machine learning and feature vectors
Machine learning frequently represents each observation as a feature vector. Random vector models describe the variability of these features and their relation to labels or targets. They underpin tasks such as dimensionality reduction, probabilistic classification, and probabilistic graphical modeling.