1 Basic concepts
A multidimensional distribution, also called a multivariate distribution, describes the probabilistic behavior of two or more random variables considered together. Its central purpose is to represent not only the distribution of each variable on its own, but also how the variables co-vary across a joint sample space. This joint viewpoint is essential when outcomes are linked, as in paired measurements, correlated signals, or several attributes observed on the same object.
In practice, multidimensional distributions provide the language for many statistical models. They make it possible to express uncertainty in several dimensions, analyze interactions among variables, and study how collective patterns differ from separate one-variable summaries. Depending on the variables involved, the distribution may be discrete, continuous, or a mixture of both.
1.1 Random vectors
A random vector is an ordered collection of random variables, usually written as a column vector. Each component represents one coordinate of a joint outcome. For example, height and weight measured on the same person can be treated as a two-dimensional random vector, while temperature, humidity, and pressure form a three-dimensional one.
Random vectors allow multivariate problems to be written compactly. Their joint values lie in a multivariate space, and probability statements concern regions, slices, or points in that space. The vector form also supports matrix notation, which is convenient for describing covariance, transformations, and estimation procedures.
1.2 Joint distribution
The joint distribution specifies the probability structure of all components of a random vector at once. It answers questions such as the likelihood that several variables simultaneously fall within given ranges or take specific values. From the joint distribution, many other features of the multivariate system can be derived, including marginal behavior and conditional relationships.
1.2.1 Joint probability mass function
For discrete random variables, the joint probability mass function assigns a probability to each combination of values. It gives the chance that the vector equals a particular tuple, such as a pair of counts or category labels. The probabilities over all possible combinations must sum to one.
1.2.2 Joint probability density function
For continuous random variables, the joint probability density function describes how probability is distributed over regions of the multivariate space. Unlike a mass function, a density does not assign probability to a single exact point; instead, probabilities are obtained by integrating over areas or volumes. The density is useful for modeling measurements that vary along a continuum.
1.2.3 Joint cumulative distribution function
The joint cumulative distribution function gives the probability that each component of the random vector is less than or equal to a specified value. It is defined for discrete, continuous, and mixed cases, making it a broadly applicable characterization. The cumulative form is especially useful for describing dependence and for defining multivariate probabilities without requiring a density.
1.3 Dimensionality and support
Dimensionality refers to the number of random variables in the vector. As the number increases, the sample space becomes higher-dimensional, and geometric intuition often becomes less direct. This growth in dimension affects visualization, computation, and estimation.
The support of a multidimensional distribution is the set of points where the distribution has positive probability or nonzero density. In multivariate settings, support can have a simple rectangular form, a curved shape, or a lower-dimensional structure embedded in a larger space. Understanding the support is important for interpreting constraints among variables and for selecting appropriate models.
1.4 Independence and dependence
Independence means that knowledge of one variable gives no information about another. In a multivariate context, full independence implies that the joint distribution factors into separate marginal distributions. This is a strong condition and is often unrealistic in applied work.
Dependence, by contrast, means that variables are linked in some way. The link may be linear, nonlinear, weak, strong, symmetric, or asymmetric. A major aim of multivariate analysis is to quantify and model this dependence structure, since it governs how variables behave together and how one component may inform another.
2 Marginal and conditional distributions
Marginal and conditional distributions are derived from the joint distribution and describe selected aspects of multivariate behavior. Marginals focus on individual components taken separately, while conditional distributions describe how one part behaves when others are fixed or known. Together, they connect the joint structure to simpler one-variable summaries and to predictive relationships.
2.1 Marginal distributions
A marginal distribution is the distribution of one component, or a subset of components, obtained by summing or integrating out the remaining variables. For example, from a joint distribution of income and age, one can derive the distribution of income alone. Marginals reveal the standalone behavior of each variable without direct reference to the rest.
Although marginals are simpler than the full joint distribution, they do not fully capture dependence. Two variables may have the same marginal distributions and still differ greatly in how they are associated. For this reason, marginal information is useful but incomplete.
2.2 Conditional distributions
Conditional distributions describe the distribution of some variables given that others take certain values or fall in specified ranges. They are central to prediction, regression, and sequential reasoning. Conditioning narrows the focus from the full population of joint outcomes to a subset compatible with the known information.
2.2.1 Conditional probability
In the discrete setting, conditional probability gives the chance of one event occurring when another event is already known. It is often expressed as the ratio of a joint probability to the probability of the conditioning event, provided the latter is not zero. This idea extends naturally to several variables and regions.
2.2.2 Conditional density and mass functions
For continuous variables, the conditional density describes how probability is distributed after fixing one or more conditioning variables. For discrete variables, the analogous object is a conditional mass function. These conditional forms are essential in Bayesian updating, prediction, and hierarchical modeling, where partial information is used to refine uncertainty.
2.3 Joint versus marginal behavior
Joint behavior concerns the full configuration of variables at the same time, while marginal behavior concerns each variable on its own. A variable may appear stable in its marginal distribution yet exhibit strong contextual variation in the joint setting. Likewise, a joint distribution can show clustering, alignment, or tail dependence that is invisible in separate marginals.
This distinction matters because many phenomena are shaped by interaction. In applied analysis, looking only at marginals can miss relationships that emerge only when variables are studied together. The comparison between joint and marginal behavior therefore lies at the heart of multivariate modeling.
2.4 Conditional expectation
Conditional expectation is the expected value of one variable given information about others. It plays a role analogous to a best prediction under squared-error loss. In multivariate analysis, conditional expectation is used to summarize the mean response associated with particular covariate values.
Because it averages over the remaining uncertainty, conditional expectation provides a compact way to describe dependence. It is widely used in regression, smoothing, time series analysis, and stochastic processes. In many models, the conditional mean is the first and most important feature to estimate.
3 Measures of association
Measures of association summarize the strength and form of dependence among variables. Some focus on linear relationships, while others are sensitive to more general patterns. These measures help compare datasets, guide model selection, and assess whether the components of a multivariate system move together in meaningful ways.
3.1 Covariance
Covariance measures how two variables vary together around their means. A positive covariance indicates that larger values of one variable tend to occur with larger values of the other, while a negative value suggests the opposite. A covariance near zero indicates little linear co-movement, though it does not imply independence.
Covariance depends on the units of measurement, so it is most useful as part of a broader matrix description or after standardization. Despite this limitation, it remains a fundamental quantity in multivariate statistics because it captures direct joint variation.
3.2 Correlation
Correlation is a standardized version of covariance that is bounded between -1 and 1 for linear association. It measures the degree to which two variables follow a linear relationship, regardless of scale. A correlation near 1 or -1 signals strong linear alignment, while a value near 0 indicates weak linear association.
Correlation is widely reported because it is easy to interpret and compare across variables. However, it should not be treated as a complete summary of dependence, since nonlinear relationships may exist even when the correlation is small.
3.3 Covariance matrix
The covariance matrix collects all pairwise covariances among the components of a random vector. Its diagonal entries are the variances of each variable, and its off-diagonal entries describe joint variation. This matrix is a central object in multivariate theory because it organizes dispersion and linear dependence in compact form.
The covariance matrix is used in principal component methods, Gaussian models, estimation, and hypothesis testing. It also helps define geometric features such as elliptical contours and directions of greatest variability. When positive definite, it supports many standard multivariate calculations.
3.4 Dependence measures
Dependence measures attempt to capture association beyond simple linear correlation. They may detect monotonic, nonlinear, or information-based relationships. Different measures are suited to different tasks, so the choice of metric depends on the kind of dependence one wishes to study.
3.4.1 Rank correlation
Rank correlation compares the ordering of values rather than their numerical magnitudes. Common versions include Spearman’s rho and Kendall’s tau. These measures are useful when the relationship is monotone but not necessarily linear, and they are often more robust to outliers than ordinary correlation.
3.4.2 Mutual information
Mutual information quantifies the reduction in uncertainty about one variable gained by observing another. It is an information-theoretic measure that can detect broad classes of dependence, including nonlinear patterns. Unlike correlation, it is not limited to linear association, though it is often harder to estimate accurately from finite data.
4 Types of multidimensional distributions
Multidimensional distributions come in several major forms depending on whether the variables are discrete, continuous, or partly both. Particular named families are especially important because they arise frequently in theory and applications. These families provide standard models with known properties and tractable formulas.
4.1 Multivariate discrete distributions
Discrete multivariate distributions describe vectors whose components take countable values. They often arise in counting experiments, categorical data, and sampling without continuous measurement. The probability structure is given by joint masses over the relevant combinations of values.
4.1.1 Multinomial distribution
The multinomial distribution generalizes the binomial distribution to more than two categories. It describes the counts observed when a fixed number of trials are classified into several mutually exclusive outcomes. The sum of the category counts equals the total number of trials, making the components dependent by construction.
4.1.2 Multivariate hypergeometric distribution
The multivariate hypergeometric distribution arises from sampling without replacement from a finite population divided into categories. Each draw changes the composition of the remaining population, which induces dependence among the category counts. This distribution is useful in survey sampling, quality control, and combinatorial counting.
4.2 Multivariate continuous distributions
Continuous multivariate distributions describe variables that vary over a continuum. They are used for measurements, latent factors, and many physical quantities. Their joint densities often encode both overall spread and dependence geometry.
4.2.1 Multivariate normal distribution
The multivariate normal distribution is one of the most important multivariate families. It is characterized by a mean vector and a covariance matrix, and its contours are typically ellipsoidal. Many statistical methods rely on it because of its mathematical tractability and the central role it plays in approximation theory.
4.2.2 Multivariate t-distribution
The multivariate t-distribution is similar to the multivariate normal distribution but has heavier tails. These heavier tails make it useful when extreme observations occur more often than a Gaussian model would predict. It is often employed as a robust alternative in multivariate inference and Bayesian modeling.
4.3 Mixed distributions
Mixed distributions combine discrete and continuous components within the same multivariate system. For example, a model may include a binary indicator together with a continuous measurement. Mixed structures appear in censored data, zero-inflated settings, and models with structural categories plus numerical outcomes.
Such distributions are more complex than purely discrete or continuous ones because they require separate treatment of different parts of the support. Nevertheless, they are common in practice and important for realistic statistical modeling.
5 Representation and characterization
Representation and characterization concern the mathematical tools used to describe a multidimensional distribution fully or partially. Different tools highlight different aspects of the same joint law, such as moments, transforms, or dependence structure. These representations are valuable for proving results, deriving estimators, and comparing models.
5.1 Joint moments
Joint moments are expectations of products of powers of the components of a random vector. They generalize ordinary moments such as the mean and variance to multiple dimensions. Joint moments describe collective shape, asymmetry, and tail behavior, and they help identify a distribution when enough of them are known.
5.2 Moment generating functions
The moment generating function is a transform that encodes all moments of a distribution when it exists in a neighborhood of zero. In the multivariate case, it uses a vector argument and summarizes the joint moments of the components. When available, it is a powerful tool for deriving properties of sums, transforms, and limiting behavior.
5.3 Characteristic functions
The characteristic function is the Fourier transform of a probability distribution. It exists more generally than the moment generating function and therefore applies to a broad class of multivariate distributions. Characteristic functions are useful for proving uniqueness, establishing convergence, and handling sums of random vectors.
5.4 Copulas
Copulas separate marginal behavior from dependence structure. They are functions that link one-dimensional marginals into a full multivariate distribution. This separation makes them especially useful when variables have different marginal shapes but share a common dependence pattern.
5.4.1 Copula-based dependence modeling
Copula-based modeling allows analysts to choose marginals and dependence structure independently. This flexibility is helpful in finance, risk analysis, and multivariate data fitting, where joint tails or asymmetric dependence may need special attention. Copulas can represent dependence that is not captured well by simple correlation alone.
5.4.2 Sklar's theorem
Sklar’s theorem states that any multivariate distribution can be expressed in terms of its marginals and an associated copula, and that this representation is unique under suitable continuity conditions. The theorem provides the theoretical foundation for copula methods. It explains why the dependence structure can be studied separately from marginal distributions.
6 Estimation and inference
Estimation and inference address how multidimensional distributions are learned from data and how uncertainty about their features is assessed. Because full joint models can involve many parameters, inference often requires careful balance between flexibility and tractability. Standard methods adapt familiar one-variable ideas to the multivariate setting.
6.1 Parameter estimation
Parameter estimation seeks numerical values for the unknown quantities that define a multivariate model. These may include means, variances, correlations, regression coefficients, or parameters governing dependence. Estimation can be based on likelihood, moments, Bayesian priors, or other criteria.
6.1.1 Maximum likelihood estimation
Maximum likelihood estimation chooses parameters that maximize the probability of the observed data under the proposed model. In multivariate settings, it often yields efficient estimates when the model is correctly specified. The method is widely used because it has a clear interpretation and strong asymptotic properties.
6.1.2 Method of moments
The method of moments matches sample moments to theoretical moments and solves for the unknown parameters. It is conceptually simple and can be easier to compute than maximum likelihood. Although it may be less efficient, it is useful when likelihood calculations are complicated or unavailable.
6.2 Hypothesis testing
Hypothesis testing in multivariate analysis examines claims about means, covariance structures, independence, equality of distributions, or model parameters. Tests may involve vector-valued statistics, matrix comparisons, or transforms of the data. Common objectives include determining whether groups differ across several variables or whether a proposed dependence structure is adequate.
6.3 Confidence regions
A confidence region is a multivariate analogue of a confidence interval. Instead of giving a range for a single parameter, it describes a region in several dimensions that is expected to contain the true parameter vector with a specified probability. Such regions are especially important when parameters are naturally linked, as with a mean vector or regression coefficient set.
6.4 Goodness-of-fit assessment
Goodness-of-fit assessment evaluates how well a proposed multidimensional distribution matches observed data. Methods may compare empirical and theoretical joint behavior, examine residual patterns, or test whether the modeled dependence is adequate. In multivariate problems, fit assessment can be challenging because the number of possible joint patterns grows rapidly with dimension.
7 Applications
Multidimensional distributions are used wherever several variables must be analyzed together. They provide a mathematical framework for understanding complex datasets and for building models that reflect interaction, uncertainty, and structure. Their applications span theoretical and applied statistics as well as many scientific disciplines.
7.1 Multivariate data analysis
Multivariate data analysis studies datasets with multiple measured variables on each observation. It includes techniques for dimension reduction, clustering, visualization, and pattern detection. Multidimensional distributions supply the probabilistic basis for these methods and help explain how observations are scattered through joint variable space.
7.2 Regression and classification
In regression and classification, multivariate distributions describe the relationship between predictors and outcomes, or among multiple response variables. They are used to model residuals, latent components, and class-conditional behavior. Such distributions help determine how prediction uncertainty changes across different combinations of inputs.
7.3 Bayesian statistics
Bayesian statistics relies heavily on joint distributions for priors, likelihoods, and posterior distributions. Multidimensional models naturally express uncertainty about several unknown quantities at once. They are especially useful in hierarchical models, where parameters at one level depend on others and conditional structure is central.
7.4 Stochastic modeling
Stochastic modeling uses random processes and random vectors to represent systems that evolve under uncertainty. Multidimensional distributions appear in transition models, hidden-state systems, and spatial data analysis. They help describe how multiple random quantities are related at a given time or across repeated observations.
7.5 Signal processing
In signal processing, multidimensional distributions are used to model vectors of measurements, noise components, and frequency-domain quantities. They support detection, estimation, filtering, and source separation. Joint probabilistic models are especially important when signals from several channels are correlated or when measurement errors share structure.