1. Definition and Basic Properties
1.1 Sub-Gaussian Random Variables
A random variable is called sub-Gaussian if its tail probabilities decrease at least as rapidly as those of a centered Gaussian variable. Intuitively, sub-Gaussian random variables concentrate strongly around their mean: deviations of size \(t\) become exponentially unlikely as \(t\) grows.
1.1.1 Tail-based definition
A common tail-based definition states that a random variable \(X\) is sub-Gaussian if there exist constants \(c>0\) and \(K>0\) such that for all \(t\ge 0\), \[
| \Pr( | X-\mathbb{E}X | \ge t)\le 2\exp\!\left(-\frac{t^{2}}{K^{2}}\right). |
|---|
\] Here \(K\) controls the “width” of the tails relative to a Gaussian distribution. Smaller \(K\) corresponds to stronger concentration.
1.1.2 Moment generating function (MGF) characterization
An equivalent, often more usable, characterization involves the moment generating function. A random variable \(X\) is sub-Gaussian if there exists \(\sigma>0\) such that for every real \(\lambda\), \[ \mathbb{E}\exp\!\left(\lambda (X-\mathbb{E}X)\right)\le \exp\!\left(\frac{\lambda^{2}\sigma^{2}}{2}\right). \] This inequality mirrors the exact MGF bound of a Gaussian with variance \(\sigma^{2}\). The parameter \(\sigma\) plays the role of an effective standard deviation for concentration purposes.
1.1.3 Equivalent formulations and comparison to Gaussian
Across many treatments, the tail and MGF forms are shown to be equivalent up to constant factors. In practice, “sub-Gaussian parameter” definitions may differ by multiplicative constants depending on conventions (e.g., whether the inequality uses \(\frac{t^2}{K^2}\) or \(\frac{t^2}{2K^2}\), or whether \(X\) is centered). Regardless of convention, the defining feature is the quadratic exponent governing tails, as in the Gaussian case.
1.2 The Sub-Gaussian Norm
The sub-Gaussian norm formalizes the idea of measuring tail decay in a way that behaves well under scaling and vector operations. A standard choice uses an Orlicz norm associated with quadratic growth.
1.2.1 ψ2-Orlicz norm (overview)
Let \(\psi_{2}(u)=\exp(u^{2})-1\). For a (centered) random variable \(X\), define the Orlicz norm \[
| \|X\|_{\psi_{2}}=\inf\left\{s>0:\ \mathbb{E}\exp\!\left(\frac{X^{2}}{s^{2}}\right)\le 2\right\}. |
|---|
\]
| Finiteness of \(\|X\|_{\psi_{2}}\) is equivalent to sub-Gaussianity. The resulting norm provides a convenient quantity for proving concentration inequalities and for tracking how parameters change under operations. |
|---|
1.2.2 Scaling and translation behavior
Sub-Gaussian norms interact predictably with basic transformations:
| - Scaling: For constant \(a\), \(\|aX\|_{\psi_{2}}\) is proportional to \( | a | \,\|X\|_{\psi_{2}}\). |
|---|
- Translation: The sub-Gaussian property typically applies to the centered variable \(X-\mathbb{E}X\). Adding a constant shifts the mean without changing tail decay relative to the mean.
- Centering: Replacing \(X\) by \(X-\mathbb{E}X\) is the usual step that makes the parameter meaningful in the concentration inequalities.
1.2.3 Typical examples and non-examples
- Bounded variables: If \(X\) is almost surely bounded, then \(X-\mathbb{E}X\) is sub-Gaussian with a parameter depending on the range.
- Gaussian variables: A Gaussian \(N(\mu,\sigma^{2})\) is sub-Gaussian with parameter tied to \(\sigma\).
- Rademacher/Bernoulli-type variables: Symmetric bounded distributions often yield finite \(\psi_{2}\) norms.
Non-examples:
- Heavy-tailed laws with polynomial decay (e.g., regularly varying distributions) generally fail sub-Gaussian conditions because their tails decrease much slower than \(\exp(-t^{2})\).
- Distributions whose MGF grows too quickly (or does not exist in a neighborhood of zero) usually cannot satisfy sub-Gaussian bounds.
1.3 Sub-Gaussian Random Vectors
For random vectors, sub-Gaussianity is typically defined through one-dimensional projections: behavior in every direction should resemble sub-Gaussian tails.
1.3.1 Directional sub-Gaussianity
A random vector \(X\in\mathbb{R}^{d}\) is sub-Gaussian if there exists \(\kappa>0\) such that for every unit vector \(u\in\mathbb{R}^{d}\), the projection \(u^{\top}X\) is sub-Gaussian with \[
| \|u^{\top}X\|_{\psi_{2}}\le \kappa. |
|---|
\] This “uniform over directions” requirement prevents anisotropy from hiding heavy tails in some directions.
1.3.2 Covariance structure and isotropy
The covariance matrix \(\Sigma=\mathbb{E}\big[(X-\mathbb{E}X)(X-\mathbb{E}X)^{\top}\big]\) summarizes second-order spread. A vector may be sub-Gaussian yet have nontrivial anisotropy: its tail behavior can scale differently along directions with larger variance. In special cases, one assumes isotropy, meaning \(\mathbb{E}[XX^{\top}]=I\) (for centered \(X\)). Isotropy often simplifies bounds in high-dimensional estimation because it ties the geometry to the identity matrix.
2. Concentration Inequalities
Sub-Gaussianity enables sharp bounds on deviations of random quantities. Many results follow from controlling exponential moments (via MGFs) or equivalently bounding \(\psi_{2}\) norms.
2.1 Hoeffding-Type Bounds
2.1.1 Bounded variables as sub-Gaussian examples
Hoeffding’s inequality applies to bounded independent random variables. Boundedness implies sub-Gaussian tails, which can be viewed as an instance of the more general sub-Gaussian framework. For \(X_i\) independent with \(X_i\in[a_i,b_i]\), the centered sum \(S=\sum_{i=1}^{n}(X_i-\mathbb{E}X_i)\) satisfies \[
| \Pr( | S | \ge t)\le 2\exp\!\left(-\frac{2t^{2}}{\sum_{i=1}^{n}(b_i-a_i)^{2}}\right), |
|---|
\] reflecting Gaussian-like decay.
2.1.2 Practical constants and interpretation
While the algebra differs across sources, the qualitative meaning is consistent: the deviation exponent involves \(t^{2}\) divided by an effective variance proxy determined by ranges or \(\psi_{2}\) parameters. Analysts interpret these bounds in terms of how many samples are needed so that random fluctuations stay within a target tolerance with high probability.
2.2 Chernoff Bounds and Exponential Decay
Chernoff-style arguments derive tail inequalities from exponential moment bounds, frequently using the inequality \[ \Pr(Y\ge t)\le e^{-\lambda t}\mathbb{E}e^{\lambda Y}\quad(\lambda>0). \]
2.2.1 From MGF control to tail bounds
If \(X-\mathbb{E}X\) has an MGF bounded by \(\exp(\lambda^{2}\sigma^{2}/2)\), then optimizing over \(\lambda\) yields \[ \Pr(X-\mathbb{E}X\ge t)\le \exp\!\left(-\frac{t^{2}}{2\sigma^{2}}\right), \] and similarly for the lower tail. For sums of independent sub-Gaussians, the same logic leads to exponentially decaying bounds with a variance-like term that adds across summands.
2.2.2 Symmetry and centering effects
Centering is essential: without subtracting \(\mathbb{E}X\), sub-Gaussian behavior pertains to the fluctuation around the mean. In symmetric distributions, some results simplify because upper and lower tails match; however, most concentration inequalities are stated for absolute deviations and thus apply regardless of symmetry.
2.3 Concentration for Sums
Sub-Gaussianity is particularly valuable when analyzing sums because independent sub-Gaussians add in a controlled way.
2.3.1 Independent sub-Gaussian sums
Let \(X_1,\dots,X_n\) be independent, centered sub-Gaussian variables with parameters \(\sigma_i\), meaning \[ \mathbb{E}e^{\lambda X_i}\le e^{\lambda^{2}\sigma_i^{2}/2}. \] Then for \(S=\sum_{i=1}^{n}X_i\), \[ \mathbb{E}e^{\lambda S}\le \exp\!\left(\frac{\lambda^{2}}{2}\sum_{i=1}^{n}\sigma_i^{2}\right), \] so \(S\) is sub-Gaussian with parameter \(\sqrt{\sum_{i=1}^{n}\sigma_i^{2}}\). Consequently, \[
| \Pr( | S | \ge t)\le 2\exp\!\left(-\frac{t^{2}}{2\sum_{i=1}^{n}\sigma_i^{2}}\right), |
|---|
\] up to convention-dependent constants.
2.3.2 Bernstein-style refinements (sub-Gaussian regime)
Classical Bernstein inequalities incorporate both variance and tail heaviness. In the sub-Gaussian regime, the variance term effectively dominates in the tail range where deviations are not extremely large, producing bounds that behave similarly to Gaussian tails. This yields sharper estimates than purely variance-based inequalities and offers smooth transitions between “moderate” and “large” deviations depending on additional moment information.
3. Operations and Stability Results
Understanding how sub-Gaussianity behaves under algebraic operations is crucial for applications involving transformations, statistics, and estimators.
3.1 Linear Transformations
Sub-Gaussian structure is preserved under many linear operations, including those used to form estimators.
3.1.1 Scalar multiplication and addition
For centered sub-Gaussian variables, scaling by a constant multiplies the sub-Gaussian parameter accordingly. Adding independent centered sub-Gaussians yields a new sub-Gaussian variable with a parameter determined by the aggregate of individual parameters (in a root-sum-squares manner for the common MGF/variance proxy interpretation).
3.1.2 Linear combinations of sub-Gaussians
Given independent centered sub-Gaussian variables \(X_i\) and coefficients \(a_i\), the linear combination \(\sum_i a_i X_i\) is sub-Gaussian. Its parameter depends on \(\left(\sum_i a_i^{2}\sigma_i^{2}\right)^{1/2}\) when using MGF-based definitions with variance proxies \(\sigma_i^{2}\). This parallels the behavior of Gaussian linear combinations, reinforcing the “Gaussian-like” analogy.
3.1.3 Matrix-valued transformations (random vectors)
| For a sub-Gaussian vector \(X\), a deterministic matrix \(A\) transforms it into \(AX\). Provided \(X\) is sub-Gaussian in every direction, \(AX\) remains sub-Gaussian. The effective parameter scales with operator norms of \(A\) and the directional control of \(X\), often summarized through bounds involving \( \|A\| \) or related matrix norms. |
|---|
3.2 Sums, Products, and Closures
3.2.1 Closure under independence vs. dependence caveats
Independence plays a central role. Many closure properties (such as clean addition of variance proxies) assume independent components. Under dependence, sub-Gaussianity can fail or require more sophisticated conditions (e.g., mixing assumptions or joint tail controls). As a result, proofs often emphasize which variables are independent and which are merely correlated.
3.2.2 How products generally differ from sub-Gaussian behavior
Unlike sums, products can lead to heavier tails. Even if \(X\) and \(Y\) are sub-Gaussian, the product \(XY\) is not automatically sub-Gaussian; at best it may be sub-exponential or fall into a different tail-decay class. This distinction guides modelers: products of random signals often require different concentration tools than sums of random variables.
3.3 Conditional and Martingale Settings
Some applications require concentration when random variables are revealed sequentially or are adaptively chosen.
3.3.1 Conditionally sub-Gaussian increments
A common framework assumes that, given past information, increments behave like sub-Gaussian variables with conditional parameter bounds. Formally, one controls exponential moments conditionally on a filtration. This extends concentration from static independent settings to adaptive procedures such as bandit-like sampling patterns (in broad terms).
3.3.2 Azuma-Hoeffding type martingale extensions
Martingale concentration inequalities generalize Hoeffding-type bounds. In the sub-Gaussian increment setting, analogous results yield high-probability bounds for martingales whose differences have sub-Gaussian conditional moments. The resulting inequalities track cumulative “conditional variance” over time, reflecting how uncertainty accumulates along the filtration.
4. Relationships to Other Classes
Sub-Gaussianity is one member of a larger family of tail-decay concepts used to calibrate concentration behavior.
4.1 Sub-Exponential vs. Sub-Gaussian
4.1.1 Tail growth intuition
Sub-exponential variables have tails that decay at least like \(\exp(-t)\) (up to constants), rather than \(\exp(-t^2)\). The squared exponent is the signature of sub-Gaussian tails, indicating stronger concentration for moderate to large deviations.
4.1.2 Typical implications between classes
Every sub-Gaussian variable is also sub-exponential, because \(\exp(-t^2)\) dominates \(\exp(-t)\) for sufficiently large \(t\). However, the reverse is false: sub-exponential distributions can have tails too heavy to satisfy the quadratic exponent required for sub-Gaussianity.
4.2 Heavy-Tailed Contrasts
4.2.1 Why power-law tails are not sub-Gaussian
| Distributions with power-law tails behave roughly like \(\Pr( | X | >t)\propto t^{-\alpha}\). Such decay is much slower than Gaussian-like \(\exp(-t^2)\), so the defining inequalities fail for large \(t\). Consequently, attempts to use sub-Gaussian concentration for heavy-tailed data can seriously understate the probability of extreme deviations. |
|---|
4.2.2 Moment growth comparisons
| Another contrast appears in moment growth. Sub-Gaussian variables exhibit controlled growth of moments of the form \((\mathbb{E} | X | ^{p})^{1/p}\) roughly proportional to \(\sqrt{p}\) for large \(p\). Heavy-tailed variables often show faster growth with \(p\), signaling a lack of exponential tail decay. |
|---|
4.3 Sub-Weibull and Generalizations (brief)
4.3.1 Placement in the tail-decay taxonomy
Sub-Weibull classes generalize the interpolation between sub-Gaussian and sub-exponential behavior by allowing tail decay comparable to \(\exp(-t^{\theta})\) for \(\theta\in(0,1]\). In this taxonomy:
- \(\theta=2\) corresponds to sub-Gaussian (quadratic exponent),
- \(\theta=1\) corresponds to sub-exponential (linear exponent),
- smaller \(\theta\) yields heavier-than-sub-exponential but still better than many purely polynomial regimes.
5. Applications and Use Cases
Sub-Gaussian models appear wherever analysts need reliable, non-asymptotic control of random fluctuations.
5.1 High-Dimensional Statistics
5.1.1 Concentration of sample means
In many statistical settings, sample means are sums of independent (or nearly independent) observations. If each observation is sub-Gaussian, then the mean concentrates around its expectation with variance-like scaling \(1/n\). This underpins finite-sample confidence bounds and guarantees for estimators in high-dimensional regimes where asymptotics may be less reliable.
5.1.2 Error bounds in regression and estimation
Linear regression and related estimation problems often rely on concentration for quantities like empirical Gram matrices or covariances. When features are sub-Gaussian, one obtains high-probability bounds for operator norms and restricted eigenvalues, which in turn support guarantees for parameter recovery, sparsity methods, and stability of optimization procedures.
5.2 Machine Learning and Learning Theory
5.2.1 Generalization via concentration
Learning-theoretic arguments often involve controlling the deviation between empirical performance and expected performance. Sub-Gaussian tail behavior allows sharper concentration for losses or stochastic gradients (under appropriate assumptions), improving bounds on generalization error with explicit dependence on sample size.
5.2.2 Robustness and noise modeling (sub-Gaussian assumption)
Modeling observation noise as sub-Gaussian is common because it captures “light-tailed” randomness: outliers exist but are exponentially rare compared with Gaussian scaling. This assumption yields predictable performance for algorithms that depend on least-squares-like objectives or that aggregate random signals through averaging.
5.3 Randomized Algorithms and Numerical Methods
5.3.1 Monte Carlo concentration insights
Monte Carlo estimates are averages of random draws. Sub-Gaussianity provides exponential tail bounds for estimator error, enabling statements about how quickly the probability of large error decays with the number of samples.
5.3.2 Sketching and dimensionality reduction heuristics
Sketching methods often rely on concentration for random linear measurements. When the measurement operator has sub-Gaussian rows or columns (or when its action on fixed vectors is sub-Gaussian), the method can preserve geometry with high probability. This supports guarantees for approximate matrix multiplication, compressed sensing-like workflows, and fast linear algebra primitives.
6. Worked Examples
6.1 Gaussian Variables
6.1.1 Computing the sub-Gaussian parameter
For \(X\sim N(\mu,\sigma^{2})\), consider the centered variable \(X-\mu\). Its MGF is \[ \mathbb{E}e^{\lambda(X-\mu)}=\exp\!\left(\frac{\lambda^{2}\sigma^{2}}{2}\right). \] Comparing with the defining MGF inequality shows that \(X-\mu\) is sub-Gaussian with parameter \(\sigma\) under the standard convention.
6.2 Bounded and Centered Distributions
6.2.1 Uniform and Bernoulli examples
- Uniform: If \(X\) is uniformly distributed on \([a,b]\), then \(X-\mathbb{E}X\) is bounded and therefore sub-Gaussian. Its parameter depends on the interval length and can be derived using Hoeffding-type arguments or direct MGF bounds.
- Bernoulli/Rademacher: For a centered Bernoulli-type variable (e.g., \(X\in\{\pm 1\}\) with equal probability), tails match those of a bounded symmetric distribution. Such variables have finite \(\psi_{2}\) norm and yield clean concentration for sums.
6.2.2 Computing/estimating ψ2 values
| Exact \(\psi_{2}\) norms are rarely needed. Instead, one uses inequalities that relate \(\|X\|_{\psi_{2}}\) to tail bounds or to moments. For bounded \(X\), the \(\psi_{2}\) norm scales with the magnitude of the bound (up to constants), enabling parameter estimates that are sufficient for concentration applications. |
|---|
6.3 Sums of Independent Variables
6.3.1 Deriving a concrete tail bound
Suppose \(X_1,\dots,X_n\) are independent, centered sub-Gaussian with parameters \(\sigma_i\). For \(S=\sum_i X_i\), \[
| \Pr( | S | \ge t)\le 2\exp\!\left(-\frac{t^{2}}{2\sum_{i=1}^{n}\sigma_i^{2}}\right). |
|---|
\] This bound can be specialized to the homogenous case \(\sigma_i=\sigma\), giving \[
| \Pr( | S | \ge t)\le 2\exp\!\left(-\frac{t^{2}}{2n\sigma^{2}}\right). |
|---|
\] Such formulas translate sub-Gaussian parameters directly into deviation probabilities.
6.3.2 Interpreting scaling with sample size
For a sample mean \(\bar{X}= \frac{1}{n}\sum_{i=1}^{n}X_i\), the deviation \( \bar{X}-\mathbb{E}\bar{X} \) satisfies \[
| \Pr( | \bar{X}-\mathbb{E}X | \ge \varepsilon)\le 2\exp\!\left(-\frac{n\varepsilon^{2}}{2\sigma^{2}}\right) |
|---|
\] in the equal-parameter case. The exponent grows linearly in \(n\), reflecting how averaging reduces randomness at a Gaussian-like rate.