1. Definition and Intuition
1.1 Conditioning as “known information”
Conditional expectation formalizes the idea of averaging a random quantity after restricting attention to the outcomes compatible with some information. Instead of averaging over all outcomes at once, the averaging is performed with respect to the probability structure induced by the conditioning information.
1.1.1 Random variables as functions of outcomes
A random variable can be viewed as a function that assigns a numerical value to each outcome in a probability space. When one “conditions” on information, the goal is to describe how the values of another random variable behave on the subset of outcomes consistent with that information.
1.1.2 Averaging with respect to a conditional distribution
If the conditioning information is associated with a rule that specifies how probabilities change after observing it, then conditional expectation is the resulting average. Conceptually, it replaces the unconditional distribution of the original random variable with a conditional distribution, and then computes an expectation using that updated distribution.
1.2 Formal Definition (Measure-Theoretic)
The most general definition is framed using σ-algebras, which represent collections of events treated as “measurable” given the information.
1.2.1 σ-algebras and measurability
Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space, let \(X\) be an integrable random variable, and let \(\mathcal{G}\subseteq \mathcal{F}\) be a σ-algebra representing the known information. The conditional expectation \( \mathbb{E}[X\mid \mathcal{G}] \) is required to be \(\mathcal{G}\)-measurable, meaning it can be determined using only information in \(\mathcal{G}\).
1.2.2 Existence and uniqueness (almost surely)
Under standard assumptions (notably integrability of \(X\)), a conditional expectation exists. It is unique up to sets of probability zero, so two versions may differ only on outcomes that almost never occur.
1.2.3 The defining property via integrals
The defining characterization is that for every event \(A\in\mathcal{G}\), \[ \int_A \mathbb{E}[X\mid \mathcal{G}]\,d\mathbb{P}=\int_A X\,d\mathbb{P}. \] Equivalently, conditional expectation is the \(\mathcal{G}\)-measurable random variable whose integrals over all \(\mathcal{G}\)-events match those of \(X\).
1.3 Discrete vs. Continuous Cases
In applied settings, conditional expectation is often computed using formulas that depend on whether the conditioning variable is discrete, continuous, or mixes both.
1.3.1 Discrete conditioning (sums)
When conditioning on an event \(A\) with \(\mathbb{P}(A)>0\), the conditional expectation reduces to an average over the conditional distribution: \[ \mathbb{E}[X\mid A]=\frac{\mathbb{E}[X\mathbf{1}_A]}{\mathbb{P}(A)}. \] For conditioning on a discrete random variable \(Y\) taking values \(\{y_k\}\), one typically uses \[ \mathbb{E}[X\mid Y]=\sum_k \mathbb{E}[X\mid Y=y_k]\mathbf{1}_{\{Y=y_k\}}. \]
1.3.2 Continuous conditioning (integrals)
For continuous conditioning, conditional expectation is computed by integrating against conditional densities. If \((X,Y)\) has a joint density and the conditional density \(f_{X\mid Y}(x\mid y)\) is defined, then \[ \mathbb{E}[X\mid Y=y]=\int x\, f_{X\mid Y}(x\mid y)\,dx. \] The resulting conditional expectation as a random variable is obtained by substituting the observed value \(Y(\omega)\) into the function.
1.3.3 Conditional density and regular conditional probabilities
In many problems, conditional densities are used as heuristics. The measure-theoretic framework replaces them with regular conditional probabilities (RCPs), which provide a rigorous way to assign conditional probabilities given continuous information. Regular conditional probabilities lead to well-defined conditional expectations even when density formulas are delicate.
2. Core Properties
2.1 Linearity and Basic Rules
Conditional expectation behaves like an averaging operator, and it inherits several algebraic properties from integration.
2.1.1 Linearity under integrability
If \(X\) and \(Z\) are integrable, and \(a,b\) are constants, then \[ \mathbb{E}[aX+bZ\mid \mathcal{G}] = a\,\mathbb{E}[X\mid \mathcal{G}] + b\,\mathbb{E}[Z\mid \mathcal{G}], \] provided the expressions are well-defined.
2.1.2 Pulling out known terms
If \(W\) is \(\mathcal{G}\)-measurable and integrable together with \(X\), then \[ \mathbb{E}[WX\mid \mathcal{G}] = W\,\mathbb{E}[X\mid \mathcal{G}]. \] This reflects that factors determined by the known information can be treated as constants inside the conditional averaging.
2.2 Tower Property (Law of Total Expectation)
A central principle is that conditioning in stages yields the same result as conditioning directly.
2.2.1 Iterated conditioning
If \(\mathcal{H}\subseteq \mathcal{G}\), then \[ \mathbb{E}[\mathbb{E}[X\mid \mathcal{G}]\mid \mathcal{H}] = \mathbb{E}[X\mid \mathcal{H}]. \] This expresses consistency under refinement or coarsening of information.
2.2.2 Total expectation with nested information
A commonly used special case takes expectations of both sides to recover the law of total expectation: \[ \mathbb{E}[\mathbb{E}[X\mid \mathcal{G}]]=\mathbb{E}[X]. \] Thus, conditional expectation is an averaging “inside” the overall expectation rather than an alternative notion of mean.
2.3 Monotonicity and Jensen-Type Results
Conditional expectation preserves inequalities and interacts naturally with convex functions.
2.3.1 Monotone transformations
If \(\phi\) is increasing and integrable conditions hold appropriately, then ordering properties can be transferred through conditional expectation. A basic form is that if \(X\le Z\) almost surely, then \(\mathbb{E}[X\mid \mathcal{G}] \le \mathbb{E}[Z\mid \mathcal{G}]\) almost surely.
2.3.2 Convexity and inequalities
Jensen’s inequality has a conditional counterpart: for convex \(\phi\), \[ \phi(\mathbb{E}[X\mid \mathcal{G}]) \le \mathbb{E}[\phi(X)\mid \mathcal{G}] \] almost surely, under standard integrability assumptions. This is used to bound nonlinear transformations by conditioning.
2.4 Conditional Expectation as an Orthogonal Projection
Conditional expectation also admits a geometric interpretation, which explains its optimality properties in estimation.
2.4.1 Hilbert space viewpoint
When working in \(L^2\) (square-integrable random variables), the set of \(\mathcal{G}\)-measurable variables forms a closed subspace. The conditional expectation \( \mathbb{E}[X\mid \mathcal{G}] \) is the orthogonal projection of \(X\) onto this subspace.
2.4.2 Minimizing mean squared error
As an immediate consequence, \( \mathbb{E}[X\mid \mathcal{G}] \) is the best predictor of \(X\) among all \(\mathcal{G}\)-measurable random variables under squared loss. In other words, it minimizes \[ \mathbb{E}\big[(X - Z)^2\big] \] over all \(Z\) measurable with respect to \(\mathcal{G}\).
2.5 Preservation and Consistency
Conditional expectation respects the information already known.
2.5.1 If already measurable, conditioning changes nothing
If \(X\) is \(\mathcal{G}\)-measurable, then \[ \mathbb{E}[X\mid \mathcal{G}] = X \] almost surely. Intuitively, no extra averaging is required because \(X\) is already determined by the conditioning information.
2.5.2 Compatibility across σ-algebras
When conditioning is changed from one σ-algebra to another, the tower property guarantees coherence. This makes conditional expectation compatible with hierarchical information structures used in probability and statistics.
3. Connections to Conditional Distributions
3.1 Conditional Expectation via Conditional PMF/PDF
In many elementary derivations, conditional expectation is expressed using conditional probability mass or density functions.
3.1.1 Expectation under conditional distributions
Given a conditional PMF \(p_{X\mid Y}(x\mid y)\) (discrete) or conditional PDF \(f_{X\mid Y}(x\mid y)\) (continuous), the conditional mean is computed by integrating or summing \(x\) against that conditional distribution: \[ \mathbb{E}[X\mid Y=y] = \begin{cases} \sum_x x\, p_{X\mid Y}(x\mid y), & \text{discrete},\\ \int x\, f_{X\mid Y}(x\mid y)\,dx, & \text{continuous}. \end{cases} \]
3.1.2 Conditioning on events
If the conditioning is an event \(A\), the conditional distribution is the distribution of \(X\) restricted to \(A\). Conditional expectation becomes the mean under that restricted distribution: \[ \mathbb{E}[X\mid A]=\mathbb{E}[X\mathbf{1}_A]/\mathbb{P}(A), \] when \(\mathbb{P}(A)>0\).
3.2 Regular Conditional Probability
Regular conditional probability provides a unified framework for conditioning on general information, especially continuous variables.
3.2.1 Existence and practical use
Under standard regularity assumptions on the underlying space, one can construct an RCP \(P(X\in\cdot \mid Y=y)\). This yields a rigorous definition of conditional expectation by integrating \(X\) with respect to that RCP.
3.2.2 Computation in common models
In familiar models (e.g., bivariate normal, many exponential family settings), conditional distributions take a known functional form. Conditional expectation can then be computed as the mean of that conditional distribution without revisiting measure-theoretic subtleties.
3.3 Transformations and Conditioning Variables
Conditioning can be performed on random variables or their functions, leading to different but related conditional expectations.
3.3.1 Conditioning on another random variable
When conditioning on a random variable \(Y\), the conditional expectation \(\mathbb{E}[X\mid Y]\) is itself a function of \(Y\). It can be defined by requiring measurability with respect to the σ-algebra generated by \(Y\).
3.3.2 Functions of conditioned variables
If one conditions on \(g(Y)\) instead of \(Y\), the resulting conditional expectation generally changes because the σ-algebra becomes coarser. The tower property provides a link: \[ \mathbb{E}[X\mid g(Y)] = \mathbb{E}\big[\mathbb{E}[X\mid Y]\mid g(Y)\big]. \]
4. Computation Techniques
4.1 Conditioning on Finite Partitions
A practical route to computation is to condition on a finite collection of events that partition the sample space.
4.1.1 Event-based averaging
Suppose \(\{A_1,\dots,A_n\}\) partitions \(\Omega\) and \(X\) is integrable. Then \(\mathbb{E}[X\mid \sigma(A_1,\dots,A_n)]\) is a random variable taking value \(\mathbb{E}[X\mid A_i]\) on \(A_i\). Each component is computed as a conditional mean over the corresponding event.
4.1.2 Implementation in examples
In categorical data settings, this approach often matches the intuitive “group average” idea: condition on the observed category, compute the mean within that group, and then assemble those group means into a function of the conditioning variable.
4.2 Using Conditional Densities
When densities exist, conditional expectation can be computed using conditional PDFs and Bayes-type relationships.
4.2.1 Deriving from Bayes’ rule
Given joint density \(f_{X,Y}\) and marginal density \(f_Y\), one has \[ f_{X\mid Y}(x\mid y) = \frac{f_{X,Y}(x,y)}{f_Y(y)} \] where \(f_Y(y)>0\). Substituting into the integral defining \(\mathbb{E}[X\mid Y=y]\) gives a computable expression.
4.2.2 Handling edge cases (support issues)
Conditional densities are typically defined only where the conditioning variable has positive probability density or mass. When \(f_Y(y)=0\) (or the conditioning event has probability zero), conditional expectations are defined through measure-theoretic limits or regular conditional probabilities, so formulas may require careful interpretation.
4.3 Using Independence
Independence can greatly simplify conditional expectations.
4.3.1 Independence leading to simplifications
If \(X\) is independent of the σ-algebra \(\mathcal{G}\), then conditioning does not change the mean: \[ \mathbb{E}[X\mid \mathcal{G}] = \mathbb{E}[X] \] almost surely. Intuitively, the available information provides no information about \(X\).
4.3.2 Conditional independence interpretations
In more complex models, \(X\) may be independent of part of the information given another part. This yields conditional factorization properties, which can be combined with the tower property to compute conditional expectations in stages.
4.4 Least Squares Regression Interpretation
Conditional expectation is connected to regression as the optimal mean under squared error.
4.4.1 Linear projection vs. general conditional mean
In linear regression, the fitted value is the best approximation of \(X\) within a linear subspace. The conditional expectation \(\mathbb{E}[X\mid \mathcal{G}]\) is the best approximation within the entire space of \(\mathcal{G}\)-measurable functions. Regression models often approximate the latter when the full conditional mean is not directly available.
4.4.2 Practical estimator viewpoint
From an estimation perspective, conditional expectation represents the target function that an algorithm attempts to learn or approximate from data, typically by minimizing a loss that encourages squared-error accuracy.
5. Special Theorems and Results
5.1 Law of Total Variance (Variance Decomposition)
Conditional expectation enables an exact decomposition of variance into explained and unexplained parts.
5.1.1 Conditional variance + variance of conditional mean
For integrable second moments, \[ \mathrm{Var}(X)=\mathbb{E}[\mathrm{Var}(X\mid \mathcal{G})]+\mathrm{Var}(\mathbb{E}[X\mid \mathcal{G}]). \] The first term captures variability remaining after conditioning, while the second term captures variability due to differences across the conditional means.
5.2 Martingale Relationship
Conditional expectation forms the foundation of martingale theory.
5.2.1 Conditional expectation as martingale update
Given a filtration \((\mathcal{F}_n)\), a process \((M_n)\) is a martingale if \[ \mathbb{E}[M_{n+1}\mid \mathcal{F}_n]=M_n. \] This characterization means the next-step expected value, given the past, equals the present value—an idea powered by conditional expectation.
5.2.2 Optional stopping (overview level)
Martingales interact with stopping times through conditional expectation identities. While the details depend on integrability conditions, the general theme is that, under suitable assumptions, expected martingale values remain stable across carefully controlled stopping rules.
5.3 Iterated Expectation and Bayesian Updating
Conditional expectation appears naturally in Bayesian inference, where updating is performed by conditioning on observed data.
5.3.1 Posterior expectation forms
If prior beliefs yield a joint model for parameter \(\theta\) and data \(D\), then conditioning on the observed data produces a posterior distribution. Posterior summaries such as \(\mathbb{E}[\theta\mid D]\) are conditional expectations with respect to that posterior.
5.3.2 Updating with new information
When new data arrive in stages, iterated conditioning expresses how posterior expectations evolve: \[ \mathbb{E}[\theta\mid D_1, D_2] = \mathbb{E}\big[\mathbb{E}[\theta\mid D_1, D_2]\mid D_1\big] \] in a way consistent with the tower property. This provides a formal justification for sequential updating procedures.
6. Applications in Statistics
6.1 Forecasting and Prediction
In prediction tasks, conditional expectation often emerges as the optimal forecast under common loss functions.
6.1.1 Best predictor under squared loss
For predicting a target \(X\) given information \(\mathcal{G}\), the conditional mean \(\mathbb{E}[X\mid \mathcal{G}]\) minimizes expected squared prediction error among all \(\mathcal{G}\)-measurable predictors.
6.1.2 Forecast error and conditional variance
The residual uncertainty after conditioning is summarized by \(\mathrm{Var}(X\mid \mathcal{G})\). Larger conditional variance indicates less predictable behavior even after observing the information in \(\mathcal{G}\).
6.2 Regression and Generalized Linear Models (Conceptual)
Regression methods can be interpreted through conditional expectations as targets.
6.2.1 Conditional mean as target
Many regression frameworks aim to model \( \mathbb{E}[Y\mid X] \) where \(Y\) is a response and \(X\) collects predictors. Even when modeling choices are parametric, the fundamental quantity being approximated is a conditional mean.
6.2.2 Residual interpretation via conditioning
In idealized settings, conditional expectation helps separate systematic signal (captured by the conditional mean) from random fluctuation (captured by the conditional variance). This view clarifies why residuals often behave like noise after accounting for the predictors.
6.3 Bayesian Methods (Expectation Under Posterior)
Bayesian analysis treats posterior quantities as conditional expectations.
6.3.1 Posterior predictive expectations
Posterior predictive distributions average over uncertainty in parameters. Means computed under these predictive distributions are expectations conditional on observed data, integrating parameter uncertainty in a coherent way.
6.3.2 Credible quantities from conditional expectations
Many credible summaries—such as posterior means used as point estimates—are direct conditional expectations of the form \(\mathbb{E}[\varphi(\theta)\mid D]\), where \(\varphi(\theta)\) is a function of the parameter.
6.4 Decision Making Under Uncertainty
Decision rules depend on how expected outcomes change after incorporating information.
6.4.1 Expected utility conditioning
When utilities are random through uncertain variables, optimal decisions under risk can involve conditional expected utilities such as \(\mathbb{E}[U(a,\theta)\mid \mathcal{G}]\), where \(a\) is an action.
6.4.2 Risk and information structure
How information is represented by a σ-algebra affects conditional expectations and thus decision criteria. The tower property ensures that risk assessments remain consistent when information is processed in stages.
7. Examples
7.1 Conditioning on a Single Event
Conditioning on an event provides a clear illustration of the “average over compatible outcomes” idea.
7.1.1 Finite probability table example
Suppose a random variable \(X\) takes values \(\{0,1,2\}\) and a related event \(A\) occurs with positive probability. A finite probability table gives \(\mathbb{P}(X=x, A)\) for each \(x\). Then \[ \mathbb{E}[X\mid A]=\sum_x x\;\frac{\mathbb{P}(X=x, A)}{\mathbb{P}(A)}. \] This produces a single numeric mean summarizing \(X\) restricted to \(A\).
7.1.2 Interpreting the conditioned mean
The conditional mean represents the expected value of \(X\) after learning that the event \(A\) has occurred. If the conditioned mean differs from \(\mathbb{E}[X]\), then \(A\) provides information about \(X\).
7.2 Conditioning on a Discrete Random Variable
When the conditioning variable is categorical, conditional expectation becomes a function of the category.
7.2.1 Conditioning on categories
Let \(Y\) be a discrete random variable with categories \(\{1,2,3\}\). For each category \(k\), compute \(\mathbb{E}[X\mid Y=k]\). The conditional expectation as a function of \(Y\) is then \[ \mathbb{E}[X\mid Y]=\mathbb{E}[X\mid Y=1]\mathbf{1}_{\{Y=1\}}+\mathbb{E}[X\mid Y=2]\mathbf{1}_{\{Y=2\}}+\mathbb{E}[X\mid Y=3]\mathbf{1}_{\{Y=3\}}. \]
7.2.2 Conditional expectation as a function
This construction yields a random variable whose value depends on the realized category. In data analysis, it corresponds to “group means” that are evaluated at the observed group label.
7.3 Conditioning in Jointly Distributed Continuous Variables
Continuous conditioning is often illustrated using multivariate distributions with tractable conditional forms.
7.3.1 Bivariate normal example
If \((X,Y)\) is jointly normal with means \(\mu_X,\mu_Y\), variances \(\sigma_X^2,\sigma_Y^2\), and correlation \(\rho\), then the conditional distribution of \(X\) given \(Y=y\) is normal. Its mean is an affine function of \(y\), commonly written as \[ \mathbb{E}[X\mid Y=y]=\mu_X+\rho\frac{\sigma_X}{\sigma_Y}(y-\mu_Y), \] and its conditional variance is constant with respect to \(y\).
7.3.2 Interpreting conditional mean and variance
The conditional mean describes how the expected value of \(X\) shifts when \(Y\) takes a particular value. The conditional variance indicates how much uncertainty remains about \(X\) after observing \(Y\); in the bivariate normal case, that residual uncertainty does not depend on the observed \(y\).