1 Definition and Intuition

1.1 Expected value as a weighted average

Expected value is a way to summarize a random outcome by averaging outcomes with weights given by their probabilities. If a random variable \(X\) can take values \(x\), then values that occur more frequently (higher probability) influence the average more strongly. This makes expected value a natural “average outcome” measure for uncertain quantities.

1.2 Long-run average interpretation

When a random experiment is repeated many times under identical conditions, the average of the observed values tends to approach a stable number. That stable number is the expected value, provided the expectation exists. This interpretation is particularly intuitive for “repeated play” scenarios such as games, sampling procedures, or repeated measurements.

1.3 Discrete vs. continuous formulations

In the discrete case, expected value is computed via a sum over all possible outcomes, each multiplied by its probability. In the continuous case, outcomes are not enumerated individually; instead, expected value is computed by integrating the product of the variable with its probability density.

1.4 Conditions for existence (finite vs. infinite expectation)

An expectation may fail to be finite. For some random variables, the weighted average does not converge to a finite number, leading to an infinite expectation (or an undefined expectation in certain setups). Many results in probability assume integrability—roughly, that enough “mass” is not too heavy in the tails to break the average.

2 Mathematical Formulation

2.1 Expected value of discrete random variables

Let \(X\) be a discrete random variable taking values \(x_i\) with probabilities \(P(X=x_i)\). The expected value is \[ E[X] = \sum_i x_i \, P(X=x_i). \] This expression adds the contribution of each possible value, scaled by how likely it is.

2.1.1 Summation over probabilities

The summation is taken over the support of \(X\). If the series does not converge absolutely, the expectation may not exist as a finite quantity, depending on the behavior of positive and negative parts.

2.2 Expected value of continuous random variables

If \(X\) is continuous with probability density function \(f(x)\), the expected value is \[ E[X] = \int_{-\infty}^{\infty} x\, f(x)\, dx \] whenever the integral is well-defined.

2.2.1 Integration with probability density functions

The density \(f(x)\) describes relative likelihood across values. Multiplying by \(x\) and integrating accumulates the weighted contributions of all possible outcomes.

2.3 Expected value with general probability measures

Probability models can be represented in a unified way using probability measures. For a random variable \(X\) defined on a probability space, expectation can be expressed as an integral with respect to the underlying probability measure \(P\).

2.3.1 Expectation as an integral

In general, \[ E[X] = \int X \, dP, \] provided the integral exists (typically in the Lebesgue sense). This framework handles discrete, continuous, and more complex mixed models uniformly.

2.4 Notation and terminology

Common notation includes \(E[X]\) for the expected value of \(X\), with variants such as \(\mathbb{E}[X]\) in mathematical texts. The terms “expectation” and “mean” are closely related in many statistical contexts, particularly when discussing random variables with finite first moments.

3 Properties of Expected Value

3.1 Linearity of expectation

A central feature of expected value is its linear behavior under addition and scaling, without requiring independence.

3.1.1 additivity: E[X+Y] = E[X] + E[Y]

For integrable random variables \(X\) and \(Y\), \[ E[X+Y] = E[X] + E[Y]. \] This remains true even if \(X\) and \(Y\) are dependent, as long as expectations exist.

3.1.2 scaling: E[aX] = aE[X]

For a constant \(a\), \[ E[aX] = aE[X]. \] Together with additivity, this establishes full linearity for affine expressions when the expectations are finite.

3.2 Expectation of constants

If \(c\) is a constant, then \[ E[c] = c. \] More generally, adding a constant shifts the expectation by that constant: \(E[X+c]=E[X]+c\).

3.3 Monotonicity and bounds

If \(X \le Y\) almost surely and both expectations exist (or at least \(E[Y]\) is not problematic), then \(E[X] \le E[Y]\). This monotonicity supports bounding arguments and comparison of models.

3.4 Invariance under almost-sure equality

If two random variables are equal with probability 1 (they differ only on events of probability zero), they have the same expectation. This reflects that expectation depends on the probability law rather than on pointwise values on null sets.

3.5 Relation to conditional expectation (preview)

Conditional expectation refines expected value by conditioning on some information or event. It serves as the key bridge between raw averages and knowledge-driven predictions, often summarized by the tower property in subsequent sections.

4 Computation Techniques

4.1 Using distributions directly

The most direct method uses the definition: compute \(E[X]\) from the distribution of \(X\).

4.1.1 Tables and probability mass functions

For small discrete models, one can enumerate each outcome and calculate \(\sum_i x_i P(X=x_i)\). In practice, tables of outcomes and probabilities often make this computation systematic and reduce algebraic errors.

4.2 Using transformations of random variables

If \(Y=g(X)\), then \(E[Y]=E[g(X)]\). The idea is to compute expectation of a transformed variable directly, either by re-expressing \(g(X)\) in terms of known probabilities or by identifying a convenient distribution for \(Y\).

4.2.1 E[g(X)] for common transforms

For certain families, transforms preserve tractability: for example, linear transforms \(aX+b\) yield \(E[aX+b]=aE[X]+b\). More complex transforms may require summation or integration tailored to the form of \(g\).

4.3 Expected value via conditioning

Conditioning can simplify computation by breaking a complex problem into conditional pieces that are easier to evaluate.

4.3.1 Law of total expectation (tower property)

A common identity is \[ E[X] = E\bigl(E[X\mid \mathcal{G}]\bigr), \] where \(\mathcal{G}\) represents available information (often a sigma-algebra). In discrete settings, the same idea appears as a weighted average over conditioning events, commonly called the law of total expectation.

4.4 Handling mixed discrete-continuous variables

Some random variables have both a point-mass component (probability at specific values) and a continuous density on intervals. Expected value can be computed by adding the contributions from the discrete atoms and the continuous part, each weighted appropriately. This decomposition follows from integrating over the full probability measure.

5 Examples and Applications

5.1 Fair games and expected winnings

In a fair game, the expected net gain per play is zero. Expected value then provides a quantitative test for fairness: if \(E[\text{winnings}]=0\), the game is fair under the assumed probability model. If the expectation is positive, the player benefits in the long run; if negative, losses accumulate on average.

5.2 Dice, cards, and simple gambling models

5.2.1 Expected value of draws and payouts

Consider a game with a payout rule tied to a draw result. If you can list the payout \(p_i\) for each outcome \(i\) and its probability \(q_i\), then the expected payout is \(E[p]=\sum_i p_i q_i\). This approach extends to multiple draws where outcomes are grouped by sufficient summaries such as counts or sums.

5.3 Expectations in everyday uncertainty (lightweight intuition)

Expected value is not only for gambling. For everyday decisions under uncertainty—such as choosing between options with different costs and success rates—it plays the role of an “average outcome score.” For instance, if a random repair estimate is sometimes low and sometimes high, the expected cost summarizes the balance of those possibilities into a single number.

5.4 Expected cost and expected utility (basic form)

In basic cost modeling, expected value summarizes average cost: \[ E[\text{cost}]. \] In decision-making with preferences, expected utility uses a transformed payoff \(u(\text{outcome})\) and evaluates \(E[u(\text{outcome})]\). The difference matters: cost is measured in money, while utility encodes how strongly outcomes are valued by an individual or system.

6 Expected Value in Statistical Modeling

6.1 Sampling interpretations

Statistical models often interpret the expected value as the average behavior of a random quantity generated by a process. For example, if measurements are modeled as random draws around a central tendency, then expected value corresponds to that central level under the assumed model.

6.2 Empirical mean as an estimate of expectation

Given independent observations \(X_1,\dots,X_n\) from the same distribution as \(X\), the empirical mean \[ \bar{X}=\frac{1}{n}\sum_{i=1}^n X_i \] is a sample-based estimate of \(E[X]\).

6.2.1 Law of large numbers connection

The law of large numbers formalizes the convergence: as \(n\) grows, \(\bar{X}\) approaches \(E[X]\) (under standard conditions). This justifies using sample averages as practical proxies for theoretical expected values.

In estimation, an estimator is characterized by how its expected value relates to the target parameter. If \(T\) is an estimator for a parameter \(\theta\), then bias describes the difference between \(E[T]\) and \(\theta\). This conceptual link uses expected value as the baseline for performance assessment.

7 Common Pitfalls

7.1 Confusing expected value with probability

Expected value is an average of outcomes, not the probability of an event. A number like \(E[X]\) does not directly indicate what outcome will occur most often, especially for skewed distributions.

7.2 Ignoring infinite expectations

Some random variables have \(E[X]=\infty\) or fail to exist as a finite number. Calculations that implicitly assume finiteness can become invalid, and certain properties may not apply in the expected manner.

7.3 Mistaking E[X^2] for (E[X])^2

In general, \[ E[X^2] \ne (E[X])^2. \] This distinction is crucial because \(E[X^2]\) relates to the spread and variability of \(X\), while \((E[X])^2\) is only determined by the mean.

7.4 Conditional expectation misunderstandings

Conditional expectation is not a simple conditioning “plug-in” of values unless the setup supports it. It yields a random variable depending on the conditioning information, and it must be computed using its defining properties rather than by informal reasoning.

8.1 Variance and standard deviation

Variance measures the typical squared deviation from the mean: \[ \mathrm{Var}(X)=E[(X-E[X])^2]. \] It complements expected value by describing dispersion rather than central tendency.

8.2 Moments and moment-based summaries

Moments generalize expected value by taking powers: the \(k\)-th moment is \(E[X^k]\) (when it exists). Moment-based summaries help characterize shape, tail behavior, and approximation properties.

8.3 Expected value of functions (higher-order expectations)

Expected value extends naturally to functions of random variables: \(E[g(X)]\). This is the foundation for many applied computations, including probability-weighted scoring rules and transforms used to quantify non-linear effects.

8.4 Martingales and expectation (high-level intuition)

In martingale theory, conditional expectations govern whether a process exhibits “fairness” over time. While expected value alone summarizes a single-time average, martingales use conditional expectation to enforce that future expected changes, given the present, behave in a particular way.