1 Definition and Notation

1.1 Formal Definition

Conditional probability is a measure of the probability of an event occurring given that another event has already occurred. It refines the probability assessment by incorporating additional information, reducing the universe of possible outcomes to those consistent with the conditioning event.

1.2 Alternative Notation

Conditional probability is commonly denoted \( P(A \mid B) \), read as “the probability of \( A \) given \( B \).” Alternative notations include \( P_B(A) \) and, in some contexts, \( \Pr(A \mid B) \).

1.3 Fundamental Formula (\( P(A \mid B) = P(A \cap B) / P(B) \))

For two events \( A \) and \( B \) with \( P(B) > 0 \), the conditional probability is defined by \[ P(A \mid B) = \frac{P(A \cap B)}{P(B)}. \] This formula expresses the probability of the joint occurrence of \( A \) and \( B \) relative to the probability of \( B \).

1.3.1 Interpretation in Terms of Sample Space Reduction

When conditioning on \( B \), the original sample space is effectively reduced to \( B \). The conditional probability of \( A \) given \( B \) is then the proportion of \( B \) that also belongs to \( A \). This viewpoint underlies many intuitive uses of conditional probability, such as updating beliefs after observing partial information.

2 Properties and Theorems

2.1 Basic Properties

2.1.1 Range of Conditional Probability

Conditional probability satisfies \( 0 \leq P(A \mid B) \leq 1 \) for any event \( A \) and any event \( B \) with \( P(B) > 0 \). It is a valid probability measure on the reduced sample space \( B \).

2.1.2 Multiplication Rule (\( P(A \cap B) = P(A \mid B) P(B) \))

Rearranging the definition yields the multiplication rule: \[ P(A \cap B) = P(A \mid B) \, P(B). \] This rule extends to more than two events via iterated conditioning.

2.2 Law of Total Probability

2.2.1 Partition of Sample Space

Let \( \{B_1, B_2, \dots, B_n\} \) be a partition of the sample space—i.e., the events are mutually exclusive and their union covers the entire space. For any event \( A \), \[ P(A) = \sum_{i=1}^{n} P(A \cap B_i) = \sum_{i=1}^{n} P(A \mid B_i) \, P(B_i), \] provided each \( P(B_i) > 0 \). This decomposition is known as the law of total probability.

2.2.2 Formula and Example

For a simple partition consisting of an event \( B \) and its complement \( B^c \), \[ P(A) = P(A \mid B) P(B) + P(A \mid B^c) P(B^c). \] For example, if a disease affects 1% of the population and a test has 99% sensitivity and 95% specificity, the overall probability of a positive test is \( P(\text{positive}) = 0.99 \times 0.01 + 0.05 \times 0.99 = 0.0594 \).

2.3 Bayes' Theorem

2.3.1 Derivation from Conditional Probability

Bayes’ theorem follows directly from the definition of conditional probability: \[ P(B \mid A) = \frac{P(A \mid B) P(B)}{P(A)}. \] It relates the conditional probability of \( B \) given \( A \) to the conditional probability of \( A \) given \( B \).

2.3.2 Inverse Probability Interpretation

The theorem provides a way to “invert” conditional probabilities. It is widely used to update the probability of a hypothesis (event \( B \)) after observing evidence (event \( A \)). The term \( P(B) \) is the prior probability, and \( P(B \mid A) \) is the posterior probability.

2.3.3 Multi-event Form (with partitions)

For a partition \( \{B_1, \dots, B_n\} \), Bayes’ theorem becomes \[ P(B_i \mid A) = \frac{P(A \mid B_i) P(B_i)}{\sum_{j=1}^{n} P(A \mid B_j) P(B_j)}. \] This form is essential in many statistical and machine learning applications.

3 Special Cases and Extensions

3.1 Conditional Probability for Independent Events

3.1.1 Definition of Independence (\( P(A \mid B) = P(A) \))

Two events \( A \) and \( B \) are independent if and only if \( P(A \mid B) = P(A) \) (or equivalently \( P(B \mid A) = P(B) \)), meaning the occurrence of one does not affect the probability of the other. Independence also implies \( P(A \cap B) = P(A) P(B) \).

3.1.2 Relationship with Zero Conditional Probability

If two events are independent and \( P(A) > 0 \), then \( P(A \mid B) > 0 \) whenever \( P(B) > 0 \). Zero conditional probability can occur between independent events only if one of them has probability zero, which is a degenerate case.

3.2 Conditional Probability for Disjoint Events

3.2.1 Zero Conditional Probability for Mutually Exclusive Events

If \( A \) and \( B \) are mutually exclusive (disjoint) events with \( P(A) > 0 \) and \( P(B) > 0 \), then \( P(A \cap B) = 0 \), so \( P(A \mid B) = 0 \). Knowing that \( B \) has occurred rules out \( A \).

3.3 Conditional Probability in Continuous Distributions

3.3.1 Conditional Density Function

For continuous random variables \( X \) and \( Y \), the conditional probability density function of \( X \) given \( Y = y \) is defined as \[ f_{X \mid Y}(x \mid y) = \frac{f_{X,Y}(x,y)}{f_Y(y)}, \] where \( f_{X,Y} \) is the joint density and \( f_Y(y) > 0 \). This parallels the discrete definition, replacing probability masses with densities.

3.3.2 Conditional Expectation

The conditional expectation \( E[X \mid Y = y] \) is the expected value of \( X \) with respect to the conditional distribution \( f_{X \mid Y}(x \mid y) \). It is a function of \( y \) and plays a central role in regression analysis and stochastic processes.

3.4 Higher-Order Conditioning

3.4.1 Conditioning on Multiple Events (\( P(A \mid B, C) \))

Conditioning on two or more events is defined as \[ P(A \mid B, C) = \frac{P(A \cap B \cap C)}{P(B \cap C)}, \] provided the denominator is positive. It represents the probability of \( A \) given that both \( B \) and \( C \) have occurred.

3.4.2 Chain Rule for Sequences

For a sequence of events \( A_1, A_2, \dots, A_n \), the multiplication rule generalizes to the chain rule: \[ P(A_1 \cap A_2 \cap \dots \cap A_n) = P(A_1) \, P(A_2 \mid A_1) \, P(A_3 \mid A_1 \cap A_2) \cdots P(A_n \mid A_1 \cap \dots \cap A_{n-1}). \] This rule is fundamental in modeling sequential stochastic processes.

4 Applications and Examples

4.1 Real-World Examples

4.1.1 Medical Testing (Sensitivity, Specificity, Predictive Values)

In diagnostic testing, sensitivity is \( P(\text{positive test} \mid \text{disease}) \) and specificity is \( P(\text{negative test} \mid \text{no disease}) \). The positive predictive value is \( P(\text{disease} \mid \text{positive test}) \), which depends on the disease prevalence via Bayes’ theorem.

4.1.2 Gambling and Card Games

In card games like poker, conditional probability is used to compute the chance of improving a hand after drawing new cards. For example, the probability of drawing a flush given that a player already holds four cards of the same suit is computed using the conditional formula.

4.1.3 Weather Forecasting

Weather predictions often rely on conditional probabilities, such as the chance of rain given current atmospheric pressure and humidity. These are estimated from historical data and updated via Bayesian methods.

4.2 Computational and Algorithmic Uses

4.2.1 Naive Bayes Classifier

The naive Bayes classifier assumes conditional independence of features given the class label. Using Bayes’ theorem, it computes \( P(\text{class} \mid \text{features}) \) and is widely used in spam filtering, document classification, and sentiment analysis.

4.2.2 Hidden Markov Models

Hidden Markov models (HMMs) use conditional probabilities to describe sequences of observed events dependent on hidden states. The transition probabilities \( P(\text{state}_{t+1} \mid \text{state}_t) \) and emission probabilities \( P(\text{observation} \mid \text{state}) \) are central to the model.

4.2.3 Bayesian Networks

Bayesian networks are graphical models that represent joint probability distributions via conditional independence relationships. Each node’s probability is conditioned on its parent nodes, enabling efficient computation of conditional probabilities for inference and decision support.

5 Common Misconceptions and Pitfalls

5.1 Confusion between \( P(A \mid B) \) and \( P(B \mid A) \)

A frequent error is to treat these two quantities as equal. For example, the probability of having a disease given a positive test is often much lower than the probability of a positive test given the disease, especially when the disease is rare.

5.2 Base Rate Fallacy

The base rate fallacy occurs when the prior probability (base rate) of an event is ignored. For instance, in medical diagnosis, a high false-positive rate combined with a low disease prevalence leads to a low positive predictive value. People often overestimate the chance of disease after a positive test because they neglect the base rate.

5.3 Conditional Probability vs. Causation

A high conditional probability does not imply causation. For example, \( P(\text{car stuck in traffic} \mid \text{umbrella sales high}) \) may be large due to a common cause (rainy weather), but umbrella sales do not cause traffic jams. Conditional probability measures correlation, not causal direction.

6.1 Joint Probability

Joint probability \( P(A \cap B) \) is the probability that both events occur simultaneously. It serves as the numerator in the definition of conditional probability.

6.2 Marginal Probability

Marginal probability \( P(A) \) is the probability of a single event without regard to other events. It is derived from the joint distribution by summing (or integrating) over all possible values of the other variables.

6.3 Conditional Independence

Two events \( A \) and \( B \) are conditionally independent given \( C \) if \( P(A \cap B \mid C) = P(A \mid C) P(B \mid C) \). This property simplifies modeling and inference in Bayesian networks and other graphical models.