1 Basic concepts
Probability studies random phenomena in a structured way. It assigns numerical values to uncertainty, allowing events to be compared, modeled, and analyzed. The basic framework consists of experiments, outcomes, events, and rules that determine how probabilities are calculated.
1.1 Random experiments
A random experiment is a process whose exact outcome cannot be predicted in advance, even if the set of possible outcomes is known. Examples include tossing a coin, rolling dice, drawing cards, or measuring a signal influenced by chance. The term does not imply pure chaos; rather, it refers to situations with variability that can still be described systematically.
1.2 Sample space
The sample space is the complete collection of possible outcomes of a random experiment. It may be finite, countably infinite, or continuous, depending on the situation. Each outcome in the sample space represents one possible result of the experiment.
1.3 Events
An event is a set of outcomes from the sample space. Simple events contain one outcome, while compound events contain several. Probability is assigned to events rather than to outcomes in isolation, which makes it possible to describe statements such as “at least one head occurs” or “the number is greater than three.”
1.4 Probability measures
A probability measure is a rule that assigns a number between 0 and 1 to each event in a way that satisfies certain logical conditions. It formalizes the idea of chance and allows probabilities to be combined consistently. In standard probability theory, the measure is defined on a collection of events that is closed under basic set operations.
1.4.1 Axioms of probability
The axioms of probability provide the foundation for the subject. They state that probabilities are nonnegative, the probability of the entire sample space is 1, and the probability of a countable union of disjoint events equals the sum of their probabilities. These assumptions make probability mathematically coherent.
1.4.2 Properties of probability
From the axioms, many useful properties follow. The probability of the empty set is 0, probabilities are monotonic with respect to set inclusion, and complementary events add to 1. Other formulas, such as the probability of a union of events, can be derived from these basic rules.
1.5 Complement, union, and intersection
The complement of an event consists of all outcomes not in that event. The union of events includes outcomes in at least one of them, while the intersection includes outcomes shared by all of them. These operations are central for expressing combinations of conditions and for computing compound probabilities.
2 Classical probability
Classical probability refers to settings where outcomes are treated as equally likely. In such cases, probability can often be computed by counting favorable outcomes and comparing them with the total number of possible outcomes. This approach is especially common in elementary problems involving dice, coins, and cards.
2.1 Equally likely outcomes
When all outcomes in a sample space have the same chance of occurring, probability is determined by symmetry. For example, a fair coin has two equally likely outcomes, and a fair die has six. The classical approach applies best when there is a clear basis for assuming equal likelihood.
2.2 Counting methods
Counting methods help determine the number of possible outcomes or favorable outcomes in a problem. They are especially useful when direct listing is impractical. By organizing outcomes systematically, counting techniques make it possible to compute probabilities in finite sample spaces.
2.2.1 Permutations
Permutations count arrangements in which order matters. If several objects are selected or arranged, different orderings are treated as distinct outcomes. Permutation formulas are widely used in problems involving seating arrangements, rankings, and ordered selections.
2.2.2 Combinations
Combinations count selections in which order does not matter. They are used when one is interested only in which objects are chosen, not in the sequence of choice. Combinations appear frequently in card problems, subset selection, and sampling without order.
2.3 Laplace definition of probability
The Laplace definition states that, when outcomes are equally likely, probability equals the number of favorable outcomes divided by the number of possible outcomes. This definition was historically influential and remains useful in elementary and combinatorial settings. It is a special case of the broader modern theory.
3 Conditional probability
Conditional probability measures the chance of an event given that another event has already occurred. It refines ordinary probability by incorporating additional information. This idea is fundamental in inference, decision-making, and many applications where evidence changes what is considered likely.
3.1 Definition and interpretation
The conditional probability of event A given event B is the probability that A occurs within the restricted setting where B is known to have occurred. Intuitively, the sample space is reduced to the outcomes compatible with B. This concept captures how new information updates uncertainty.
3.2 Independence
Two events are independent if knowledge of one does not change the probability of the other. In that case, the conditional probability of one given the other equals its ordinary probability. Independence is a strong and important relationship, but it should not be confused with mutual exclusivity, which is a different notion.
3.3 Total probability
The law of total probability expresses the probability of an event by breaking the sample space into several disjoint cases. It is useful when a problem can be analyzed by conditioning on one of several possible underlying conditions. This principle often serves as a bridge between simpler subproblems and a final overall result.
3.4 Bayes' theorem
Bayes’ theorem relates conditional probabilities in the reverse direction. It allows one to update the probability of a cause or hypothesis after observing evidence. The theorem is widely used in statistics, diagnostics, machine learning, and reasoning under uncertainty.
4 Random variables
A random variable assigns a numerical value to each outcome in a sample space. This transformation makes it possible to study random phenomena using algebra and calculus. Random variables can represent counts, measurements, times, or other quantities derived from chance events.
4.1 Discrete random variables
A discrete random variable takes values in a countable set, such as integers or finite categories. Examples include the number of heads in a series of coin tosses or the number of arrivals in a time interval. Discrete variables are described by probability assignments to individual values.
4.1.1 Probability mass function
The probability mass function gives the probability that a discrete random variable equals a particular value. It lists the possible values and their associated probabilities. The total of all probabilities in the function must equal 1.
4.1.2 Cumulative distribution function
The cumulative distribution function gives the probability that a random variable is less than or equal to a specified value. For discrete variables, it increases in steps as values are included. It provides a compact summary of the distribution.
4.2 Continuous random variables
A continuous random variable can take values throughout an interval or another uncountable set. Exact point probabilities are typically zero, so probabilities are assigned to ranges of values instead. Such variables commonly model measurements like length, time, or temperature.
4.2.1 Probability density function
The probability density function describes how probability is distributed over the range of a continuous variable. Probabilities are obtained by integrating the density over intervals. A higher density indicates a greater concentration of probability, not a direct probability at a single point.
4.2.2 Cumulative distribution function
For continuous random variables, the cumulative distribution function again gives the probability that the variable does not exceed a given value. It is usually smooth or piecewise smooth and can be derived from the density when one exists. The function completely characterizes the distribution.
4.3 Expectation
Expectation is the average value of a random variable in the long run, weighted by probability. It is also called the mean. Expectation plays a central role in prediction, optimization, and theoretical analysis.
4.4 Variance and standard deviation
Variance measures how spread out the values of a random variable are around the mean. Standard deviation is the square root of variance and is expressed in the same units as the variable itself. These quantities describe variability and are essential in assessing uncertainty.
4.5 Moments
Moments are numerical summaries of a distribution, usually defined through powers of the random variable. The first moment is related to the mean, while higher moments capture features such as spread, skewness, and tail behavior. Moments help distinguish between distributions and support theoretical analysis.
5 Common probability distributions
Probability distributions provide standard models for random variables encountered in theory and practice. Some distributions are discrete, others continuous, and each has characteristic properties that make it suitable for particular kinds of data or processes.
5.1 Bernoulli distribution
The Bernoulli distribution describes a single trial with two outcomes, often called success and failure. It is the simplest nontrivial discrete distribution and serves as a building block for more complex models. Many binary events can be represented in this form.
5.2 Binomial distribution
The binomial distribution models the number of successes in a fixed number of independent Bernoulli trials with the same success probability. It is used for repeated yes-or-no experiments. Its shape depends on both the number of trials and the underlying success rate.
5.3 Geometric distribution
The geometric distribution describes the number of trials needed to obtain the first success in repeated independent Bernoulli trials. It is closely related to waiting-time problems in discrete settings. A notable feature is its memoryless property, which means the future does not depend on how many failures have already occurred.
5.4 Poisson distribution
The Poisson distribution models counts of events occurring independently over a fixed interval or region when events are relatively rare. It is common in queueing, arrival processes, and natural counting phenomena. The distribution is characterized by a single rate parameter.
5.5 Uniform distribution
The uniform distribution assigns equal likelihood across a specified interval or set of outcomes. In the continuous case, all subintervals of equal length have the same probability. It is often used as a baseline model when no preferred values are assumed.
5.6 Normal distribution
The normal distribution is one of the most important continuous distributions in probability and statistics. It has the familiar bell-shaped curve and is determined by its mean and variance. Many naturally occurring measurements and aggregated random effects are well approximated by it.
5.7 Exponential distribution
The exponential distribution models waiting times between events in a process with a constant average rate. It is continuous and has the memoryless property. This distribution appears frequently in reliability theory, queueing models, and stochastic processes.
6 Joint probability
Joint probability concerns multiple random variables considered together. It describes how variables behave in relation to one another, rather than separately. This area is essential for studying dependence, shared variation, and multivariate systems.
6.1 Joint distributions
A joint distribution assigns probabilities to combinations of values for two or more random variables. It provides the full probabilistic description of a multivariable system. From the joint distribution, many other quantities, such as marginals and conditional distributions, can be derived.
6.2 Marginal distributions
Marginal distributions describe the behavior of each variable alone, obtained from the joint distribution by summing or integrating over the other variables. They show the individual pattern of each component without reference to the others. Marginals are useful for separating overall structure from joint dependence.
6.3 Conditional distributions
Conditional distributions describe one random variable after fixing the value of another. They show how the distribution changes when additional information is available. These distributions are central in prediction, regression, and iterative probabilistic methods.
6.4 Covariance and correlation
Covariance measures how two random variables vary together, while correlation standardizes this relationship to a dimensionless scale. Positive values indicate that the variables tend to move in the same direction, while negative values indicate the opposite. Correlation is widely used but should not be mistaken for causation.
7 Limit theorems
Limit theorems describe the behavior of random phenomena as the number of observations becomes large. They explain why stable patterns emerge from uncertainty and why certain distributions recur in aggregate data. These results are among the most powerful in probability theory.
7.1 Laws of large numbers
The laws of large numbers state that averages of repeated random observations tend to stabilize as the number of observations grows. They justify the use of sample averages as estimates of expected values. Different versions of the law vary in strength, but all express the same general principle of convergence.
7.2 Central limit theorem
The central limit theorem states that sums or averages of many independent, suitably scaled random variables tend toward a normal distribution under broad conditions. It explains the widespread appearance of the normal law in practice. The theorem is a cornerstone of probability and statistical inference.
7.3 Convergence in probability
Convergence in probability means that a sequence of random variables becomes increasingly likely to lie close to a target value as the index grows. It is a precise way of saying that random quantities stabilize in a probabilistic sense. This notion is important in asymptotic analysis.
7.4 Convergence in distribution
Convergence in distribution concerns the convergence of distribution functions rather than pointwise values of random variables. It is weaker than convergence in probability but still captures important limiting behavior. This concept is central in studying approximations and asymptotic laws.
8 Advanced topics
Advanced probability extends the basic framework to more abstract and powerful settings. These topics are designed to handle infinite sample spaces, evolving random systems, and phenomena that change over time. They provide the mathematical foundation for much of modern applied probability.
8.1 Measure-theoretic probability
Measure-theoretic probability formulates the subject using measure theory, which allows rigorous treatment of continuous spaces and limiting processes. It unifies discrete and continuous probability within one abstract framework. This approach is standard in advanced mathematics.
8.2 Stochastic processes
A stochastic process is a family of random variables indexed by time or another parameter. It models systems that evolve unpredictably, such as queues, signals, or particle motion. Different processes are distinguished by their dependence structure and state evolution.
8.3 Martingales
Martingales are stochastic processes with a fair-game property: given the present, the expected future value equals the current value. They are fundamental in probability theory because they formalize balanced, nonanticipating dynamics. Martingale methods are used in convergence theory and applied modeling.
8.4 Markov chains
Markov chains are stochastic processes in which the next state depends only on the current state, not on the full past history. This memoryless transition rule makes them tractable and versatile. They are used to model systems that move among states step by step.
8.5 Brownian motion
Brownian motion is a continuous-time stochastic process that models random movement in a highly idealized way. It is central in mathematical physics and probability theory. The process has continuous paths and is often used as a limit model for many small random fluctuations.
9 Applications
Probability is used wherever uncertainty must be described, measured, or managed. Its applications range from theoretical science to practical decision systems. The subject provides tools for analyzing variability, forecasting outcomes, and constructing reliable models.
9.1 Statistics
Statistics relies on probability to draw conclusions from data. Sampling distributions, estimation, hypothesis testing, and confidence intervals all depend on probabilistic reasoning. Probability gives a principled basis for interpreting observed information.
9.2 Physics
In physics, probability is used to model random motion, particle interactions, and systems with many components. It also appears in statistical mechanics and quantum theory, where uncertainty is built into the description of phenomena. Probabilistic methods help connect microscopic randomness with macroscopic behavior.
9.3 Computer science
Computer science uses probability in algorithms, randomized computation, machine learning, and network analysis. Randomization can improve efficiency, simplify design, or provide approximate solutions. Probabilistic analysis is also important in data structures, cryptography, and performance evaluation.
9.4 Finance
Finance uses probability to evaluate uncertain returns, price risk, and model market fluctuations. Quantitative methods often depend on distributions, stochastic processes, and expectation-based decision rules. Probability helps describe uncertainty in investment and portfolio settings.
9.5 Reliability engineering
Reliability engineering applies probability to the study of failures, lifetimes, and system performance. It is used to estimate how long components will function and how likely a system is to remain operational. Probabilistic models support maintenance planning and risk assessment.
10 Historical development
The history of probability reflects a gradual shift from practical counting and gambling problems to a broad mathematical theory. Its development involved contributions from many fields, including combinatorics, analysis, and statistics. Over time, probability became an independent discipline with deep theoretical foundations.
10.1 Early origins
Early ideas of chance appeared in gambling, astronomy, insurance, and legal reasoning. Ancient and medieval thinkers considered questions involving uncertainty, but these discussions were not yet organized into a formal theory. The practical need to quantify risk helped motivate later developments.
10.2 Development in the 17th and 18th centuries
The modern study of probability emerged in the 17th century through problems involving games of chance. Mathematicians developed methods for calculating expected outcomes and fair divisions. During the 18th century, the subject expanded through work on combinatorics, actuarial science, and the analysis of random events.
10.3 Modern probability theory
Modern probability theory was shaped by rigorous axiomatic methods and by the study of limit behavior. In the 20th century, probability became closely linked with measure theory, stochastic processes, and advanced statistics. This transformation turned probability into a central mathematical framework for uncertainty across many disciplines.
</INTERNAL_LINK_CANDIDATES> Sample space (the complete set of possible outcomes) Event (a set of outcomes from a sample space) Probability measure (a rule assigning probabilities to events) Axioms of probability (the basic rules of probability) Complement (the set of outcomes not in an event) Union (outcomes belonging to at least one of two events) Intersection (outcomes belonging to both events) Permutation (an ordered arrangement of objects) Combination (a selection where order does not matter) Conditional probability (the probability of an event given another) Independence (a relationship where one event does not affect another) Bayes' theorem (a formula for reversing conditional probabilities) Random variable (a numerical value assigned to each outcome) Probability mass function (the distribution of a discrete random variable) Cumulative distribution function (the probability of being at or below a value) Probability density function (a function describing continuous probability) Expectation (the weighted average value of a random variable) Variance (a measure of spread around the mean) Moment (a numerical summary based on powers of a variable) Bernoulli distribution (a two-outcome probability model) Binomial distribution (a model for successes in repeated trials)