1 Definition and basic properties
A probability density function is a function used to describe a continuous random variable. It assigns relative density across possible values, allowing probabilities to be computed for intervals rather than for isolated points. In practice, a density serves as the continuous analogue of a probability mass function.
1.1 Continuous random variables
A continuous random variable can take any value within one or more intervals on the real line. Because there are infinitely many possible outcomes in any interval, the probability of an exact value is typically zero. The random variable is instead characterized by how probability is distributed across ranges of values.
1.2 Formal mathematical definition
For a random variable X, a function f is called a probability density function if the probability that X lies in an interval [a, b] is given by the integral of f over that interval. More generally, probabilities for sets of values are obtained by integrating the density over the corresponding region, provided the set is measurable.
1.3 Nonnegativity and normalization
A valid density must be nonnegative everywhere. It must also be normalized so that the total area under the curve over the full support equals 1. This condition ensures that the distribution assigns all probability to the possible outcomes of the random variable.
1.4 Support of a density function
The support of a density is the set of values where the density is nonzero or, more precisely, where the variable can occur with positive density. Some densities are defined on a finite interval, while others extend across all real numbers or only a half-line. Outside the support, the density is usually zero.
1.5 Relationship to probability intervals
For continuous distributions, probabilities are associated with intervals rather than points. The probability that X falls between two values a and b is the area under the density curve between those bounds. This makes the density function a tool for measuring relative concentration of probability across ranges.
2 Interpretation and intuition
A density function is best understood as a rate of probability concentration. Higher values indicate that values near that point are more concentrated, but not that the point itself has a large probability. The actual probability depends on the width of the interval being considered.
2.1 Density versus probability
Density and probability are related but distinct ideas. A density value can exceed 1, yet the total probability remains bounded by 1 because probabilities come from integrating over intervals. This distinction is one of the main features separating continuous from discrete probability models.
2.2 Area under the curve
The graph of a PDF is often interpreted geometrically. The probability of an interval is equal to the area under the curve across that interval. This area-based interpretation gives a visual way to compare intervals of different widths and positions.
2.3 Values at individual points
For a continuous random variable, the probability of any single exact point is usually zero. The value of the density at a point therefore does not directly represent a probability. Instead, it indicates the local intensity of the distribution near that point.
2.4 Visual representations
Densities are commonly displayed as smooth curves or, in some cases, piecewise shapes. Symmetric bell-shaped curves, skewed curves, and bounded shapes each reveal different features of the distribution. Graphical displays help identify concentration, spread, tails, and possible asymmetry.
3 Connection to other functions
A probability density function is closely tied to other functions used in probability theory. These include distribution functions and transform-based summaries, which provide alternative ways to describe the same random variable. Together, these functions support both theoretical analysis and computation.
3.1 Cumulative distribution function
The cumulative distribution function gives the probability that a random variable is less than or equal to a given value. For continuous distributions, it is obtained by accumulating the density up to that point. The CDF and PDF are therefore linked by integration and differentiation under suitable conditions.
3.1.1 Deriving the CDF from a PDF
The CDF can be formed by integrating the density from negative infinity to a specified value. This produces a nondecreasing function that starts near 0 and approaches 1 across the support. It summarizes all probabilities of left-sided intervals in a single function.
3.1.2 Deriving a PDF from a CDF
When a CDF is differentiable, its derivative is the corresponding density function. This relationship holds on intervals where the distribution is absolutely continuous. Points where the CDF is not smooth may indicate features not captured by an ordinary density.
3.2 Probability mass function
A probability mass function applies to discrete random variables and assigns probability directly to individual outcomes. A PDF does not do this, because continuous outcomes have zero probability at single points. The two concepts are analogous in purpose but differ in how probability is represented.
3.3 Characteristic function
The characteristic function is the expected value of a complex exponential of the random variable. It encodes the distribution in frequency space and can be used to study sums of random variables and limit theorems. When a PDF exists, the characteristic function is often obtained by integrating the density against the exponential kernel.
3.4 Moment-generating function
The moment-generating function, when it exists, provides a compact way to derive moments of the distribution. It is defined through the expected value of an exponential function of the random variable. For continuous distributions with a density, it is computed by integrating the exponential expression against that density.
4 Common probability density functions
Many familiar distributions are described by specific density functions. Each has characteristic shape, support, and parameters that control location, spread, or tail behavior. These examples are widely used in statistics, science, and engineering.
4.1 Uniform distribution
The uniform distribution assigns equal density across a finite interval. Its PDF is constant on the support and zero elsewhere. It is often used as a simple baseline model for values that are equally likely within a range.
4.2 Normal distribution
The normal distribution has a smooth bell-shaped density extending over the entire real line. It is symmetric about its mean and is controlled by parameters for location and scale. Because of its mathematical properties and frequent empirical approximation, it is one of the most important continuous distributions.
4.3 Exponential distribution
The exponential distribution models waiting times and decay-like processes. Its density is highest near zero and decreases monotonically as the variable increases. It is commonly associated with memoryless behavior in probabilistic models.
4.4 Gamma distribution
The gamma distribution is a flexible family on the positive real line. Its shape can vary from highly skewed to nearly symmetric depending on parameters. It is widely used for waiting times, positive data, and as a building block in statistical modeling.
4.5 Beta distribution
The beta distribution is defined on the interval from 0 to 1. Its density can take many shapes, including flat, U-shaped, or strongly skewed forms. It is often used to model proportions, probabilities, and rates.
4.6 Weibull distribution
The Weibull distribution is a versatile positive-valued distribution often used in reliability and survival analysis. Its density can represent increasing, decreasing, or constant hazard behavior depending on its shape parameter. This flexibility makes it useful in lifetime modeling.
4.7 Cauchy distribution
The Cauchy distribution has a characteristic peak and very heavy tails. Unlike the normal distribution, it does not have finite mean or variance in the usual sense. Its density is an important example of a distribution with extreme tail behavior.
5 Computation and applications
Probability density functions are used to compute numerical summaries and to support statistical inference. They provide the basis for interval probabilities, expectations, dispersion measures, and percentile calculations. Many applied methods depend on evaluating or approximating integrals of a density.
5.1 Calculating probabilities over intervals
To find the probability that a continuous random variable falls within a range, one integrates the density over that interval. This computation may be exact for simple functions or approximate for more complicated ones. Interval probabilities are central to hypothesis testing, confidence intervals, and predictive analysis.
5.2 Expected value and moments
The expected value of a continuous random variable is computed by integrating the variable times its density. More generally, moments are obtained by integrating powers of the variable against the PDF. These quantities summarize central tendency, shape, and tail behavior.
5.3 Variance and standard deviation
Variance measures how far values typically lie from the mean. For a continuous random variable, it is found by integrating the squared deviation from the mean with respect to the density. The standard deviation is the square root of the variance and provides a scale for typical dispersion.
5.4 Quantiles and percentiles
Quantiles are values that divide the distribution into specified cumulative proportions. They are determined from the CDF, which is derived from the PDF. Percentiles are widely used to summarize medians, thresholds, and relative position within a distribution.
5.5 Parameter estimation
In statistical practice, the form of the density may be assumed and its parameters estimated from sample data. Common methods include maximum likelihood and method of moments. Parameter estimation links observed data to a continuous probability model.
6 Multivariate density functions
Density functions can also describe several continuous random variables jointly. In this setting, the density is defined over a region in higher-dimensional space. Multivariate densities are essential for modeling dependence and relationships among variables.
6.1 Joint probability density functions
A joint density describes the distribution of two or more continuous random variables at the same time. Probabilities for regions in multidimensional space are obtained by integrating the joint density over those regions. Joint densities provide the full probabilistic description of the variables together.
6.2 Marginal density functions
A marginal density is obtained from a joint density by integrating out one or more variables. It describes the distribution of a subset of the variables without regard to the others. Marginalization is a basic tool for isolating individual components of a multivariate model.
6.3 Conditional density functions
A conditional density describes the distribution of one variable given that another has a specified value. It is derived from the joint density and the corresponding marginal density. Conditional densities are used to study dependence and to make probabilistic predictions.
6.4 Independence and factorization
When two continuous random variables are independent, their joint density factors into the product of their marginal densities. This factorization is a defining property of independence in the continuous setting. It greatly simplifies calculations and model construction.
6.5 Transformation of variables
New random variables can be formed by transforming existing ones. The resulting density is found by accounting for how the transformation stretches or compresses regions of probability. Change-of-variable methods are especially important in multivariate calculus and statistical modeling.
7 Advanced topics
Beyond standard definitions, density functions appear in specialized forms and more elaborate estimation frameworks. These topics include distributions that are not smooth everywhere, mathematical extensions, and practical methods for learning densities from data. Such ideas broaden the usefulness of the PDF concept in theory and application.
7.1 Piecewise-defined densities
Some densities are defined by different formulas on different parts of their support. These piecewise forms may arise from mixtures, truncation, or practical modeling needs. As long as the function remains nonnegative and integrates to 1, it is a valid PDF.
7.2 Improper and generalized densities
Not every nonnegative function qualifies as a probability density. Some functions are improper because they do not integrate to one, while others appear in generalized settings that require normalization or distributional interpretation. These cases are useful in advanced analysis but must be handled carefully.
7.3 Densities in Bayesian statistics
In Bayesian statistics, densities are used to represent prior beliefs, likelihoods, and posterior distributions. Continuous parameter models often rely on PDFs to express uncertainty about unknown quantities. The resulting posterior density combines prior information with observed data.
7.4 Estimating densities from data
When the underlying distribution is unknown, a density can be estimated from sample observations. Density estimation aims to produce a smooth representation of the data-generating process. This is a common task in exploratory data analysis and nonparametric inference.
7.4.1 Parametric methods
Parametric density estimation assumes a specific family of distributions and fits its parameters to data. This approach is efficient when the chosen family is a good description of the sample. Its accuracy depends strongly on the validity of the model assumption.
7.4.2 Nonparametric methods
Nonparametric methods estimate density shape with fewer assumptions about the underlying distribution. They are useful when the data exhibit features not well captured by standard parametric families. These methods often provide greater flexibility at the cost of additional smoothing choices.
7.4.2.1 Kernel density estimation
Kernel density estimation constructs a smooth density by placing a small kernel function at each data point and summing the contributions. The bandwidth controls the level of smoothing and strongly affects the result. This technique is widely used for visualizing and approximating unknown continuous distributions.