1 Definition and basic concepts
A sampling distribution is the probability distribution of a statistic obtained from repeated samples of the same size drawn from a population. It shows how a sample-based measure, such as a mean, proportion, or variance, changes from one sample to another. In practice, sampling distributions connect the data observed in a sample with the larger population that the sample represents.
Sampling distributions are fundamental to statistical inference because they describe the expected behavior of statistics under repeated sampling. By understanding this behavior, statisticians can judge how reliable an estimate may be, how much random variation to expect, and how unusual a result is under a given model.
1.1 Statistic and parameter
A statistic is a numerical summary computed from sample data, such as the sample mean or sample proportion. A parameter is a fixed numerical feature of the population, such as the population mean or population proportion. Sampling distributions are used to study how statistics behave when they are used to estimate parameters.
The distinction matters because a parameter is usually unknown, while a statistic is observable and varies from sample to sample. Much of statistical inference is built on using sample statistics to make conclusions about population parameters.
1.2 Population and sample
The population is the full set of individuals, items, or measurements of interest. A sample is a subset drawn from that population. Since it is often impractical to observe the entire population, researchers rely on samples to gather information and draw conclusions.
A sample is expected to reflect the population only approximately. Differences between samples arise through random variation, and these differences are what sampling distributions describe.
1.3 Repeated sampling
Sampling distributions are defined through the idea of repeated sampling. If many samples of the same size were drawn from the same population and the same statistic were calculated for each one, the resulting values would form a distribution. This imagined or simulated process is the basis for understanding uncertainty in statistical estimates.
Repeated sampling does not require physically collecting endless samples. The concept is often used theoretically or approximated through computation, especially when exact formulas are difficult to obtain.
1.4 Probability distribution of a statistic
A statistic is itself a random variable because its value depends on which sample is selected. Its sampling distribution assigns probabilities to the possible values the statistic may take. The distribution may be discrete or continuous, depending on the statistic and the population model.
This probability distribution can be summarized by its center, spread, and shape. These features determine how stable the statistic is and how it should be interpreted in inference.
2 Common types of sampling distributions
Different statistics have different sampling distributions. Some are familiar because they appear often in basic analysis, while others are used in specialized settings. The form of the distribution depends on both the statistic and the structure of the data.
2.1 Sampling distribution of the sample mean
The sample mean is one of the most widely studied statistics. Its sampling distribution describes how the average of repeated samples varies around the population mean. For many populations, the distribution of the sample mean becomes approximately normal as sample size increases.
This distribution is especially important because means are commonly used to estimate expected values, compare groups, and build confidence intervals.
2.2 Sampling distribution of the sample proportion
The sample proportion summarizes the fraction of observations in a sample with a particular characteristic. Its sampling distribution describes how that fraction varies across repeated samples. When sample size is large and the population proportion is not too close to 0 or 1, the distribution is often well approximated by a normal curve.
Sampling distributions for proportions are central in surveys, election polling, quality control, and studies of categorical outcomes.
2.3 Sampling distribution of the sample variance
The sample variance measures the spread of values in a sample. Its sampling distribution shows how observed variability changes across repeated samples. Unlike the mean, the variance has a more complex distribution, often tied to the chi-square distribution in standard models.
Because variance is a measure of dispersion, its sampling distribution is important in studies of reliability, measurement error, and comparisons of spread.
2.4 Sampling distribution of differences between samples
Many analyses compare two or more groups using a difference between sample statistics, such as the difference between two means or two proportions. The sampling distribution of such a difference describes how much the observed contrast would vary under repeated sampling.
These distributions are used to evaluate whether an observed difference may be due to random fluctuation or reflects a genuine population effect.
3 Properties of sampling distributions
Sampling distributions have several key properties that determine their usefulness in inference. The most important are center, variability, and shape. These features are influenced by the underlying population and by the size of the sample.
3.1 Center or expected value
The center of a sampling distribution is often given by its expected value. For many common statistics, the expected value equals the corresponding population parameter, making the statistic an unbiased estimator. In such cases, repeated samples tend to average out to the true quantity of interest.
When a statistic is biased, its sampling distribution is shifted away from the parameter. Understanding this shift helps analysts correct or account for systematic error.
3.2 Variability and standard error
The spread of a sampling distribution is measured by its variability. The standard error is the standard deviation of the sampling distribution of a statistic. It reflects how much the statistic is expected to fluctuate from sample to sample.
A smaller standard error indicates a more precise statistic. Standard error decreases as sample size increases for many statistics, which is one reason larger samples are generally more informative.
3.3 Shape of the distribution
The shape of a sampling distribution may be symmetric, skewed, bell-shaped, or otherwise structured depending on the statistic and the sample size. Shape affects how well normal-based methods apply and how extreme values should be interpreted.
3.3.1 Symmetry and skewness
Some sampling distributions are nearly symmetric, especially when based on large samples or when the population itself is symmetric. Others are skewed, particularly for statistics that cannot take negative values, such as variances. Skewness often decreases as sample size grows.
The degree of asymmetry matters because it influences the accuracy of confidence intervals and significance tests that assume approximate normality.
3.3.2 Normal approximation
Many sampling distributions are approximated by the normal distribution under suitable conditions. This approximation is one of the most practical results in statistics because it simplifies calculations and supports widely used inferential methods.
The quality of the approximation depends on sample size, population shape, and the statistic being studied. For proportions and means, the approximation is often effective when sample sizes are moderate to large.
3.4 Effect of sample size
Sample size strongly affects the form of a sampling distribution. Larger samples usually produce smaller standard errors and distributions that are more tightly concentrated around the center. They can also make the distribution more nearly normal, even when the population is not.
Smaller samples tend to produce greater variability and may yield distributions that are more irregular or skewed. This is why the precision of a statistic is closely tied to sample size.
4 Theoretical results
Several theoretical results explain why sampling distributions behave as they do. These results provide the mathematical foundation for much of modern statistics and help justify common approximations and methods.
4.1 Central limit theorem
The central limit theorem states that, under broad conditions, the sampling distribution of the sum or mean of independent observations tends toward a normal distribution as sample size increases. This result holds even when the population distribution is not normal, provided certain regularity conditions are met.
It is one of the most important theorems in statistics because it explains the widespread usefulness of normal approximations. It also underpins many methods for estimating uncertainty and testing hypotheses.
4.2 Law of large numbers
The law of large numbers states that as sample size grows, the sample statistic often approaches the corresponding population quantity. For example, the sample mean tends to get closer to the population mean with larger samples.
This result complements the central limit theorem. While the central limit theorem describes the shape of the sampling distribution, the law of large numbers describes convergence toward the true value.
4.3 Exact distributions for small samples
For small samples, exact sampling distributions are often preferred when available. These distributions are derived from precise mathematical relationships rather than asymptotic approximations. They are especially useful when sample sizes are not large enough for normal methods to be fully reliable.
4.3.1 t distribution
The t distribution is used when working with sample means from normally distributed populations when the population standard deviation is unknown. It has heavier tails than the normal distribution, reflecting additional uncertainty from estimating variability from the sample.
As sample size increases, the t distribution becomes closer to the normal distribution. It is widely used in confidence intervals and tests involving means.
4.3.2 Chi-square distribution
The chi-square distribution commonly arises in the sampling distribution of a scaled sample variance under normality assumptions. It is also used in tests of goodness of fit and independence in categorical data.
Its shape is typically right-skewed for small degrees of freedom and becomes less skewed as degrees of freedom increase.
4.3.3 F distribution
The F distribution often appears in the sampling distribution of ratios of variances. It is central in analysis of variance and in tests comparing variability across groups. Like the chi-square distribution, it is usually right-skewed.
The F distribution is useful whenever a statistic is formed by comparing two independent sources of variation.
5 Construction and derivation
Sampling distributions can be constructed in several ways. Some are derived exactly from probability theory, while others are approximated numerically. The method used depends on the complexity of the statistic and the amount of available information.
5.1 Enumerating all possible samples
When a population is small, it may be possible to list all possible samples of a given size and compute the statistic for each one. The frequencies or probabilities of these values then define the sampling distribution exactly.
This approach is conceptually straightforward, though it becomes impractical for large populations or large sample sizes because the number of possible samples grows rapidly.
5.2 Using probability models
Sampling distributions are often derived from a probability model for the population and sampling process. Under specified assumptions, algebraic formulas can be used to obtain the distribution of the statistic. This method is common in theoretical statistics and in standard inferential procedures.
The resulting distribution depends on the model assumptions, such as independence and the form of the population distribution. If those assumptions are not appropriate, the derived distribution may no longer be accurate.
5.3 Simulation and resampling
When exact derivations are difficult, simulation and resampling methods can approximate sampling distributions. These methods repeatedly generate samples or pseudo-samples and compute the statistic of interest each time. The resulting collection of values estimates the distribution empirically.
5.3.1 Monte Carlo methods
Monte Carlo methods use random simulation to study the behavior of a statistic under a chosen model. By generating many repeated samples, analysts can approximate means, variances, quantiles, and other features of the sampling distribution.
These methods are especially useful for complex statistics, unusual models, or situations where closed-form formulas are unavailable.
5.3.2 Bootstrap methods
Bootstrap methods resample from the observed data, usually with replacement, to approximate the sampling distribution of a statistic. This approach treats the sample as a stand-in for the population and uses repeated resampling to estimate uncertainty.
The bootstrap is widely used because it can be applied to many statistics without requiring strong mathematical assumptions. It is especially helpful when sample sizes are moderate and theoretical formulas are hard to obtain.
6 Applications in statistical inference
Sampling distributions are the foundation of many inferential tools. They allow researchers to quantify uncertainty, compare hypotheses, and decide how much confidence to place in sample results.
6.1 Estimation of population parameters
Point estimates from samples are used to estimate population parameters, and sampling distributions indicate how accurate those estimates are likely to be. A statistic with a narrow sampling distribution is more precise than one with a wide distribution.
This information helps analysts choose estimators and evaluate their performance in terms of bias, variance, and overall reliability.
6.2 Confidence intervals
Confidence intervals are constructed using the sampling distribution of an estimator. They provide a range of plausible values for a population parameter based on sample data and a specified confidence level.
The width of a confidence interval depends on the standard error and the shape of the sampling distribution. Better precision generally leads to narrower intervals.
6.3 Hypothesis testing
In hypothesis testing, the sampling distribution under the null hypothesis is used to judge how surprising an observed statistic would be if the null were true. This comparison leads to a p-value or decision rule.
The test’s validity depends on having an appropriate sampling distribution, either exact or approximate. If the distribution is misspecified, test results may be misleading.
6.4 Margin of error
The margin of error is a measure of the expected difference between a sample estimate and the true parameter due to random sampling. It is closely tied to the standard error and the selected confidence level.
Sampling distributions provide the basis for calculating or approximating this quantity. In surveys and polling, the margin of error is often reported as a summary of statistical uncertainty.
7 Related concepts
Several concepts are closely connected to sampling distributions. They often appear together in statistical explanations and are important for understanding variability in data analysis.
7.1 Standard error
Standard error is the standard deviation of a statistic’s sampling distribution. It measures the typical amount by which the statistic varies from sample to sample. A smaller standard error indicates greater precision.
It should not be confused with the standard deviation of the data itself, which describes spread within a single sample or population rather than variation in a statistic across samples.
7.2 Sampling bias
Sampling bias occurs when the method of selecting a sample systematically favors certain outcomes or groups. In such cases, the sampling distribution may no longer reflect the intended population process.
Bias is a problem of sample selection rather than random variation. Even a statistic with small sampling variability can be inaccurate if the sample is not representative.
7.3 Sampling error
Sampling error refers to the random difference between a sample statistic and the corresponding population parameter. It arises naturally because samples are only partial observations of the population.
Unlike bias, sampling error is unavoidable in ordinary sampling, but it can often be reduced by increasing sample size. Sampling distributions quantify the expected extent of this random variation.
7.4 Population distribution
The population distribution describes how values are distributed across the entire population. It is the starting point from which samples are drawn. The sampling distribution of a statistic depends partly on this underlying distribution.
Although the population distribution concerns individual observations, the sampling distribution concerns statistics computed from groups of observations. The two are related but distinct.
8 Limitations and assumptions
Sampling distributions rely on assumptions about how data are collected and how the underlying model behaves. When these assumptions fail, conclusions drawn from the sampling distribution may be less reliable.
8.1 Random sampling
Random sampling helps ensure that each member of the population has a known chance of being selected. This supports the use of probability theory in deriving sampling distributions and reduces the risk of systematic distortion.
Without random sampling, the resulting distribution of a statistic may reflect selection effects rather than genuine population variability.
8.2 Independence of observations
Many sampling distribution results assume that observations are independent. Independence means that one observation does not directly determine another. This assumption simplifies derivations and is essential for many classical formulas.
When observations are dependent, such as in clustered or repeated-measures data, the effective sampling distribution may differ substantially from the standard one.
8.3 Finite population effects
When samples are drawn without replacement from a finite population, the variability of a statistic can be smaller than in an infinite-population model. This finite population effect becomes more pronounced when the sample is a large fraction of the total population.
Adjustments may be needed to account for this reduction in variability, especially in survey sampling and audit settings.
8.4 Model assumptions
Exact and approximate sampling distributions often depend on assumptions about the population form, such as normality, equal variance, or independent errors. These assumptions influence both the center and spread of the distribution.
If the assumptions are only approximately true, the methods may still work well. If they are badly violated, however, the sampling distribution may no longer provide an accurate guide for inference.