1 Fundamentals of Confidence Intervals
1.1 Definition and interpretation
A confidence interval (CI) is a random interval constructed from sample data that is intended to bracket an unknown population parameter. Because the interval endpoints vary from sample to sample, the interval is treated as a procedure rather than a fixed statement about a single dataset.
For a chosen confidence level, the procedure is designed so that, in repeated sampling under the same conditions, the constructed intervals contain the true parameter a corresponding fraction of times. In practice, after observing data, the analyst reports the realized interval as an estimate of where the parameter likely lies.
1.2 Confidence level and coverage probability
The confidence level (commonly 90%, 95%, or 99%) is linked to the interval’s coverage probability. Coverage probability is the probability, under the assumed data-generating model and sampling scheme, that the interval includes the true parameter.
A CI does not assign a probability directly to the unknown parameter being inside the reported numerical bounds. Instead, it reflects how often the method would succeed in the long run if repeated indefinitely under the same assumptions.
1.3 Point estimate and margin of error
Most confidence intervals are built from:
- a point estimate (such as a sample mean or sample proportion), and
- a margin of error that scales the estimated standard uncertainty.
A common form is \[ \text{estimate} \pm \text{critical value} \times \text{standard error}. \] The critical value depends on the target confidence level and the reference distribution being used. The standard error reflects how variability in the estimator shrinks as sample size grows.
1.4 Assumptions and conditions
Confidence intervals rely on modeling and sampling assumptions. Examples include:
- independence (or dependence structures accounted for),
- correct distributional form (exact methods) or conditions for approximation (asymptotic methods),
- adequate sample size for approximations,
- correct variance assumptions or consistent estimators of uncertainty.
When assumptions fail—such as heavy skewness with very small samples, extreme outliers, or dependence that is ignored—the stated confidence level may not match the true coverage.
2 Types of Confidence Intervals
2.1 One-parameter intervals
2.1.1 Mean (normal-based) intervals
When sampling from a normal population with unknown variance, CIs for the mean often use a t reference distribution; if the variance is known, the normal (z) distribution is appropriate. For approximately normal data, large-sample methods may yield valid intervals even if the underlying distribution is not exactly normal.
The interval width increases with variability and decreases with sample size, reflecting the standard error’s dependence on both.
2.1.2 Proportion intervals
For binomial proportions, CI construction is influenced by how success/failure variability is modeled. Classical approaches include normal approximations (with or without continuity corrections) and exact or near-exact constructions. Choice depends on whether the sample proportion is near 0 or 1 and on sample size.
The goal is to account for the discreteness and bounded nature of proportions.
2.1.3 Variance and standard deviation intervals
Variance-related intervals require careful handling because variance is nonnegative and its sampling distribution can be skewed. For normal data, exact CIs commonly use distributions derived from the chi-square statistic. For broader contexts, approximate or transformation-based methods are used, sometimes combined with robustness considerations.
2.2 Two-parameter and group comparisons
2.2.1 Difference of means
To compare two population means, CIs are built for \(\mu_1-\mu_2\) using estimated standard errors that depend on within-group variability and sample sizes. If variances are assumed equal, pooled approaches can be used; otherwise, methods that allow unequal variances are typically preferred.
Interpretation focuses on whether the interval includes zero, while emphasizing that inclusion is a statement about uncertainty rather than a probability claim about the parameter.
2.2.2 Ratio or difference of proportions
For proportions, group comparisons may target differences (\(p_1-p_2\)) or ratios (\(p_1/p_2\)). Ratio intervals are often constructed on a transformed scale (such as log scale) to stabilize variance and ensure sensible behavior away from boundaries.
As with single-proportion CIs, the accuracy depends strongly on sample size and event frequency.
2.2.3 Paired-data intervals
When the same subjects are measured before and after an intervention or matched across conditions, paired CIs use the distribution of within-pair differences. This approach can reduce variability compared with treating observations as independent across groups, provided the pairing is meaningful.
2.3 Regression-related confidence intervals
2.3.1 Coefficient confidence intervals
In linear regression, CIs for regression coefficients quantify uncertainty in how predictors relate to the outcome. They depend on estimated residual variance, the design matrix, and assumptions such as linearity, independence of errors, and appropriate error distribution or large-sample behavior.
A coefficient CI reflects both sampling variability and model specification, not merely the noise in observed outcomes.
2.3.2 Prediction interval vs. confidence interval
A confidence interval for a mean response focuses on uncertainty about the expected value at a given predictor setting. A prediction interval instead quantifies uncertainty about a new individual outcome, incorporating both uncertainty in the mean and the irreducible variability around it.
Thus, prediction intervals are typically wider than confidence intervals for the mean.
3 Mathematical Foundations
3.1 Sampling distributions and standard errors
Many CI derivations use the sampling distribution of an estimator. The standard error is a measure of dispersion in that sampling distribution, often estimated from the data when the true variability is unknown.
A central idea is that, after normalization by an appropriate standard error, the estimator behaves according to a known or approximable reference distribution.
3.2 Central Limit Theorem and asymptotics
The Central Limit Theorem (CLT) underpins many large-sample CI constructions. Under broad conditions, standardized sums and many smooth functions of sample averages converge toward a normal distribution as sample size increases.
Asymptotic approximations lead to “approximate” confidence levels that become more accurate as sample size grows, even when exact formulas are unavailable.
3.3 pivotal quantities and transformation
A pivotal quantity is a statistic whose distribution does not depend on unknown parameters. Exact CI methods often arise by identifying such statistics and using their reference distributions to determine critical values.
Transformations are also common: if a parameter is better treated on a transformed scale (e.g., log scale for ratios), the resulting interval may achieve better symmetry or more stable coverage.
3.4 Exact vs approximate methods
Exact methods provide coverage without relying on asymptotic approximations, typically under strong distributional assumptions (such as normality for mean/variance CIs). Approximate methods rely on asymptotic reasoning, approximate normality, or large-sample behavior.
In practice, analysts weigh simplicity, assumptions, and the degree of approximation error given the sample size and data characteristics.
3.5 Normal, t, and other reference distributions
Common reference distributions include:
- the normal distribution for z-based intervals,
- the t distribution for mean intervals with unknown variance under normality,
- the chi-square distribution for variance-related exact intervals under normality,
- binomial-related discrete distributions or approximations for proportions,
- F and related distributions in regression contexts.
Selecting the right reference distribution is essential for aligning the nominal and achieved coverage.
4 Construction Methods
4.1 Known-variance (z) intervals
When the population variance is known (a rare condition in practice), the standard error is fixed, and the standardized estimator follows a normal distribution. The resulting CI endpoints are formed using a normal critical value tied to the desired confidence level.
This method provides an instructive benchmark and becomes an approximation when variance is replaced by a consistent estimate.
4.2 Unknown-variance (t) intervals
With unknown variance and normal sampling, the standardized mean uses a t distribution whose degrees of freedom depend on sample size. The t critical value grows slightly for smaller samples, widening intervals appropriately.
For non-normal data, t-based intervals may still be reasonable under approximate normality or with sufficiently large samples.
4.3 Bootstrap confidence intervals
The bootstrap constructs CIs by resampling from the observed data and recalculating the estimator across many resamples. Different bootstrap CI styles exist, including percentile and bias-corrected variants.
Bootstrap methods can handle complex estimators and nonstandard distributions, but their accuracy depends on sample size and whether the resampling scheme captures the underlying uncertainty correctly.
4.4 Likelihood-based intervals
Likelihood-based approaches use the likelihood function to derive intervals, often via curvature (Wald-type) approximations or profile likelihood methods. In some settings, likelihood ratio ideas lead to intervals derived from test statistics with known asymptotic distributions.
These methods are common in generalized linear models and other parametric frameworks.
4.5 Transform-based intervals (e.g., log, arcsine)
Transform-based constructions modify the scale of estimation to improve interval behavior. For example, ratio or relative-risk intervals are frequently built using a log transformation, yielding more stable variance and symmetric behavior on the transformed scale.
For proportions, transformations such as the arcsine square-root have historical usage; modern practice often emphasizes methods that perform well near boundaries.
5 Practical Computation and Reporting
5.1 Selecting an appropriate confidence level
Choice of confidence level is a trade-off: higher confidence yields wider intervals and therefore more conservative conclusions. The appropriate level depends on the context, the cost of being wrong, and conventions in the field.
In many applied settings, 95% is a default, but 90% or 99% may be used to reflect different tolerance for uncertainty.
5.2 Sample size and precision planning
Precision planning uses CI width or margin of error as a target. For mean estimation, interval width decreases roughly like \(1/\sqrt{n}\), so doubling sample size improves precision but with diminishing returns.
Planning is also influenced by the expected variance, effect size, and assumptions about distributional form.
5.3 Handling skewness and small samples
Skewness and outliers can distort normal-based intervals. With small samples, asymptotic approximations may be inadequate. Remedies include using transformations, robust variance estimators, exact or near-exact methods, or bootstrap procedures designed to handle irregular distributions.
Careful diagnostics and sensitivity checks help ensure that reported uncertainty is credible.
5.4 Reporting format and units
A standard reporting convention includes:
- the point estimate,
- the lower and upper bounds,
- the confidence level (e.g., “95% CI”).
Units should be consistent with the parameter being estimated. For transformed-scale intervals, it is typically preferable to report back-transformed bounds in the original scale to avoid confusion.
5.5 Common software outputs and parameters
Statistical software often reports confidence intervals for parameters such as means, regression coefficients, and predictions. Understanding the output requires knowing:
- whether the interval is for a mean or a single observation,
- whether assumptions like normality or homoscedasticity were used,
- how degrees of freedom were chosen,
- how standard errors were computed.
Analysts should verify settings such as confidence level, model type, and the method used for interval computation.
6 Interpreting and Using Confidence Intervals
6.1 Misinterpretations to avoid
Common misconceptions include:
- treating the confidence level as the probability that the parameter lies within the numeric interval,
- assuming that non-overlap of two CIs implies a statistically significant difference without considering dependence and interval construction,
- ignoring that CIs are only guaranteed under the method’s assumptions.
A CI is a summary of uncertainty under a specific procedure, not a universal statement about truth.
6.2 Relationship to hypothesis testing
For many standard settings, two-sided hypothesis tests at a given significance level correspond closely to confidence intervals at a related confidence level. For example, rejecting a null value in a two-sided test often corresponds to the confidence interval excluding that null value.
However, this equivalence can break for nonstandard methods, one-sided intervals, or when the interval is not derived from a test statistic in a straightforward way.
6.3 Multiple comparisons and family-wise coverage
When many parameters are estimated simultaneously, individual CIs at 95% confidence do not guarantee high probability that all intervals simultaneously contain their true values. This is a form of multiple comparison problem.
Simultaneous confidence intervals and procedures controlling family-wise error are designed to address this, though they typically widen intervals.
6.4 Statistical significance vs practical significance
A narrow interval that excludes a null value indicates precision and statistical evidence, but it does not automatically imply that the effect size is meaningful for real decisions. Practical significance depends on magnitude, context, and costs/benefits.
Reporting both effect sizes and uncertainty helps distinguish detectability from relevance.
6.5 Communication with non-technical audiences
Presenting confidence intervals often benefits from plain-language phrasing such as “a range of plausible values given the data and assumptions.” Visual tools like error bars or uncertainty bands can convey interval width as a proxy for precision.
It is also useful to clarify that the interval reflects uncertainty about an underlying quantity, and that results can change with different samples.
7 Advanced Topics
7.1 Confidence intervals under complex sampling
In surveys with stratification, clustering, or unequal selection probabilities, standard CI formulas that assume simple random sampling may be invalid. Complex-sampling CIs incorporate design weights and variance estimation methods that reflect the survey mechanism.
Techniques often rely on specialized variance estimators and adjustments tailored to the sampling design.
7.2 Robust and nonparametric intervals
Robust intervals aim to maintain coverage under weaker assumptions, such as allowing certain forms of heteroskedasticity or deviations from idealized distributional models. Nonparametric intervals attempt to reduce reliance on specific parametric forms of the data-generating process.
These approaches may trade off tightness for reliability across a broader class of underlying distributions.
7.3 Simultaneous confidence intervals
Simultaneous intervals provide coverage for multiple parameters at once, typically ensuring that the probability all true values fall within their respective intervals meets a target level. Methods include Bonferroni-type corrections and adjustments based on the joint distribution of estimators.
Because they guard against multiplicity, simultaneous intervals tend to be wider than marginal (individual) intervals.
7.4 Bayesian credible intervals vs confidence intervals
Bayesian credible intervals are posterior probability intervals for parameters given observed data and a prior distribution. Confidence intervals, by contrast, are frequentist procedures defined by long-run coverage properties under repeated sampling.
Both frameworks can produce numerically similar intervals in some settings, but they differ conceptually: credible intervals describe probability conditional on data, while confidence intervals describe coverage properties of a method over hypothetical repetitions.
7.5 Coverage properties and interval performance
Interval performance is evaluated using coverage probability, average interval width, and behavior under model misspecification. Coverage can be exact, asymptotically correct, or improved in finite samples depending on the construction method.
Modern discussions emphasize that achieved coverage may differ from nominal confidence, especially when assumptions are only approximately satisfied.
8 Worked Examples
8.1 CI for a sample mean (step-by-step)
1 Fundamentals of Confidence Intervals
2 Types of Confidence Intervals
3 Mathematical Foundations
4 Construction Methods
5 Practical Computation and Reporting
\[ \bar{x} \pm t_{\alpha/2,\,n-1}\frac{s}{\sqrt{n}}. \]
6 Interpreting and Using Confidence Intervals
8.2 CI for a population proportion (step-by-step)
1 Fundamentals of Confidence Intervals
2 Types of Confidence Intervals
\[ \hat{p} \pm z_{\alpha/2}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}. \] Alternative methods (exact or transformed) may be preferable for small samples or extreme proportions.
3 Mathematical Foundations
4 Construction Methods
5 Practical Computation and Reporting
6 Interpreting and Using Confidence Intervals
8.3 CI for a regression coefficient
1 Fundamentals of Confidence Intervals
2 Types of Confidence Intervals
3 Mathematical Foundations
4 Construction Methods
\[ \hat{\beta} \pm t_{\alpha/2,\text{df}}\cdot \text{SE}(\hat{\beta}). \]