1 Introduction to Credible Intervals

1.1 Bayesian interval estimation vs. frequentist confidence intervals

Credible intervals arise in Bayesian statistics when an unknown parameter is treated as random and updated using observed data. A credible interval summarizes uncertainty about the parameter after incorporating both the data likelihood and prior beliefs via Bayes’ rule. In contrast, frequentist confidence intervals are built so that—under repeated sampling from the data-generating process—the interval procedure covers the true parameter at a prescribed long-run frequency. The two interval types can coincide in special cases but are conceptually defined in different probability frameworks.

1.2 Posterior probability and interpretation

The defining feature of a Bayesian credible interval is its probabilistic meaning relative to the posterior distribution. After computing the posterior for the parameter, a credible interval is chosen to contain parameter values that together carry a specified posterior probability mass (for example, 95%). This makes it natural to interpret the interval as a statement about plausible parameter values given the observed data and the prior.

1.3 Assumptions behind interval construction

Although credible intervals are Bayesian objects, their validity depends on modeling choices and computational adequacy. Key assumptions include selecting a likelihood model for the data, specifying a prior distribution, and performing inference that accurately approximates the posterior. The interval’s behavior can also reflect whether the posterior is well-behaved (e.g., unimodal, sufficiently regular) and whether the numerical method used to compute posterior summaries is converged.

2 Mathematical Foundations

2.1 Posterior distribution of a parameter

Let \( \theta \) denote an unknown parameter and \( y \) denote observed data. Bayes’ rule gives the posterior distribution \[ p(\theta \mid y) \propto p(y \mid \theta)\, p(\theta), \] where \( p(y \mid \theta) \) is the likelihood and \( p(\theta) \) is the prior. Credible intervals are derived from this posterior by selecting subsets of parameter space that achieve a target probability level.

2.2 Definition of a credible interval (general form)

For a scalar parameter, a \(100(1-\alpha)\%\) credible interval is typically defined as an interval \([a,b]\) such that \[ \Pr(\theta \in [a,b] \mid y) = 1-\alpha \] under the posterior distribution. For multivariate parameters, the analogue is a credible set: a region of parameter space containing a specified posterior probability mass.

2.3 Credibility levels and notation

Common notation includes \(1-\alpha\) as the posterior mass of the interval (e.g., \(0.95\) for a 95% credible interval). In practice, the same credibility level can lead to different intervals depending on the chosen construction rule, especially when the posterior distribution is skewed or multimodal.

2.4 Conditioning on data and priors

Credible intervals are conditional on both the data and the prior model. Changing the prior can alter the posterior distribution and thus the interval. This dependency motivates sensitivity analysis and careful reporting of the prior assumptions, particularly in applications where prior information is substantive or where data are limited.

3 Types of Credible Intervals

3.1 Equal-tailed credible intervals

An equal-tailed credible interval splits the remaining posterior probability mass symmetrically between the tails. For a 95% equal-tailed interval, the bounds satisfy \[ \Pr(\theta < a \mid y) = 0.025,\quad \Pr(\theta > b \mid y) = 0.025. \] This construction is straightforward for continuous posteriors and aligns with quantile-based reporting.

3.2 Highest posterior density (HPD) intervals

An HPD interval is defined as the set of parameter values with the highest posterior density that accumulates to the desired probability mass. For unimodal posteriors, HPD intervals are often shorter than equal-tailed intervals because they concentrate probability in regions where the posterior is densest. For multimodal distributions, HPD sets can be disjoint, complicating the representation as a single contiguous interval.

3.3 Symmetric vs. asymmetric intervals

Some models yield posterior distributions that are approximately symmetric around a central tendency, leading to intervals that appear symmetric on the natural parameter scale. However, many practical posteriors are skewed, producing asymmetric credible intervals even when the central tendency is meaningful (e.g., mean, median, or mode). The selected interval type further influences apparent asymmetry.

3.4 Transformations and invariance considerations

Credible intervals are not always invariant under reparameterization. If one reports intervals on a transformed scale (such as log scale for positive parameters), the bounds change because “equal-tailed” or “HPD” selection is tied to the posterior distribution on that scale. Analysts typically report intervals on a scale that matches the interpretability needs of the problem and may also provide intervals under transformations for better communication.

4 Construction Methods

4.1 Closed-form solutions for simple models

In conjugate Bayesian models, posterior distributions often have known analytic forms. This makes it possible to compute credible interval bounds using closed-form quantiles or analytic expressions for the posterior CDF. Such cases include Beta-Binomial, Gamma-Poisson, and Normal-Normal conjugacy, where credible intervals can be computed efficiently.

4.2 Numerical integration approaches

When analytic quantiles are unavailable but the posterior can be evaluated pointwise, numerical integration can approximate the posterior CDF and invert it to find interval bounds. Approaches include adaptive quadrature and numerical root-finding for quantiles. Accuracy depends on the resolution of the posterior evaluation and the stability of integration near the tails.

4.3 Simulation-based interval estimation (e.g., MCMC)

Markov chain Monte Carlo generates draws from the posterior, enabling interval estimation from empirical quantiles or density-based criteria. For equal-tailed intervals, bounds can be taken as sample quantiles of the posterior draws. For HPD intervals, one may estimate density from samples and search for the shortest region containing the required posterior mass, with careful attention to sampling error.

4.4 Grid approximation and posterior sampling diagnostics

For low-dimensional parameters, grid-based methods approximate the posterior by evaluating it on a discretized parameter space and normalizing. Credible intervals then follow from cumulative probability computed over the grid. When using MCMC or other sampling methods, diagnostics such as trace plots, effective sample size estimates, and checks for autocorrelation help ensure the interval reflects the posterior rather than artifacts of insufficient mixing.

5 Example Models and Use Cases

5.1 Bernoulli/Binomial proportion with Beta priors

Consider observing \(y\) successes out of \(n\) Bernoulli trials. With a Beta prior \( \theta \sim \text{Beta}(\alpha,\beta) \), the posterior is \( \theta \mid y \sim \text{Beta}(\alpha+y,\beta+n-y) \). Credible intervals can be obtained from Beta quantiles for equal-tailed intervals, while HPD intervals can be computed by finding density-maximizing regions that accumulate to the desired posterior probability. This example illustrates how prior pseudo-counts shift posterior uncertainty and central location.

5.2 Normal mean with conjugate priors

For data \(y_i\) modeled as \(y_i \sim \mathcal{N}(\mu,\sigma^2)\) with known \(\sigma^2\), a conjugate Normal prior \(\mu \sim \mathcal{N}(\mu_0,\tau_0^2)\) yields a Normal posterior for \(\mu\). Credible intervals then correspond to quantiles of that posterior. When \(\sigma^2\) is unknown, conjugate extensions lead to Student-\(t\) forms, again enabling analytic or near-analytic interval construction.

5.3 Linear regression and posterior uncertainty

In Bayesian linear regression, coefficients are updated from a prior using the regression likelihood. With Gaussian priors and Gaussian noise, the posterior is multivariate normal (under standard assumptions), allowing credible intervals for each coefficient and credible bands for predictions. Even when conjugate priors simplify inference, the posterior covariance encodes correlations among coefficients, which affects multivariate uncertainty quantification and joint credible regions.

5.4 Hierarchical models and partial pooling

Hierarchical models introduce group-level parameters so that estimates for each group borrow strength from others. Credible intervals reflect both within-group data variability and between-group uncertainty. This partial pooling typically reduces overfitting to small-sample groups, producing intervals that can be narrower than naive estimates when information is shared, yet wider when data signal substantial heterogeneity.

6 Practical Considerations

6.1 Choice of prior and sensitivity analysis

Prior selection influences credible intervals, particularly when the likelihood is weak or data are sparse. Sensitivity analysis examines how intervals change under alternative plausible priors. A good practice is to separate prior-driven uncertainty from data-driven uncertainty by comparing posterior summaries across prior choices, thereby avoiding a misleading sense of precision.

6.2 Assessing posterior convergence and effective sample size

For simulation-based inference, credible intervals depend on the quality of posterior draws. If chains have not converged or if autocorrelation is high, interval bounds can be biased or overly certain. Effective sample size provides a measure of how many independent draws the correlated samples approximate, guiding whether additional computation is warranted.

6.3 Multimodal posteriors and interval ambiguity

When a posterior has multiple modes, different credible interval definitions can produce materially different results. Equal-tailed intervals may span low-density regions, while HPD intervals may become disjoint sets. Reporting can require either summarizing the modal structure, providing multiple intervals, or using credible sets rather than forcing a single contiguous range.

6.4 Reporting intervals for multiple parameters

In multivariate settings, reporting marginal credible intervals for each component is common, but it ignores dependence among parameters. Analysts may also report joint credible regions or summaries of predictive quantities. When multiple parameters are reported, clarity about whether intervals are marginal or joint, and on what scale they are computed, helps prevent misinterpretation.

7 Communication and Visualization

7.1 Plotting posterior distributions with intervals

A standard visualization overlays the posterior density or histogram with vertical lines marking credible interval bounds. For quantile-based intervals, the bounds correspond directly to posterior CDF levels. For HPD intervals, marking boundaries requires more care when the HPD region is not contiguous. In time series or regression contexts, plotting credible bands for fitted curves communicates uncertainty across the domain of interest.

7.2 Credible interval coverage as a descriptive measure

“Coverage” is frequently discussed for confidence intervals, but credible interval coverage can still be evaluated descriptively by simulation studies under a known data-generating process. Such evaluations clarify how often Bayesian intervals capture the true parameter when the model is correct or when misspecification occurs. The result is not a defining property of credible intervals, but it can help assess performance.

7.3 Common pitfalls in interpretation

A frequent pitfall is treating credible intervals as though they guarantee correct long-run frequency coverage in the frequentist sense. Another issue is confusing posterior probability statements with statements about future data without appropriate posterior predictive calculations. Clear definitions and careful wording—explicitly referencing “given the observed data and the prior”—help reduce ambiguity.

7.4 Guidelines for clear statistical reporting

Good reporting practice includes stating the credible level, the interval type (equal-tailed vs HPD), the parameterization or scale, and the prior model. For simulation-based intervals, describing convergence diagnostics and the effective sample size is useful when interval width is sensitive to sampling quality. When multiple parameters or nonlinear transformations are involved, it is also helpful to specify whether intervals are marginal or joint.

8.1 Bayesian credible sets and generalizations

Credible sets extend the idea of credible intervals to arbitrary dimensions and parameter spaces. Instead of an interval on a line, a credible set is a region that captures a chosen posterior probability mass. This generalization is particularly relevant for complex models where dependence among parameters is central.

8.2 Posterior predictive intervals

Posterior predictive intervals quantify uncertainty about future observations rather than about parameters. They integrate over the posterior distribution of parameters and the likelihood of new data. While credible intervals summarize where parameters likely lie, predictive intervals summarize where new data values are likely to fall given current modeling assumptions.

8.3 Uncertainty quantification in Bayesian workflows

Credible intervals are one component of a broader uncertainty quantification workflow in Bayesian modeling. They often complement other summaries such as posterior means, medians, and full posterior samples used for simulation. In decision-oriented settings, uncertainty about both parameters and predictions supports robust choices and communicates risk.

8.4 Model comparison and interval-based conclusions

Credible intervals can influence conclusions when comparing models or deciding among competing hypotheses, but they should not be treated as definitive evidence by themselves. Bayes factor and other model comparison tools evaluate models directly, while interval estimates help describe how parameter uncertainty differs across models. Interval-based conclusions are strongest when aligned with a coherent decision criterion and validated with predictive performance.