1 Introduction to Cluster Bootstrap

1.1 Motivation: Dependence and the Limits of IID Resampling

Many statistical quantities are computed from data assumed to be independent and identically distributed (IID). Standard bootstrap methods exploit this by resampling observations independently and estimating uncertainty from the resulting variability. In clustered or grouped datasets, however, observations within the same cluster often share common shocks, selection mechanisms, measurement processes, or unobserved factors. This produces within-cluster dependence that IID resampling does not preserve, potentially leading to underestimation or overestimation of uncertainty.

Cluster bootstrap addresses this mismatch by treating each cluster as the basic resampling unit. By resampling at the level at which dependence is strongest, the method aims to reflect the dependence structure more faithfully in the resampled data.

1.2 Core Idea: Resample Clusters Instead of Points

The defining operation in cluster bootstrap is to generate bootstrap replicates by selecting clusters—often with replacement—and recombining their data to form pseudo-samples. Within each chosen cluster, all observations belonging to that cluster are kept together, ensuring that any internal correlation remains intact.

The approach is therefore “block-like” but in the sense of grouping: clusters play the role of blocks that are resampled as whole units. The target statistic (e.g., a regression coefficient, a mean difference, or a distributional functional) is recomputed on each replicate, and the empirical spread across replicates is used to approximate sampling uncertainty.

1.3 When to Use: Typical Data-Generating Scenarios

Cluster bootstrap is commonly used when the natural unit of dependence is a group. Typical scenarios include:

  • Panel or longitudinal studies, where observations are repeatedly collected on the same subject.
  • Multilevel or hierarchical designs, such as students within schools or patients within clinics.
  • Clustered experiments, where randomization occurs at the group level rather than at the individual level.
  • Studies with geographic or organizational grouping, such as firms within regions or households within communities.

The method is most relevant when within-group correlation is nontrivial and standard IID bootstrap would not accurately mimic the data’s dependence structure.

1.4 Terminology and Notation for Clustered Data

A typical setup includes:

  • Clusters indexed by \(g = 1,\dots,G\).
  • Within each cluster \(g\), observations \( \{(Y_{gi}, X_{gi}) : i = 1,\dots,n_g\} \), where \(n_g\) is the number of observations in cluster \(g\).
  • A parameter or statistic of interest \( \theta = T(\text{data}) \), estimated by \( \hat{\theta} \).

Cluster bootstrap uses the clusters \(\{1,\dots,G\}\) as sampling units. Resampling produces bootstrap indices \(g_1^\*,\dots,g_G^\*\) (often of size \(G\)), and the pseudo-sample consists of all observations from the selected clusters.

2 Methodology

2.1 Basic Cluster-Resampling Algorithm

2.1.1 Defining Clusters and Mapping Observations to Groups

The first step is to define what constitutes a cluster. Conceptually, a cluster should group observations that share dependence due to shared latent factors or common error components. Operationally, data must be mapped so every observation is assigned to exactly one cluster (or, in some settings, to multiple overlapping clusters with additional complexity).

Once clusters are defined, the analyst’s mapping ensures that resampling at the cluster level corresponds to a coherent reassembly of the dataset.

2.1.1.1 Bootstrap Replicates and Recomputing the Target Statistic

Let the target statistic be computed as \( \hat{\theta} = T(\{ \mathcal{D}_g \}_{g=1}^G ) \), where \(\mathcal{D}_g\) denotes the data from cluster \(g\). For bootstrap replicate \(b = 1,\dots,B\), the algorithm creates a resampled dataset and recomputes the statistic: \[ \hat{\theta}^{(b)} = T(\{ \mathcal{D}_{g_1^{(b)}} , \dots, \mathcal{D}_{g_G^{(b)}} \}). \] The collection \(\{\hat{\theta}^{(b)}\}_{b=1}^B\) becomes the empirical basis for uncertainty quantification.

2.1.2 Sampling Clusters with Replacement

Cluster bootstrap commonly samples the cluster indices with replacement. A standard practice draws \(G\) cluster indices from \(\{1,\dots,G\}\) uniformly with replacement, producing bootstrap replicate \(b\). This preserves the idea that the empirical distribution over clusters approximates the underlying sampling distribution, while allowing repeats of clusters and omitting others.

Variants exist (e.g., subsampling without replacement, or drawing fewer clusters), but with-replacement resampling is the most common default.

2.1.3 Handling Multiple Observations per Cluster

Because the unit of resampling is the cluster, all observations within a selected cluster are included together in the replicate. If clusters have multiple observations, the algorithm uses the entire set for that cluster rather than resampling individual observations.

This approach automatically respects within-cluster correlation patterns that the analyst intends to preserve. The only randomness introduced by the bootstrap is which clusters appear in the replicate and how many times each appears.

2.2 Variants of Cluster Bootstrap

2.2.1 Pairs Bootstrap vs. Residual Bootstrap in Cluster Settings

Two widely discussed families are:

  • Pairs bootstrap: resample the observed data units (cluster-level data). For clustered models, this typically means resampling entire clusters with all their observed pairs \((Y,X)\).
  • Residual bootstrap: fit a model to obtain fitted values and residuals, then resample residual-related components. In cluster contexts, residual bootstrap must be structured so that dependence is handled appropriately, often by resampling residuals at the cluster level (e.g., resampling residual vectors for each cluster) or by incorporating random effects structure.

Residual-based variants can be advantageous for certain model-based targets, but they add modeling assumptions and require careful alignment between the model’s error structure and the resampling mechanism.

2.2.2 Block/Cluster Extensions for Additional Dependence

Sometimes dependence is not fully captured by a single clustering variable. Analysts may consider:

  • Dependence across time within subject combined with cluster sampling at the subject level.
  • Dependence across clusters connected through network-like structure.
  • Additional correlation along another grouping dimension.

Extensions include block bootstrap over time within clusters, two-dimensional resampling schemes, or hierarchical resampling where multiple grouping structures are resampled jointly.

These designs aim to capture the dominant dependence directions, though they may become more complex and require more assumptions.

2.2.3 Smoothed or Weighted Cluster Bootstrap Options

When clusters differ substantially in size or influence, unweighted resampling may not reflect the intended asymptotic regime. Weighted cluster bootstrap variants introduce weights for sampling clusters or for combining observations in the replicate. Smoothed bootstrap methods may also be used to reduce artifacts from discrete resampling, for example by adding small perturbations or using continuous approximations in constructing pseudo-data.

These options can improve stability in challenging scenarios, but they change the implied resampling distribution and therefore must be chosen consistently with the inferential target.

2.3 Practical Implementation Details

2.3.1 Dealing with Unequal Cluster Sizes

Clusters often have different numbers of observations \(n_g\). Unequal sizes affect how a cluster contributes to the statistic in each replicate. Common practical strategies include:

  • Using standard cluster bootstrap directly (include all observations from selected clusters as-is).
  • Applying weights or normalization in the statistic so that contribution per cluster is comparable.
  • Adjusting resampling schemes (e.g., sampling clusters with probabilities that reflect size) when theory supports such an approach.

Choice depends on the statistic and on which asymptotic framework is considered appropriate (e.g., fixed within-cluster size versus increasing cluster size).

2.3.2 Choice of Number of Bootstrap Replications \(B\)

The number of replicates \(B\) controls Monte Carlo error in estimated standard errors and interval endpoints. Too small a \(B\) may produce noisy uncertainty estimates. In practice, analysts choose \(B\) large enough that results stabilize (often using a few hundred to several thousand replicates depending on computation time and desired precision).

It is also common to monitor convergence of standard error estimates as \(B\) increases.

2.3.3 Computational Considerations and Efficiency

Cluster bootstrap can be computationally intensive because each replicate requires refitting or recomputing the statistic on a pseudo-sample containing many observations. Efficiency improves when:

  • The statistic can be computed quickly once clusters are selected.
  • The model fitting supports warm starts or fast re-estimation.
  • Parallel computing is used to distribute bootstrap replicates across cores.

Memory management can also matter when clusters are large or when the statistic requires storing intermediate objects.

3 Inference Using Bootstrap Output

3.1 Estimating Standard Errors from Replicates

Given bootstrap estimates \(\hat{\theta}^{(1)},\dots,\hat{\theta}^{(B)}\), a bootstrap standard error is typically computed as the sample standard deviation: \[ \widehat{\mathrm{SE}}_{\text{boot}} = \sqrt{\frac{1}{B-1}\sum_{b=1}^B \left(\hat{\theta}^{(b)} - \bar{\theta}^\*\right)^2}, \] where \(\bar{\theta}^\* = \frac{1}{B}\sum_{b=1}^B \hat{\theta}^{(b)}\).

This produces an empirical measure of uncertainty that accounts for variability induced by cluster resampling. For multivariate targets, analogous covariance estimates are obtained using replicate vectors.

3.2 Confidence Intervals

3.2.1 Percentile Intervals

The percentile method constructs an interval from the empirical quantiles of \(\{\hat{\theta}^{(b)}\}\). If \(q_{\alpha/2}^\*\) and \(q_{1-\alpha/2}^\*\) denote the corresponding quantiles, the \(1-\alpha\) percentile interval is \([q_{\alpha/2}^\*,\, q_{1-\alpha/2}^\*]\).

This method is simple and does not require explicit standardization, but it can be sensitive to bias and skewness in the bootstrap distribution.

3.2.2 Bias-Corrected and Accelerated (BCa) Intervals

BCa intervals attempt to correct for both bias and acceleration effects by transforming percentile cutoffs using quantities computed from bootstrap replicates and influence-like measures. The technique is designed to improve coverage when the estimator’s distribution is skewed or biased.

Implementing BCa in clustered settings usually relies on having a valid bootstrap distribution generated from the cluster resampling mechanism, but the calculation of acceleration may require additional diagnostics or careful handling to align with the statistic.

3.2.3 Studentized Intervals and Variance Stabilization

Studentized intervals standardize the estimator by an estimate of its variability. For cluster bootstrap, one may compute a bootstrap estimate of the standard error for each replicate and use these to form a pivot. This can improve performance when the variance depends on the parameter value or when distributional features vary across the range of estimates.

The trade-off is complexity: studentization adds extra computation and depends on stable variance estimation across replicates.

3.3 Testing Hypotheses with Cluster Bootstrap

3.3.1 Bootstrap-Based P-values

To test a hypothesis \(H_0\), one typically constructs a test statistic \(S(\text{data})\) and evaluates how extreme the observed statistic is relative to its bootstrap distribution. In the simplest approach, the p-value can be approximated by the fraction of bootstrap replicates whose test statistic is at least as extreme as the observed one (directional depending on the test).

For more accurate inference, permutation-like or re-centered bootstrap strategies may be used to better reflect the null distribution, especially when the bootstrap distribution under the observed data is biased away from the null.

3.3.2 Calibration and Multiplicity Considerations

When multiple parameters or multiple hypotheses are tested, uncertainty estimates must be combined with multiple-testing adjustments. Bootstrap output can support resampling-based calibration procedures, such as max-statistic methods that use the largest statistic across outcomes within each replicate.

Care must be taken to ensure the resampling scheme matches the dependence structure among outcomes, since ignoring this can invalidate multiplicity control.

4 Assumptions and Diagnostics

4.1 Exchangeability/Independence Across Clusters

Cluster bootstrap typically relies on the idea that clusters are exchangeable or approximately independent relative to one another. In other words, dependence is assumed to be primarily within clusters, not across clusters.

If there is strong dependence between clusters—such as shared time trends across regions or network spillovers—the resampling mechanism may not mimic the true dependence structure, affecting coverage and calibration.

4.2 Cluster Size and Number of Clusters: Asymptotic Intuition

Inferential quality often depends on both:

  • The number of clusters \(G\).
  • The within-cluster sizes \(n_g\).

Common intuition is that uncertainty estimators become more reliable when \(G\) is sufficiently large, because cluster bootstrap approximates an empirical distribution over clusters. With very few clusters, the resampling distribution can be too coarse, and bootstrap intervals and standard errors may perform poorly.

Within-cluster size matters mainly through how accurately within-cluster correlation and estimation variability are captured by recomputing the statistic on resampled clusters.

4.3 Diagnosing Poor Bootstrap Performance

4.3.1 Sensitivity to Cluster Definitions

Bootstrap results can be sensitive to how clusters are defined. If clusters are too granular, they may fail to capture dependence; if too broad, important structure may be lost or the effective number of resampling units may shrink.

Diagnostics can include experimenting with alternative cluster definitions that reflect plausible dependence mechanisms and checking whether uncertainty estimates remain stable.

4.3.2 Outliers, Leverage Clusters, and Robustness Checks

Some clusters may exert disproportionate influence due to outlying values, high leverage, or unusual variability. Cluster bootstrap will reproduce these influences in replicates only when the influential clusters are sampled, which can lead to unstable intervals.

Robustness checks may involve examining influence diagnostics, comparing bootstrap estimates under alternative resampling settings, or using robust estimators in the target statistic before applying the bootstrap.

5 Theoretical Properties (Conceptual)

5.1 Consistency and Conditions for Valid Inference

Conceptually, cluster bootstrap aims to deliver consistent approximations to the sampling distribution of \(\hat{\theta}\) (or to its suitably standardized form). Consistency typically requires:

  • Correct identification of the level at which dependence is preserved by resampling.
  • Sufficiently many clusters to stabilize the empirical resampling distribution.
  • Regularity conditions ensuring that the target statistic behaves smoothly with respect to the resampled data.

These conditions vary by statistic and model class, but the overarching theme is that resampling at the correct dependence level is necessary for validity.

5.2 Impact of Within-Cluster Correlation Strength

Within-cluster dependence influences the effective amount of information. Strong correlation reduces the “independent information” contributed by additional observations within the same cluster. Cluster bootstrap is designed to reflect this reduction by resampling whole clusters rather than individual observations.

However, extreme dependence patterns can challenge the assumptions behind the bootstrap approximation, particularly if dependence varies across clusters in ways not captured by exchangeability.

5.3 Finite-Sample Behavior and Common Failure Modes

In finite samples, cluster bootstrap can fail or degrade in several ways:

  • Too few clusters: resampled replicates do not adequately represent variability.
  • Mis-specified clustering: dependence persists across clusters or is ignored within clusters.
  • Heavy-tailed outcomes: rare extreme clusters dominate variability, producing unstable quantiles.
  • Model mismatch in residual bootstrap: residual resampling can distort distribution if the fitted model does not capture the error structure.

These failure modes are often revealed through diagnostics such as unstable intervals, sensitivity to cluster definitions, or poor performance compared with alternative methods.

6 Applied Use Cases

6.1 Regression with Clustered Errors

A common application is regression where residuals are correlated within groups (e.g., within individuals over time, within schools, or within firms). Cluster bootstrap can estimate uncertainty for regression coefficients by resampling clusters and refitting the model on each replicate.

This helps produce standard errors and confidence intervals that better reflect the correlation structure than IID bootstrap or naive formulas.

6.2 Panel Data and Repeated Measures

Panel data naturally features dependence across time within the same unit. Cluster bootstrap at the unit (e.g., subject) level resamples entire time series for each unit, preserving serial dependence patterns within units as long as those patterns are contained within the defined cluster.

Analysts must ensure that the cluster definition matches the unit over which repeated measurements occur.

6.3 Clustered Experiments and Group-Level Randomization

In group-randomized trials, randomization may be assigned to clusters, such as schools or clinics. The resulting outcomes for units within a cluster can share treatment assignment effects and thus dependence. Cluster bootstrap supports uncertainty quantification by resampling clusters, aligning the resampling mechanism with the design’s randomization unit.

This can be especially relevant when treatment effects are estimated by comparing cluster-level or aggregated outcomes.

6.4 Estimating Distributional Functionals Under Clustering

Beyond means and regression coefficients, cluster bootstrap can estimate uncertainty for distributional functionals such as quantiles, Gini-type measures, and other non-linear statistics. The method remains applicable as long as the target statistic can be recomputed on each bootstrap replicate and the resampling mechanism preserves the dominant dependence.

Performance depends on how the functional responds to extreme clusters and the number of clusters available.

7 Reporting and Reproducibility

7.1 What to Report in a Methods Section

A methods description typically includes:

  • How clusters were defined and assigned.
  • The bootstrap scheme (e.g., resampling clusters with replacement).
  • The number of replicates \(B\).
  • The interval or testing approach (percentile, BCa, studentized, etc.).
  • Any weighting or special handling for unequal cluster sizes.

These details allow readers to understand what dependence was preserved and how uncertainty was generated.

7.2 Choosing and Justifying the Bootstrap Scheme

Justification should connect the scheme to the dependence structure. For instance, if the main correlation is within subjects, a subject-level cluster bootstrap is appropriate. If dependence is more complex, a more elaborate block or weighted scheme may be warranted.

The justification can also reference practical considerations, such as computation time and stability, especially when multiple bootstrap variants are plausible.

7.3 Reproducible Workflows and Random Seeds

Reproducibility requires:

  • Setting a random seed for bootstrap resampling.
  • Documenting software versions and relevant numerical settings.
  • Saving bootstrap replicate outputs or intermediate artifacts when feasible, especially for expensive computations.

For large \(B\), it is also helpful to record whether computation was parallelized and how results were aggregated.

7.4 Interpreting Results Responsibly

Interpretation should acknowledge the resampling assumptions and the effective sample size measured by the number of clusters. When there are very few clusters, analysts should be cautious about interval accuracy and consider supplementary checks, such as alternative bootstrap variants or asymptotically motivated alternatives.

Responsible reporting also includes noting whether uncertainty estimates appear stable under reasonable perturbations of cluster definitions or resampling settings.

8 Common Pitfalls and FAQs

8.1 Mistakes: Resampling Individuals Instead of Clusters

A frequent error is to apply standard bootstrap by resampling individual observations independently. When within-cluster dependence exists, this breaks the correlation structure and can yield misleadingly narrow intervals or incorrect coverage. Cluster bootstrap avoids this by ensuring whole clusters are resampled together.

8.2 Pitfall: Too Few Clusters

Even with correct clustering, a small number of clusters can make the bootstrap distribution too discrete. As a result, standard errors may be unstable and confidence intervals may have poor coverage. This issue is often more severe for tail probabilities used in interval construction.

8.3 Pitfall: Misaligned Cluster Definitions

If the defined clusters do not reflect the true source of dependence, the bootstrap may preserve the wrong structure. This can happen when clusters are created for convenience rather than for dependence. Diagnostics based on substantive knowledge and sensitivity analyses can help detect such misalignment.

8.4 FAQ: How to Choose Between Bootstrap Variants

A practical rule is to match the variant to both the statistical target and the model assumptions:

  • Use pairs cluster bootstrap for minimal modeling assumptions and when the goal is uncertainty around the observed-data statistic.
  • Consider residual bootstrap when a well-specified fitted model is available and residual dependence can be resampled in a way consistent with the error structure.
  • Use block or weighted variants when dependence extends beyond a single cluster dimension or when cluster sizes/influence vary strongly.

When theoretical guidance is limited, comparing results across a small set of plausible variants and checking stability can provide reassurance.