1 Introduction to Dependent-Data Resampling

1.1 Why standard bootstrap fails under dependence

The classical bootstrap assumes that observations are independent and identically distributed. When data exhibit serial or spatial dependence, drawing individual observations at random breaks the dependence pattern. As a result, the resampled data can have an unrealistic joint structure, and uncertainty estimates such as standard errors or confidence intervals may be biased or have incorrect coverage.

1.2 Core idea of resampling contiguous blocks

Block bootstrap methods address this mismatch by resampling groups of neighboring observations together. For a time series, the “neighbors” are consecutive points in time. Resampling contiguous blocks keeps local dependence intact within each block, while allowing the overall sample to vary across bootstrap replicates. The method thus trades some realism in global dependence for improved fidelity of short-range dependence.

1.3 Typical statistical goals (standard errors, confidence intervals, distribution estimates)

Block bootstrap is used to approximate the sampling distribution of an estimator in dependent-data settings. Typical targets include:

  • Estimating standard errors for smooth or non-smooth estimators.
  • Constructing confidence intervals using percentile, basic, or studentized approaches.
  • Approximating distribution functionals such as quantiles or tail probabilities.
  • Calibrating test statistics for hypothesis testing, particularly when analytical formulas are difficult.

2 Basic Block Bootstrap Framework

2.1 Defining the data and the estimator

2.1.1 Observations as a dependent sequence

Consider a dependent sequence \((X_1,\dots,X_n)\), where dependence may arise from temporal correlation or spatial proximity. The dependence may be weak or strong; practical designs typically aim to preserve enough local structure so that bootstrap replicates mimic the estimator’s dominant sources of variability.

2.1.2 Target functional and asymptotic perspective

Let \(\hat\theta = T(X_1,\dots,X_n)\) be an estimator of a parameter or functional \(\theta\). Block bootstrap aims to approximate the distribution of \(\hat\theta\) around its sampling behavior. Under suitable assumptions, one expects that the bootstrap distribution of \(\hat\theta^\ast\) approximates the law of \(\hat\theta\) (often in an asymptotic sense as \(n\to\infty\)).

2.2 Choosing block partitions

2.2.1 Contiguous blocks and indexing

A common setup selects a block length \(l\) and forms overlapping or non-overlapping contiguous blocks from the observed sequence. For overlapping blocks, a block starting at position \(i\) contains \((X_i,\dots,X_{i+l-1})\). Indexing choices determine how many candidate blocks are available and which dependence patterns are represented.

2.2.2 Overlapping vs. non-overlapping blocks

  • Overlapping blocks reuse information: adjacent blocks share observations, providing many candidate blocks for resampling and typically reducing Monte Carlo variability.
  • Non-overlapping blocks partition the data into disjoint segments: fewer candidates can increase variability but may better reflect dependence at the block scale.

The overlap decision influences both computational cost and how closely the resampling scheme matches the estimator’s dependence structure.

2.3 Resampling mechanism

2.3.1 Drawing blocks to form bootstrap samples

To create a bootstrap replicate, blocks are randomly selected and concatenated to build a synthetic series of length comparable to the original. The resampling distribution is induced by the block selection rule, not by individual observation draws.

Many implementations sample blocks independently with replacement from the candidate set. This generates variability across replicates while maintaining the internal dependence within each chosen block.

2.3.2 Handling sample length and end effects

Concatenating blocks can overshoot the target length \(n\). Standard practice uses trimming (discard extra observations) or padding (extend by a rule such as wrap-around in circular schemes). End effects can matter because dependence near the boundaries may differ from dependence in the middle. Methods that mitigate boundary artifacts often improve finite-sample performance.

2.4 Constructing bootstrap replicates

2.4.1 Recomputing the estimator on each resample

For each bootstrap replicate \(b=1,\dots,B\), the estimator is recomputed: \[ \hat\theta^{\ast(b)} = T(X_1^{\ast(b)},\dots,X_n^{\ast(b)}), \] where the superscript \(\ast(b)\) denotes the resampled sequence assembled from blocks. Recomputing from scratch ensures that the bootstrap properly propagates dependence effects through the estimator.

2.4.2 Aggregating results across replicates

The collection \(\{\hat\theta^{\ast(1)},\dots,\hat\theta^{\ast(B)}\}\) is used to estimate sampling uncertainty. One can compute:

  • Empirical standard deviations for standard errors.
  • Empirical quantiles for percentile intervals.
  • Distribution-based quantities for distribution approximation or test calibration.

3 Common Variants of Block Bootstrap

3.1 Moving block bootstrap (MBB)

3.1.1 Block length selection in MBB

In MBB, one typically uses overlapping blocks of a fixed length \(l\). Candidate blocks slide along the series, and bootstrap replicates are formed by sampling these blocks to obtain a full-length synthetic sequence. The key tuning parameter is \(l\), which controls how much dependence is preserved.

3.1.2 Overlapping blocks interpretation

Overlapping blocks increase the number of candidate segments and hence the diversity of bootstrap replicates. This flexibility can improve approximation quality when local dependence matters, though the computational burden may rise due to the larger block pool.

3.2 Non-overlapping block bootstrap (NBB)

3.2.1 Block partitioning schemes

NBB divides the data into consecutive disjoint blocks and resamples those blocks. Depending on the implementation, the starting point can be chosen in multiple ways (e.g., shifting the partition) to reduce sensitivity to the initial alignment.

3.2.2 Trade-offs with dependence preservation

Because NBB uses disjoint segments, dependence within each segment is preserved, but the dependence across segment boundaries is not represented. The method can have higher variability when the effective number of blocks is small, yet it may be simpler and sometimes competitive for estimators driven mainly by within-block behavior.

3.3 Stationary bootstrap

3.3.1 Random block lengths

Stationary bootstrap randomizes block lengths rather than using a fixed \(l\). This can mimic dependence structures that decay gradually by producing variable-sized segments across replicates.

3.3.2 Benefits for weak stationarity

For data that are approximately weakly stationary, randomization of block lengths can improve alignment between the bootstrap’s dependence decay and the observed dependence. The scheme aims to avoid systematic artifacts tied to a single chosen block length.

3.4 Circular block bootstrap

3.4.1 Wrapping blocks to reduce boundary effects

Circular block bootstrap treats the data as if arranged on a circle. Blocks that would extend beyond the end wrap around to the beginning. This reduces boundary discontinuities and can yield more homogeneous dependence across the resampled sequence.

3.4.2 Use cases and practical considerations

Circular methods are particularly useful when dependence is expected to behave similarly near the boundaries as in the interior. In practice, whether wrapping is appropriate depends on whether the series endpoints are meaningfully connected (e.g., periodic behavior) or merely consecutive segments.

4 Block Length and Tuning Parameters

4.1 The role of block size

Block length \(l\) is central: it determines the scale on which dependence is preserved. Small \(l\) may fail to capture relevant correlation, while large \(l\) preserves more dependence but reduces the number of effectively independent blocks, increasing estimation noise in the bootstrap distribution.

4.2 Bias–variance trade-off

4.2.1 Small blocks: bias from broken dependence

If \(l\) is too short relative to the dependence horizon, resamples disrupt dependence beyond the block boundaries. This can bias the estimated sampling distribution, leading to inaccurate standard errors and confidence intervals.

4.2.2 Large blocks: higher variance due to fewer effective blocks

Large \(l\) reduces the number of blocks available in each resample, which inflates Monte Carlo error and can make bootstrap intervals less stable. Even with many bootstrap replications, the induced bootstrap variability can be higher when the scheme has fewer degrees of freedom.

4.3 Data-driven block length selection

4.3.1 Heuristics and rule-of-thumb methods

A common heuristic relates \(l\) to a fractional power of sample size (e.g., scaling like \(n^{1/3}\) or \(n^{1/2}\) depending on asymptotic regimes). While simple, these rules may not match the observed dependence strength, so practitioners often treat them as starting points.

4.3.2 Plug-in and optimization-based approaches

More systematic methods estimate an optimal block length by minimizing an error criterion that balances approximation bias and variance. Plug-in strategies use estimates of quantities governing dependence, while optimization-based approaches search across candidate \(l\) values using bootstrap-based criteria or stability measures.

4.4 Sensitivity analysis

4.4.1 Repeating with multiple block sizes

Because block length selection is rarely exact, practitioners often repeat the analysis over a range of plausible \(l\) values and compare the resulting uncertainty estimates. Consistent conclusions across nearby choices suggest robustness.

4.4.2 Reporting robustness of conclusions

An encyclopedia-style summary often emphasizes that final reporting should include some indication of sensitivity, such as whether confidence intervals or p-values remain stable under moderate changes in tuning parameters.

5 Theoretical Properties (High-Level)

5.1 Consistency under dependence assumptions

Theoretical results aim to show that, under appropriate assumptions about the dependence structure and the estimator, the bootstrap distribution converges to the true sampling distribution. “Consistency” typically means that empirical quantiles, moments, or distribution functions derived from the bootstrap approximate their target counterparts as \(n\) grows.

5.2 Weak convergence and distribution approximation

Convergence statements are commonly expressed in terms of weak convergence of random processes or functionals. At a high level, one shows that the bootstrap replicates mimic the limiting behavior of properly normalized estimators in the presence of dependence.

5.3 Convergence rates and their dependence on block length

Rates of approximation often depend on how \(l\) grows with \(n\). Faster growth can reduce bias but worsen variance, so theoretical regimes identify ranges of \(l\) that optimize overall error.

5.4 Conditions on the data-generating process

Assumptions vary across frameworks, but they often include:

  • A notion of dependence decay (e.g., mixing-type conditions or summability of autocovariances).
  • Regularity conditions on the estimator (e.g., smoothness, moment bounds).
  • Requirements linking block length \(l\) to sample size \(n\).

The interplay between dependence strength and tuning determines the credibility of approximation.

5.5 Comparison to alternative resampling methods (overview)

Other dependent-data approaches include subsampling (using fewer observations in each replicate) and wild bootstrap variants tailored for specific dependence forms. Block bootstrap is often preferred when preserving local dependence through contiguous resamples is essential and when block length can be reasonably tuned.

6 Inference with Block Bootstrap

6.1 Percentile and basic bootstrap intervals

  • Percentile intervals use empirical quantiles of \(\hat\theta^\ast\) directly.
  • Basic intervals reflect around the original estimate, using quantiles of \(2\hat\theta-\hat\theta^\ast\).

These methods avoid direct variance estimation formulas but can be sensitive to bias in \(\hat\theta\) or to imperfect bootstrap approximation.

6.2 Studentized/standard-error based intervals

6.2.1 Bootstrap variance estimation

Studentized methods standardize the estimator using a bootstrap estimate of its variability. The idea is that dividing by an estimated standard deviation can improve interval calibration, particularly when the scale of fluctuations is not constant across samples.

6.2.2 t-type statistics under dependence

A common construction is a “t-type” statistic: \[ T^\ast = \frac{\hat\theta^\ast - \hat\theta}{\hat\sigma^\ast}, \] where \(\hat\sigma^\ast\) is computed from bootstrap replicates or from a bootstrap-specific variance estimator. The distribution of \(T^\ast\) is then used to determine critical values.

6.3 Bias-corrected approaches (overview)

6.3.1 When bias correction is helpful

If the estimator has non-negligible finite-sample bias, percentile intervals may shift systematically. Bias-corrected procedures attempt to adjust for that shift. While they can improve accuracy, they rely on reasonable bootstrap approximation quality and on stable estimates of bias-related quantities.

6.4 Hypothesis testing using bootstrap

6.4.1 Bootstrap calibration of test statistics

Bootstrap tests often proceed by defining a statistic \(S\) measuring discrepancy under the null hypothesis. Bootstrap replicates are generated under a calibration strategy, and the empirical distribution of \(S^\ast\) is used to compute p-values or reject when \(S\) is extreme relative to \(S^\ast\). In dependent data, the validity of such tests depends strongly on whether the bootstrap scheme respects the dependence pattern under the null.

7 Practical Implementation Details

7.1 Algorithmic steps and pseudocode structure

A typical implementation includes:

  1. Choose block length \(l\) and block scheme (e.g., MBB, NBB, stationary, circular).
  2. Form candidate blocks from the observed sequence.
  3. For each bootstrap replication \(b\):
  • Sample blocks according to the scheme and concatenate to length \(n\) (with trimming or wrapping as required).
  • Compute \(\hat\theta^{\ast(b)}\).
  1. Use the empirical distribution of \(\hat\theta^{\ast(b)}\) to construct intervals or test calibration values.

7.2 Computational considerations

7.2.1 Number of bootstrap replications

The number of replications \(B\) controls Monte Carlo error. Too small \(B\) yields unstable quantiles and inaccurate standard errors, while very large \(B\) can be computationally expensive, especially for complex estimators.

A practical rule is to choose \(B\) large enough that results do not change materially with further increases, assessed through pilot runs.

7.2.2 Efficient handling of overlapping blocks

Overlapping-block methods can be implemented efficiently by precomputing block indices and avoiding repeated data slicing overhead. Memory management and vectorized operations can substantially reduce runtime, particularly when the estimator is expensive to compute.

Block bootstrap assumes a dependence structure that is reasonably stable at the block scale. If the series has strong trends or structural breaks, dependence patterns may change across time. Guidance includes:

  • Detrending or differencing when appropriate for the estimator.
  • Using shorter blocks when change points are localized.
  • Considering extensions tailored for nonstationary settings (often problem-specific).

7.4 Diagnostics for dependence preservation

Diagnostics are used to assess whether resampled sequences preserve key aspects of dependence, such as autocorrelation patterns or empirical dependence measures. A practical approach is to compare summary dependence statistics computed on original data versus typical bootstrap replicates.

7.5 Common pitfalls and failure modes

Frequent issues include:

  • Choosing \(l\) too small so that dependence is not captured.
  • Choosing \(l\) too large so that too few blocks exist, inflating variability.
  • Inappropriate handling of boundaries, especially in non-circular schemes.
  • Applying bootstrap blindly to strongly nonstationary series without preprocessing or tailored methods.

8 Special Settings and Extensions

8.1 Multivariate time series block bootstrap (overview)

For vector-valued observations, blocks may be defined jointly across components so that cross-series dependence within blocks is preserved. The key design question is whether to resample based on a single joint block (preserving contemporaneous and lagged dependence) or to treat components separately (usually less faithful).

8.2 Block bootstrap for statistics beyond means (overview)

8.2.1 Quantiles and distribution functionals

Quantile estimation can be sensitive to dependence and to sample irregularities. Block bootstrap can approximate the sampling distribution of quantiles by resampling blocks and recomputing the quantile functional on each replicate.

8.2.2 Regression-based estimators (overview)

For estimators derived from regression models, bootstrap replicates require recomputing model fits on the resampled data. Care must be taken that the resampling scheme aligns with how dependence enters the regression (e.g., time-correlated errors versus covariate-driven dependence).

8.3 Spatial block bootstrap (overview)

For spatial data, blocks are contiguous regions in space rather than intervals in time. The shape and size of regions affect dependence preservation, and boundary handling can be more complex due to irregular domains and anisotropic dependence.

8.3.1 Contiguous regions as blocks

Spatial blocks can be constructed using moving windows, partitioned grids, or adaptive shapes. The guiding principle parallels time-series block bootstrap: preserve local spatial dependence within each region while resampling regions to reflect uncertainty.

8.4 Bootstrap for dependent residuals (conceptual overview)

In regression contexts, some approaches aim to resample residuals while maintaining dependence in the error structure. Conceptually, the bootstrap may be applied to residuals using block or moving-block designs, then recombined with fitted values. The method’s validity depends on how well the residual bootstrap approximates the true dependence in errors.

9.1 Wild bootstrap vs. block bootstrap (overview)

Wild bootstrap is designed primarily for heteroskedasticity and depends on constructing new error terms with certain properties. Block bootstrap is aimed at serial or spatial dependence by resampling contiguous structures. In some problems, both issues are present, and hybrid strategies may be considered.

9.2 Subsampling vs. bootstrap (overview)

Subsampling uses smaller portions of the dataset in each replicate without attempting to match the original estimator’s distribution through rescaling in the same way as bootstrap. It can be useful when bootstrap consistency is uncertain or when block bootstrap calibration is difficult. The choice between subsampling and bootstrap depends on theoretical assumptions and practical stability.

9.3 Moving-block techniques vs. other resampling schemes

Moving-block techniques include MBB and related variants that preserve local dependence via overlapping segments. Alternative schemes may resample at different granularity (e.g., subsampling) or use different dependence models. Block bootstrap methods are generally distinguished by their explicit contiguous-block construction.

10 Examples and Worked Workflows

10.1 Step-by-step example for a time series estimator

10.1.1 Choose block length and resampling scheme

A workflow often begins by:

  1. Plotting the series and examining autocorrelation to gauge dependence range.
  2. Selecting a block scheme (e.g., MBB) that matches the data context.
  3. Choosing initial block length \(l\) via a heuristic or data-driven rule.
  4. Running sensitivity checks over nearby \(l\) values.

10.2 Compute confidence intervals

Once bootstrap replicates are computed:

  1. Generate \(B\) resamples and compute \(\hat\theta^{\ast(b)}\).
  2. Select an interval method (percentile or studentized).
  3. Extract the appropriate empirical quantiles and report interval endpoints.
  4. Document \(l\), scheme type, and \(B\).

10.3 Interpreting results and uncertainty reporting

Interpretation focuses on whether uncertainty bands are plausibly sized given the dependence in the data. Reporting typically includes:

  • Block bootstrap variant used (MBB, NBB, stationary, circular).
  • Block length choice or selection method.
  • Number of replications \(B\).
  • Interval type (percentile, basic, studentized).
  • Any sensitivity analysis summary.

10.4 Reproducibility and parameter documentation

Reproducible analysis requires recording random seeds (if relevant), tuning parameters, and the exact block construction rules, including overlap, trimming, and boundary handling. Transparent documentation is especially important because different implementation details can materially affect outcomes.

11 Summary

11.1 When to use block bootstrap

Block bootstrap is appropriate when observations are dependent and the goal is to quantify uncertainty for an estimator whose sampling distribution is analytically complex. It is commonly used for time series and spatial datasets where preserving local dependence is crucial.

11.2 Key design choices checklist

Key choices include:

  • Dependence-preserving block scheme (moving, non-overlapping, stationary, circular).
  • Block length selection (fixed or data-driven, and its sensitivity).
  • Overlapping versus disjoint block candidates.
  • Boundary treatment (trimming versus wrapping).
  • Interval or test construction method (percentile, studentized, calibration strategy).

11.3 Quick reference for variant selection

  • MBB: often a default choice when dependence is mostly short-range and preserving local structure is important.
  • NBB: useful for simpler partition-based resampling, with attention to fewer effective blocks.
  • Stationary bootstrap: advantageous when dependence decay suggests variable block lengths.
  • Circular bootstrap: suitable when endpoint behavior should match interior behavior (e.g., periodic or wrap-around contexts).