1. Motivation and Core Idea

1.1 Why “sequential” decision-making is useful

In many problems, data arrive over time or can be generated in an incremental way. A sequential rejection rule leverages this structure by updating a decision as each new batch of evidence becomes available. Instead of committing to a single fixed-size experiment, the procedure can terminate early when evidence is decisive, thereby reducing wasted sampling.

This approach is also useful when the cost of collecting additional observations is nontrivial, or when timely conclusions are required. By allowing earlier rejection when evidence against a claim becomes sufficiently strong, sequential decision-making can improve practical efficiency while still offering formal guarantees about error rates.

1.2 Comparison with one-shot (fixed-sample) rules

A fixed-sample (one-shot) rule evaluates the claim using data from a predetermined sample size. Regardless of whether the evidence is overwhelming or negligible partway through data collection, the analyst must wait until the full dataset is observed.

Sequential rejection rules, in contrast, permit stopping at multiple intermediate stages. This means the final decision depends not only on the observed evidence but also on when the procedure stops. As a consequence, two datasets of equal final size may lead to different decisions if the sequential rule would have already stopped earlier for one of them.

1.3 Evidence accumulation and stopping logic

The defining feature is stagewise evidence accumulation followed by a stopping decision. At stage \(n\), the rule uses information up to that point to compute a criterion that reflects support for or against the claim. Based on the criterion’s value, the procedure may:

  • Reject the claim immediately if the evidence exceeds a rejection boundary.
  • Continue to the next stage if the evidence is inconclusive.
  • Accept (or fail to reject) if the accumulated information supports the claim sufficiently.

This produces a decision path governed by boundary crossings and by the rule’s “continue” region.

2. Formal Definition

2.1 Rejection rule as a mapping from data to decisions

Formally, a sequential rejection rule can be viewed as a mapping from the evolving data to a set of decisions. Let \(X_1, X_2, \dots\) denote sequentially collected observations, and let \(X_{1:n}\) denote the data available through stage \(n\). The rule assigns a decision at each stage based on \(X_{1:n}\), typically through a test statistic or a likelihood-based quantity.

The key point is that the decision at stage \(n\) uses only information from stages \(1\) through \(n\), ensuring the procedure is consistent with the incremental nature of the data.

2.2 Stopping times and stagewise decision criteria

The stopping stage is typically defined as a stopping time: a random index determined by the data observed so far. The procedure continues until either a rejection condition is met, an acceptance condition is met, or a pre-specified maximum horizon is reached.

Stagewise decision criteria are commonly expressed as comparisons to boundaries, such as:

  • reject when a statistic falls below (or rises above) a threshold,
  • accept when it crosses a different threshold,
  • otherwise continue.

With finite horizons, the procedure is forced to end by the maximum stage even if neither boundary is reached earlier.

2.3 Output types: reject, continue, or accept

Sequential rejection rules can be one-sided (only designed to reject) or two-sided (allowing both rejection and acceptance). In some implementations, “accept” may be formalized as “do not reject” once sufficient support is accumulated in favor of the claim, rather than proving the claim outright.

At each stage, the rule’s output is therefore one of three actions:

  1. Reject,
  2. Continue (collect more data),
  3. Accept (or terminate in favor of the claim).

The accept/reject labels depend on the convention used for the hypotheses and on the intended interpretation of “acceptance.”

2.4 Notation and assumptions

A standard setup uses two hypotheses, a claim \(H\) (often corresponding to the null or an alternative) and a set of alternatives. The data generating process is assumed to be governed by probability distributions indexed by the true state (e.g., \(P_\theta\)).

Sequential designs require regularity conditions so that stagewise statistics behave appropriately and so that the stopping time is measurable with respect to the observed data sigma-algebra. Assumptions may include independence, identical distribution within stages, or known variance structures, depending on the application.

3. Error Control Guarantees

3.1 Family-wise vs stagewise error perspectives

Error control can be framed in different ways in sequential settings. One perspective emphasizes the overall probability of making a particular error at any time point—analogous to family-wise control in multiple testing. Another perspective focuses on error rates at each stage, though these are harder to interpret because the rule adapts by deciding when to stop.

Because a sequential rule effectively performs repeated checks until a boundary is hit, error control must account for the dependence between stages induced by adaptive stopping.

3.2 Controlling type I error in sequential settings

Type I error corresponds to rejecting when the claim is true (under the null-like scenario). Sequential rejection rules are typically constructed to ensure that the probability of eventual rejection under the true claim does not exceed a target level \(\alpha\).

This guarantee is not automatic; it depends on how boundaries are chosen and on the statistical properties of the accumulating evidence. Techniques often aim to calibrate the rejection boundary so that the running criterion crosses it with probability bounded by \(\alpha\).

3.3 Controlling type II error (and power) in practice

Type II error corresponds to failing to reject when the claim is false. In sequential frameworks, controlling type II error exactly can be more intricate than type I control, because the probability of stopping early in the “continue” region interacts with the evidence distribution under alternatives.

Rather than strict control, many designs focus on ensuring adequate power, meaning a sufficiently high probability of eventual rejection under parameter values representing practically relevant departures from the claim. In practice, power is often evaluated through operating characteristics computed under candidate alternatives.

3.4 Trade-offs between strictness and sampling cost

Sequential rules that enforce tighter error bounds typically require wider decision boundaries or more conservative criteria, which can increase the expected number of stages before reaching a decision. Conversely, designs that attempt to reach conclusions quickly may increase the chance of stopping at less definitive evidence, potentially reducing power or inflating error.

Therefore, the designer balances:

  • error-rate guarantees, and
  • efficiency, often measured by expected sample size or expected time to termination.

4. Design of Sequential Rejection Rules

4.1 Choosing boundaries for rejection

A central design task is selecting rejection (and acceptance) boundaries. Boundaries define when the statistic indicates enough evidence to stop. Popular boundary designs include linear boundaries in certain transformed statistics and boundaries derived from likelihood ratios.

The boundary choice also determines the geometry of the continuation region—regions where the rule keeps sampling rather than stopping.

4.2 Calibration using null distributions

Calibration ensures the sequential rule meets the desired error control level under the claim being tested. This can be done analytically in special cases or numerically when exact calculations are intractable.

Calibration often relies on knowledge of the data distribution under the null, including its moments or full likelihood structure. When the null distribution is complex, simulation-based calibration is commonly used to estimate the probability of crossing the rejection boundary.

4.3 Strategies for adaptive sample size

Sequential rejection rules are inherently adaptive: the stopping time depends on the observed trajectory of the statistic. Designers may also incorporate structured adaptation, such as varying the number of observations per stage or using information-weighted stage sizes.

In some frameworks, the rule is allowed to continue up to a maximum horizon but is designed so that expected stage counts shrink when evidence is strong and increase when the data are near ambiguous.

4.4 Robustness considerations under model misspecification

Real applications rarely match the assumed statistical model perfectly. Robustness concerns arise when boundary calibration depends on distributional assumptions such as normality, independence, or correctly specified variance.

To address this, designers may use robust test statistics, choose boundaries based on conservative approximations, or evaluate performance empirically under perturbed assumptions. Robustness is rarely absolute; instead, procedures are assessed for stability of error control and power under plausible deviations.

5. Relationship to Sequential Testing Frameworks

5.1 Connection to sequential probability ratio testing concepts

Sequential probability ratio testing (SPRT) is a classical framework in which decisions are based on likelihood ratios accumulated over time. While not every sequential rejection rule is literally an SPRT, many share the same conceptual core: compare an evidence measure to boundaries, stop when the evidence strongly favors rejection or acceptance, otherwise continue.

This connection is useful for understanding the role of likelihood-based evidence and for guiding intuition about why sequential rules can be efficient.

5.2 Likelihood-based vs test-statistic-based formulations

Sequential procedures can be constructed in different representations:

  • Likelihood-based formulations update likelihood ratios or related quantities as new data arrive.
  • Test-statistic-based formulations update a chosen statistic (e.g., standardized mean differences) and compare it to stagewise thresholds.

Likelihood-based approaches often provide elegant theoretical properties under correct model assumptions. Test-statistic approaches can be easier to implement and may align better with practical reporting conventions, though their optimality depends strongly on the chosen statistic.

5.3 When asymptotic approximations apply

In many practical settings, exact finite-sample error control may be hard. Sequential designs may therefore rely on asymptotic results, approximating boundary crossing probabilities using limiting distributions of scaled statistics.

Asymptotic approximations can work well when sample sizes are large or when the underlying statistic satisfies central-limit-type conditions. However, approximation error can accumulate in sequential contexts, so validation through simulation remains important.

6. Implementation and Computational Aspects

6.1 Practical stopping rules in simulation studies

When implementing a sequential rule in simulation, one typically:

  1. Generate or resample observations stage by stage,
  2. Compute the running evidence criterion at each stage,
  3. Apply the boundary logic to decide whether to stop,
  4. Record the decision and the stopping stage.

Simulation studies then estimate error rates, average sample numbers, and power by repeating this process across many simulated trajectories.

6.2 Numerical stability and discretization issues

Sequential procedures can be sensitive to numerical implementation details, especially when using likelihood ratios that may underflow or overflow. Computations are often stabilized by using log-likelihood ratios and by rescaling intermediate quantities.

Discretization also matters: since the rule checks boundaries at discrete stages, the realized crossing behavior is tied to the stage grid. Finer stage grids can produce decisions that better reflect a continuous-time ideal but at greater computational cost.

6.3 Monitoring and logging decisions over stages

Because the final decision depends on the entire stopping path, logging is essential in both research and operational contexts. Key records include:

  • stage index at termination,
  • decision type (reject/accept),
  • the statistic or evidence values at stopping,
  • any intermediate boundary comparisons.

Such logs support auditing, debugging, and post hoc analysis of how often decisions occur at early versus late stages.

7. Performance Evaluation

7.1 Average sample number and efficiency metrics

A core efficiency metric is the average sample number (or expected sample size), measuring how many observations the procedure uses before reaching a decision. Lower values generally indicate faster conclusions, but they must be evaluated alongside error and power performance.

Other efficiency measures can include expected runtime or expected number of stages, especially when stages correspond to computationally expensive batches.

7.2 Operating characteristic curves

Operating characteristic (OC) curves summarize the probability of rejection as a function of the underlying parameter value. In sequential settings, these curves reflect the interplay between boundary geometry and the evolving evidence distribution.

OC curves are useful for comparing designs, since two procedures can share the same nominal error control yet exhibit different rejection probabilities across plausible alternatives.

7.3 Sensitivity analysis for different effect sizes

Sequential rules are often tailored to detect certain departures from the claim. Sensitivity analysis evaluates how performance changes when the effect size is smaller or larger than intended.

For example, designers may examine:

  • power near the boundary of detectability,
  • the expected stopping time when effects are moderate,
  • the behavior under effect sizes that are very large, where early stopping is more likely.

7.4 Comparing alternative sequential rejection designs

Comparisons typically involve multiple metrics rather than a single criterion. A design that has slightly more favorable error control might require substantially more sampling, or might achieve power primarily at extreme alternatives while performing weakly near moderate alternatives.

Comparative evaluation therefore often uses a matrix of results: error rates, expected sample sizes, and power across a grid of parameter values.

8. Variants and Extensions

8.1 Multi-stage rejection rules with finite horizons

Many sequential rules operate with a finite maximum horizon, meaning the procedure must stop by some stage \(N\). This simplifies implementation and ensures termination but can reduce the theoretical efficiency benefits of fully unbounded designs.

Multi-stage rejection rules are still “sequential” even when the number of stages is predetermined, because the stopping time remains data-dependent up to that horizon.

8.2 Two-sided vs one-sided rejection criteria

One-sided rules are designed to reject in only one direction of deviation from the claim, while two-sided rules handle departures in either direction. Two-sided designs usually require more complex boundary structure to balance error control across both tails.

Acceptance regions also differ: in two-sided settings, acceptance may be defined when the evidence remains sufficiently close to the claim, whereas in one-sided settings acceptance may simply mean evidence never becomes strong enough to trigger rejection before stopping.

In more advanced applications, the evidence statistic can be adjusted using covariates or incorporate systematic time trends. This can involve regression-type updating, stratified accumulation, or changes in information weighting across stages.

The core sequential logic remains intact: compute an evidence measure using all data up to the current stage, then apply stagewise criteria. What changes is the form of the evidence measure and the assumptions behind it.

8.4 Group sequential vs fully sequential variants

Group sequential methods restrict the set of possible stopping times to predetermined analysis points, often corresponding to interim looks at the data. Fully sequential methods allow stopping at every possible observation time.

Both are sequential, but fully sequential designs can be more efficient in theory because they avoid the granularity of planned interim points. In practice, group sequential rules are often easier to schedule and monitor, particularly in regulated or operational environments.

9. Example Walkthroughs

9.1 Simple staged testing with a toy statistic

Consider a toy scenario where observations are collected in batches of size \(m\). After each batch, a standardized statistic \(T_n\) is computed from the cumulative data through stage \(n\). The sequential rejection rule compares \(T_n\) to a rejection threshold \(c\).

For illustration, suppose the rule rejects when \(T_n \ge c\), accepts (or stops without rejection) when \(T_n \le d\), and otherwise continues. The numbers \(c\) and \(d\) are selected to control error behavior under the claim.

9.2 Illustrative boundary crossings and stopping behavior

In a simulated trajectory, early stages might produce \(T_1\) near the center of the continuation region, causing the procedure to continue. Later, if the observed data shift \(T_n\) upward, the statistic may cross the rejection boundary at some stage \(n^\*\), triggering rejection.

Another trajectory might oscillate near the continuation region without crossing until a later stage, possibly stopping due to a maximum horizon. This illustrates how sequential outcomes depend on the entire path of evidence, not just the final stage statistic.

9.3 Interpreting outcomes across stages

When rejection occurs at an early stage, it indicates that the evidence accumulated rapidly satisfied the boundary criterion. A late rejection suggests that evidence was weaker initially and only became decisive after additional data were gathered.

If the procedure accepts or terminates without rejection, it implies that the evidence never crossed the rejection boundary within the allowed stages (or crossed an acceptance boundary first). Interpreting these outcomes benefits from examining both the stopping stage and the statistic value at termination.