1 Acquisition functions in sequential model-based optimization

1.1 Purpose and role in the loop

In sequential model-based optimization, an acquisition function provides a rule for selecting the next candidate input to evaluate. At each iteration, a surrogate model predicts the unknown objective at candidate points. The acquisition function then converts those predictions—typically including both central estimates and uncertainty—into a single scalar score. Candidates with higher scores are prioritized for evaluation, allowing the method to focus evaluations where they are expected to improve the objective or where additional information is likely to be valuable.

1.2 Relationship to Bayesian optimization

Bayesian optimization is a prominent framework for sequential optimization with expensive black-box evaluations. In this setting, the surrogate model is updated after each observation, producing a posterior predictive distribution. The acquisition function is the mechanism that turns this posterior into an actionable selection criterion. While Bayesian optimization defines a probabilistic modeling approach, the acquisition function specifies the specific decision strategy within that probabilistic framework.

1.3 Surrogate models and predictive distributions

Surrogate models approximate the objective function using past evaluations. Because many Bayesian optimization methods are probabilistic, the surrogate yields a predictive distribution rather than only point estimates. Commonly, this distribution is summarized by a predictive mean and variance. The acquisition function uses these quantities to quantify both likely performance and epistemic uncertainty, enabling principled trade-offs between trying promising candidates and gathering information about less certain regions.

1.4 Candidate selection and optimization of the acquisition

The acquisition function must be maximized (or minimized, depending on convention) over the candidate space to decide the next query. This is itself an optimization problem, often easier than optimizing the original objective because the surrogate and acquisition are analytic or cheaply computable. Practical algorithms therefore include inner-loop solvers—gradient-based methods when differentiable, or derivative-free strategies when the acquisition is not smooth or when the search space is discrete and mixed.

2 Mathematical foundations

2.1 Objective and notation

Let \(f(x)\) denote the unknown objective over an input domain \(\mathcal{X}\). Sequential optimization maintains a dataset \(D_n=\{(x_i,y_i)\}_{i=1}^n\), where \(y_i=f(x_i)\) are observed values. A surrogate model provides a predictive distribution for \(f(x)\) given \(D_n\). Acquisition functions are then derived from the posterior predictive behavior.

2.1.1 Surrogate predictive mean and variance

For probabilistic surrogates such as Gaussian processes, the posterior predictive distribution at a candidate \(x\) can be characterized by a mean \(\mu_n(x)\) and a variance \(\sigma_n^2(x)\). These summaries are sufficient for many standard acquisition functions because they approximate key probabilistic events (e.g., surpassing a threshold) using Gaussian assumptions.

2.1.2 Uncertainty quantification (e.g., posterior over functions)

Uncertainty quantification reflects both limited data and model ambiguity. In Bayesian models, uncertainty is represented through the posterior distribution over functions or, equivalently, through the predictive variance induced by the posterior. Acquisition rules use this uncertainty to assess how much the objective is expected to vary at a point if one were to evaluate it.

2.2 Expected utility formulation

A general view is that an acquisition function represents an expected utility of evaluating a candidate \(x\), computed under the surrogate’s predictive distribution. The utility is often designed to reward improvements over the current best observed value, or to favor information gain. Many popular acquisition functions can be derived as special cases of expected utility with specific choices of utility and approximations.

2.2.1 Maximization vs. minimization conventions

Most formulas are written for either maximization or minimization. If the problem is posed as minimization, a common approach is to transform it into a maximization by considering \(-f(x)\), or to adapt improvement definitions accordingly. Careful attention to sign conventions is necessary because the same symbolic form can correspond to opposite behaviors under different conventions.

2.3 Conditioning on observations

At iteration \(n\), the surrogate’s posterior—and therefore \(\mu_n(x)\) and \(\sigma_n^2(x)\)—is computed conditional on the observed dataset \(D_n\). This conditioning ensures that the acquisition function reflects current knowledge, updating the selection bias as new evaluations arrive.

2.4 Constraints and feasibility-aware scoring

2.4.1 Modeling constrained objectives

When feasibility constraints are present, selection must account for both objective value and the probability that a candidate satisfies constraints. A common strategy introduces an additional probabilistic model for feasibility (or multiple constraint models). The acquisition score is then modified to downweight infeasible candidates or to combine objective improvement with the likelihood of satisfying constraints, producing a feasibility-aware utility.

3 Core acquisition function families

3.1 Improvement-based acquisition functions

Improvement-based criteria aim to preferentially sample points expected to yield an improvement over the current best observation. They typically compare the posterior predictive distribution at \(x\) with a reference level, such as the best known objective value so far.

3.1.1 Expected Improvement (EI)

Expected Improvement computes the expected amount by which the objective at \(x\) exceeds the current best value, averaged over the predictive distribution. EI has two key effects: it is larger when the predicted mean is high and it increases when uncertainty is high enough to make improvement plausible. For Gaussian predictive models, EI admits closed-form expressions that make it computationally attractive.

3.1.2 Probability of Improvement (PI)

Probability of Improvement focuses on the chance that the objective at \(x\) will exceed (or beat) the best value by some threshold. Unlike EI, which accounts for the magnitude of improvement, PI is primarily concerned with whether improvement occurs. This can make PI simpler but sometimes less robust in practice, especially when improvement magnitudes vary significantly.

3.1.3 Upper Confidence Bound (UCB)

Upper Confidence Bound constructs a score using a trade-off between the posterior mean and uncertainty, often in the form \(\mu_n(x)+\kappa\,\sigma_n(x)\) for maximization. The exploration weight \(\kappa\) controls how aggressively the method samples uncertain regions. UCB is widely used because it yields an intuitive criterion and offers theoretical guarantees under appropriate assumptions, even though the exact choice of \(\kappa\) strongly affects behavior.

3.2 Uncertainty-driven acquisition functions

3.2.1 Pure exploration criteria

Some acquisition functions prioritize uncertainty reduction by directly targeting high predictive variance without explicitly optimizing for expected improvement. These rules can be useful when the primary goal is to learn the objective landscape or when little is known initially. Pure exploration, however, may be inefficient if the objective optimum is concentrated in regions that could be exploited earlier.

3.2.2 Information gain and entropy-based approaches

Information gain methods define acquisition in terms of how much evaluating \(x\) is expected to reduce uncertainty about the function or about some summary quantity. Entropy-based approaches compute expected reductions in entropy under the posterior predictive distribution. Although these methods can be powerful, they may require more expensive approximations because expected entropy reduction often lacks simple closed forms.

3.3.1 One-step lookahead intuition

Knowledge-gradient methods are grounded in the expected improvement in the chosen decision after one additional observation. Informally, they look ahead by asking how much the best predicted objective value would be expected to increase once a candidate \(x\) is evaluated. This “one-step” perspective tends to produce acquisition scores that are sensitive to the impact of measurements on future decisions.

3.3.2 Multi-step extensions

Multi-step variants extend lookahead to consider sequences of future evaluations, balancing longer-term benefits against immediate gains. While such approaches can outperform single-step criteria, their computational cost grows substantially because they require planning over branching futures and updating beliefs repeatedly.

4 Hyperparameters and trade-offs

4.1 Exploitation–exploration balance

A defining feature of many acquisition functions is the controlled tension between exploitation (sampling points likely to be good) and exploration (sampling points with high uncertainty). Hyperparameters regulate this balance. When tuned well, the acquisition directs early exploration to learn the landscape and later exploitation to refine around likely optima.

4.2 Tuning parameters (e.g., exploration weight)

In UCB-type rules, an exploration weight parameter determines the relative contribution of uncertainty. In EI/PI variants, additional parameters such as jitter or improvement thresholds may control sensitivity to marginal gains. These tuning parameters influence convergence speed and final performance, and they often interact with surrogate model assumptions.

4.3 Effects of surrogate calibration

Acquisition performance depends on the quality of uncertainty estimates. If the surrogate is miscalibrated—predictive variances that are too small or too large—the acquisition may become overconfident or overly exploratory. Calibration issues can arise from limited data, misspecified kernels, or insufficient modeling of noise, all of which distort the posterior predictive distribution used by the acquisition function.

4.4 Batch/parallel acquisition and diversity

4.4.1 Penalization or local penalization strategies

When multiple evaluations can be executed in parallel, a batch acquisition strategy must select a set of candidates rather than a single point. Common approaches use penalization: after selecting one candidate, the acquisition landscape is modified to discourage selecting nearby points that are expected to yield redundant information. This encourages diversity and can improve batch efficiency.

Thompson sampling draws candidate points by sampling from the surrogate posterior and optimizing under that sampled function. Although not always classified strictly as an acquisition function family, Thompson sampling can be interpreted as a stochastic acquisition mechanism. Batch versions often combine posterior sampling with diversity or with sequential selection under updated temporary beliefs.

5 Practical implementation considerations

5.1 Acquisition maximization strategies

5.1.1 Gradient-based optimization when differentiable

Many acquisition functions are differentiable with respect to \(x\) when the surrogate provides differentiable predictive quantities. Gradient-based inner solvers (e.g., quasi-Newton or stochastic gradient methods) can efficiently find maxima of the acquisition in continuous spaces. They typically require multiple restarts to avoid poor local optima because the acquisition surface can be multimodal.

5.1.2 Derivative-free search for black-box candidates

If the input domain is discrete, mixed, or the acquisition is not differentiable, derivative-free optimization is used. Techniques include evolutionary strategies, random search with local refinement, Bayesian optimization applied to the acquisition itself, and candidate library approaches where the acquisition is evaluated over a precomputed set.

5.2 Input scaling, bounds, and normalization

Surrogate models and acquisition optimization often assume inputs are scaled to comparable ranges. Normalization improves numerical stability for kernel methods and supports consistent behavior across dimensions. Bounds define the feasible domain for candidate search, and careful handling is required to prevent the inner optimizer from proposing invalid points.

5.3 Handling categorical and mixed variables

5.3.1 Embedding approaches (general concept)

For categorical variables, one common strategy is to embed categories into continuous vectors and treat them as inputs to the surrogate. Another approach uses specialized kernels or probabilistic models designed for mixed data types. In practice, embedding must be chosen to preserve meaningful similarity patterns among categories and to avoid introducing artificial ordering effects.

5.4 Computational cost and approximation

5.4.1 Monte Carlo estimation when closed forms are unavailable

Some acquisition functions or surrogate models yield no closed-form expression for the acquisition score. Monte Carlo estimators can approximate expected utilities, information gains, or multi-step lookahead criteria. The accuracy-cost trade-off depends on the number of samples used in the estimator and on how sensitive the acquisition is to variance in the Monte Carlo estimate.

6 Evaluation and benchmarking

6.1 Regret and sample-efficiency metrics

Performance is often measured by regret relative to the optimum or by the objective value attained after a fixed budget of evaluations. Sample efficiency metrics track how quickly the method improves as observations accumulate. For noisy objectives, comparisons may include repeated runs and distributional summaries.

6.2 Common experimental setups

Benchmarking typically includes synthetic test functions with known optima, as well as curated black-box tasks. Experiments vary in dimensionality, noise level, and the structure of constraints. Fair comparisons keep the surrogate family and optimization budget consistent across acquisition strategies to isolate the contribution of the acquisition function.

6.3 Robustness across objectives

An acquisition function may perform well on smooth functions but degrade on irregular landscapes, changing noise regimes, or discontinuities. Robustness testing evaluates stability across objective classes, including those with non-stationary behavior or mixed smoothness. Results are often summarized as averages over multiple random seeds and multiple problem instances.

6.4 Failure modes and diagnostics

6.4.1 Overconfident surrogates

If the surrogate underestimates uncertainty, improvement-based criteria may repeatedly focus on a small region and miss better alternatives. Diagnostic signs include acquisition surfaces that become sharply peaked early and lack of exploration despite limited data. Remedies can include adjusting noise modeling, kernel choice, or adding mechanisms that temper uncertainty estimates.

6.4.2 Acquisition “hacking” due to poor candidate optimization

When the inner optimization of the acquisition is flawed, the algorithm may select points that are artifacts of numerical issues or local maxima in the acquisition optimizer rather than true global maxima. This can lead to wasted evaluations. Practical diagnostics include verifying candidate quality using multiple restarts, checking acquisition values at candidate points, and monitoring whether improvements correlate with high acquisition scores.

7 Variants and extensions

7.1 Multi-objective acquisition functions

7.1.1 Pareto-front improvement ideas (high level)

In multi-objective optimization, instead of a single scalar objective, there is a vector of objectives. Acquisition functions then aim to guide sampling toward the Pareto front. High-level approaches define utility in terms of expected improvements to nondominated sets, such as increasing the expected coverage of the Pareto frontier or improving dominance relations.

7.2 Transfer learning and warm-start acquisition

When a related problem has prior data, acquisition functions can be adapted to exploit this information. Warm-start strategies initialize the surrogate with transferred observations or adjust priors so that the acquisition reflects knowledge already available. This can reduce the number of expensive evaluations needed on the new task.

7.3 Online updating and non-stationary objectives

For objectives that change over time, standard stationary surrogates may become outdated. Online updating strategies incorporate new data with recency weighting or time-aware kernels. The acquisition function is then recomputed under the updated posterior, enabling adaptation to drifting optima.

7.4 Risk-sensitive or robust acquisition

7.4.1 Incorporating safety margins conceptually

Risk-sensitive acquisition introduces preferences against unfavorable outcomes, especially in noisy settings. Conceptually, this can be done by optimizing conservative utility functions that account for lower confidence bounds or penalties for tail risk. Safety margins can be integrated so that candidate selection avoids regions likely to produce unacceptable performance, even if they appear attractive under mean-based criteria.

8 Summary and selection guide

8.1 When to use EI, PI, UCB, or uncertainty criteria

  • EI is often a strong default when the goal is to efficiently improve the objective value while leveraging both mean and uncertainty.
  • PI can be useful when the main question is whether improvement occurs, though it may be less informative about improvement magnitude.
  • UCB provides a direct exploration–exploitation lever and is effective when choosing an exploration weight is practical.
  • Uncertainty-driven criteria are preferred when exploration is paramount, or when a reliable objective model is not yet established.

8.2 Choosing surrogate models alongside acquisition

Acquisition function behavior is coupled to the surrogate’s predictive distribution. Gaussian process surrogates are common for their uncertainty estimates and analytic predictive mean/variance, enabling closed-form acquisition functions. Alternative surrogates can be used, but they may require approximate acquisition computation if predictive uncertainty is not available in a convenient form.

A typical workflow is to select a standard acquisition family (often EI or UCB), normalize inputs, tune key hyperparameters using a small validation budget, and run multiple seeds to estimate variability. Validation should include checks for calibration and for the quality of the inner maximization of the acquisition. If performance is unstable, adjustments often involve changing surrogate assumptions, improving candidate optimization for the acquisition, or switching to a more robust acquisition strategy.