1 Likelihood ratio test (LRT) fundamentals

1.1 Hypothesis setup and model nesting

The likelihood ratio test compares two statistical models: a null model \(H_0\) and an alternative model \(H_1\). In its most common form, the models are *nested*, meaning the null model is a constrained special case of the alternative model. Nesting creates a natural comparison because the alternative contains at least all degrees of freedom available under the null plus additional parameters.

When models are not nested, the LRT can still be defined by comparing the maximized likelihoods under each model, but classical large-sample reference results may require extra conditions. The nested case is therefore the standard starting point in likelihood-ratio theory and practice.

1.2 Likelihood functions and maximum likelihood estimates

Let \(y\) denote observed data and let \(\theta\) represent model parameters. The likelihood function \(L(\theta \mid y)\) is the probability (or probability density) of the data as a function of \(\theta\). The LRT uses the *maximum likelihood estimate* (MLE) within each hypothesis:

  • Under \(H_0\): maximize the likelihood over the parameter space constrained by \(H_0\), yielding \(\hat{\theta}_0\).
  • Under \(H_1\): maximize over the full alternative space, yielding \(\hat{\theta}_1\).

The maximization steps embody the idea that each model should be allowed to fit the data as well as it can, within its own restrictions.

1.3 Test statistic definition (ratio vs. log-ratio)

The defining object is the likelihood ratio: \[ \Lambda = \frac{\sup_{\theta\in H_0} L(\theta\mid y)}{\sup_{\theta\in H_1} L(\theta\mid y)}. \] Because the alternative model typically includes the null, the denominator is at least as large as the numerator, so \(0<\Lambda\le 1\).

Many presentations use the equivalent log form for numerical and theoretical convenience. A common statistic is \[ D = 2\left[\log L(\hat{\theta}_1\mid y)-\log L(\hat{\theta}_0\mid y)\right] = -2\log \Lambda. \] Large values of \(D\) indicate that allowing the extra parameters under \(H_1\) substantially improves fit, providing evidence against \(H_0\).

1.4 Interpreting the strength of evidence

In hypothesis testing, the LRT statistic is converted into a tail probability under \(H_0\). If the computed p-value is small, the data are unlikely to have arisen if \(H_0\) were true, so the analyst rejects \(H_0\) at the chosen significance level.

Beyond binary rejection, the magnitude of \(D\) can be interpreted as quantifying how much additional likelihood the alternative achieves. This is closely related to changes in model fit measured on a likelihood scale, and it often corresponds to a clinically or substantively meaningful improvement when the additional parameters reflect realistic structure.

2 Asymptotic theory and distributions

2.1 Wilks’ theorem and chi-squared limits

A central result in LRT theory is *Wilks’ theorem*, which states that under suitable regularity conditions and for large samples, the statistic \[ D = 2\left[\ell(\hat{\theta}_1)-\ell(\hat{\theta}_0)\right] \] converges in distribution under \(H_0\) to a chi-squared random variable: \[ D \ \xrightarrow[]{d}\ \chi^2_{\nu}, \] where \(\ell(\theta)\) denotes log-likelihood and \(\nu\) is the difference in dimension between the alternative and null parameter spaces.

This asymptotic reference distribution enables the use of standard chi-squared tail probabilities to compute p-values without simulating from the null.

2.2 Degrees of freedom and parameter counting

In nested parametric models, \(\nu\) typically equals the number of constraints imposed by \(H_0\). Equivalently, it is the difference in the number of free parameters estimated under \(H_1\) versus \(H_0\). For example, if the null fixes \(k\) parameters that are free under the alternative, then \(\nu=k\).

Correct degrees-of-freedom accounting is essential: using the wrong \(\nu\) can distort p-values and lead to over- or under-rejection.

2.3 Regularity conditions (what they ensure)

Wilks’ theorem relies on conditions that prevent pathological behavior in likelihoods and estimators. While formulations vary, they generally include:

  • The true parameter under \(H_0\) lies in the interior of the parameter space.
  • The model is identifiable (distinct parameters correspond to distinct data distributions).
  • Sufficient smoothness of the likelihood with respect to parameters.
  • Finite, well-behaved information and consistent MLEs.
  • Appropriate scaling and large-sample stability.

Intuitively, these conditions ensure that log-likelihood differences behave quadratically near the maximum and that asymptotic expansions are valid.

2.4 Boundary and nonstandard cases

2.4.1 Mixtures of chi-squared distributions

When parameters lie on the boundary of the null parameter space, the limiting distribution of the LRT statistic may not be a single chi-squared law. A common phenomenon is a *mixture of chi-squared distributions*, reflecting that different local behaviors can occur depending on which boundary face the likelihood concentrates on.

Such mixtures arise, for example, when testing whether a variance component is exactly zero, or when testing constrained parameters that can change sign or only take nonnegative values.

2.4.2 When Wilks’ theorem may fail

Wilks’ theorem may fail or require modification in several situations, including:

  • The MLE under the null is not an interior point.
  • Regularity assumptions break due to non-identifiability or singularities.
  • Models are only partially nested in a way that disrupts local asymptotic geometry.
  • The sample size is small relative to the complexity of the models.

In these cases, relying on the naive chi-squared reference distribution can miscalibrate inference, motivating alternative calibrations.

3 Practical computation

3.1 Constructing the null and alternative models

Practical use begins with specifying the null and alternative model structures. For nested comparisons, the null is defined by imposing constraints that simplify the alternative. For instance, the null might set regression coefficients to zero, remove a component, or constrain a parameter function to a simpler form.

The analyst then verifies that both models are fitted to the same dataset under comparable assumptions, and that the likelihoods used are compatible (e.g., same response definition, same conditioning, and consistent handling of missingness).

3.2 Using optimization output to get maximized likelihoods

LRT requires the maximized likelihood (or log-likelihood) under each hypothesis. In modern workflows, maximization typically relies on numerical optimization and returns parameter estimates and log-likelihood values at convergence.

Because optimization can be sensitive to starting values and tolerances, it is good practice to confirm that the null and alternative fits achieve stable maxima. If the optimizer terminates prematurely or finds a local optimum, the computed likelihood ratio may be unreliable.

3.3 Numerical stability for log-likelihood differences

Implementations commonly compute log-likelihoods and form differences to avoid underflow that could occur if likelihoods themselves are extremely small. Even on the log scale, subtracting nearly equal large negative numbers can introduce rounding errors when the models fit similarly.

A standard approach is to use the log-likelihood values returned by the optimizer directly in a well-scaled formula such as \[ D = 2(\ell_1-\ell_0), \] and to use robust numerical types and tolerances consistent with the statistical software’s internal handling.

3.4 Software implementations and defaults

3.4.1 Common workflows in statistical software

Many packages provide built-in likelihood ratio tests for model objects, automatically extracting the relevant log-likelihoods and degrees of freedom. Common workflows include:

  • Fit both models using maximum likelihood (not only penalized objectives) and request an LRT.
  • Use functions that return the LRT statistic and p-value, sometimes with an option to specify reference distribution or simulate critical values.
  • In generalized linear model contexts, use model-based functions that compute deviance differences corresponding to likelihood ratios.

Users typically must still verify model nesting, confirm that the fitting method is appropriate for LRT (e.g., true likelihood rather than quasi-likelihood), and ensure that the reported degrees of freedom reflect the parameterization.

4 LRT in common model classes

4.1 Linear models and residual sum of squares connections

4.1 Nested linear model comparisons

For classical normal linear models with constant variance, the likelihood ratio test becomes closely linked to comparisons of residual sum of squares. When nested models differ by adding or removing predictors, the improvement in log-likelihood corresponds to the reduction in residual variance after fitting.

In this setting, the LRT often agrees with familiar F-tests under normality and equal-variance assumptions. This equivalence is a practical reason LRT is widely understood through simpler regression diagnostics, even when the likelihood-ratio viewpoint is not always emphasized.

4.2 Generalized linear models (GLMs)

In GLMs, the likelihood is specified through a distributional family and a link function, and the LRT is defined using the maximized log-likelihoods under competing models. Many software systems also express likelihood-based discrepancy using *deviance*, which can be interpreted as a scaled measure tied to log-likelihood differences.

Under standard assumptions, the LRT statistic computed from deviance differences has an asymptotic chi-squared reference distribution, making it a convenient tool for testing subsets of parameters or comparing nested GLMs.

4.2.1 Deviance as a likelihood-based measure

Deviance is constructed so that larger deviance indicates poorer fit relative to a saturated model. In nested comparisons, the difference in deviance between the alternative and null is proportional to twice the log-likelihood difference, aligning deviance-based tests with the likelihood ratio framework.

This relationship helps unify GLM output: the same underlying likelihood information can be reported either as a likelihood ratio or as deviance change.

4.3 Nonlinear and semiparametric contexts

For nonlinear models, the LRT definition remains the same—compare maximized likelihoods under constrained and unconstrained forms. However, asymptotic behavior may be more sensitive to identifiability and optimization challenges than in linear settings. Nonlinear models may also exhibit multiple local maxima, which complicates the reliability of the computed statistic.

In semiparametric models, likelihood construction may involve nuisance components that are estimated nonparametrically. LRT can still be used in principle, but the asymptotic distribution may deviate from standard chi-squared forms unless additional structure is available.

4.4 Random-effects and mixed models (high-level view)

Mixed models include random effects and typically require specialized likelihood approaches (such as integration over random-effect distributions). The LRT compares a reduced model against a more flexible one, for instance testing whether a variance component is zero.

In mixed-model settings, boundary issues are common because testing a variance component against zero often places parameters on a boundary. Consequently, the null distribution may be nonstandard, and some software either uses approximations or suggests simulation-based calibration when default chi-squared references are questionable.

5 Relation to other tests and criteria

5.1 Connection to score tests and Wald tests

Beyond the LRT, hypothesis testing for nested models often uses:

  • *Wald tests*, which rely on asymptotic normality of parameter estimates and test constraints through estimated standard errors.
  • *Score tests* (Lagrange multiplier tests), which evaluate the gradient of the log-likelihood under the null without fitting the full alternative.

In large samples, these tests can be asymptotically equivalent under regularity conditions. The LRT is distinct in that it uses differences in fit across models; Wald and score tests use local information at the null. Practical differences can emerge in finite samples or under near-boundary constraints.

5.2 Power and sample-size considerations

Test power depends on effect size, model complexity, and sample size. The LRT is designed to detect discrepancies that meaningfully improve likelihood under the alternative. In practice, power often improves as sample size grows, but finite-sample calibration matters when asymptotic reference distributions are imperfect.

When the alternative introduces parameters that are weakly identifiable or only become informative in certain regimes, LRT power can be uneven across alternatives, even at comparable sample sizes.

5.3 Likelihood ratio vs. information criteria (AIC/BIC)

Information criteria are model selection tools rather than formal hypothesis tests, but they share the same likelihood foundation. Common choices are:

  • AIC, which penalizes model complexity using an estimate of expected predictive discrepancy.
  • BIC, which penalizes using a stronger complexity term tied to asymptotic model probability under certain assumptions.

While the LRT compares fit between nested models for a hypothesis decision, AIC/BIC compare models using overall scores and typically apply across non-nested candidate sets. In nested settings, changes in log-likelihood that underlie LRT results also drive differences in AIC/BIC, but the penalty strengths and intended interpretations differ.

5.4 LRT versus cross-validation approaches

Cross-validation assesses predictive performance by repeatedly training on subsets of data and evaluating on held-out portions. Compared with LRT, it is less focused on statistical hypothesis calibration and more focused on out-of-sample adequacy.

In many applied settings, LRT provides interpretable inference about specific parameter constraints, whereas cross-validation supports broader generalization claims. Using both can be beneficial: LRT for targeted structural questions and cross-validation for predictive validation.

6 Assumptions, diagnostics, and robustness

6.1 Data requirements and independence considerations

LRT theory assumes the data generating mechanism matches the model’s likelihood structure, and that observations are sufficiently independent (or that dependence is correctly represented in the model). Violations such as unmodeled correlation can inflate false rejection rates by making effective information larger than the model assumes.

Diagnostics often focus on residual patterns, influence, and checks of model-specific assumptions (e.g., distributional fit in GLMs). While no diagnostic guarantees correct asymptotic behavior, they help detect when the reference distribution is likely to be misaligned with the data.

6.2 Handling overdispersion and model misspecification

Overdispersion occurs when variability exceeds what the assumed likelihood family captures, such as extra dispersion beyond binomial or Poisson assumptions. Misspecification can make likelihood-based statistics behave unpredictably, altering calibration and interpretability.

Robust strategies include adjusting the model (e.g., using alternative families, adding dispersion parameters, or employing quasi-likelihood approaches). However, quasi-likelihood typically changes the meaning of the likelihood ratio, so care is needed: LRT p-values generally require a bona fide likelihood framework consistent with the test statistic.

6.3 Overparameterization and identifiability issues

When models contain redundant parameters or fail identifiability conditions, the likelihood surface can become flat or irregular, undermining MLE stability. Overparameterization can produce large variance in estimates and make the LRT statistic sensitive to optimization artifacts.

Practical steps include checking for convergence warnings, verifying that parameters correspond to identifiable features of the data, and simplifying the model when necessary. Identifiability issues are especially common in mixture-like constructions and complex nonlinear models.

6.4 Small-sample corrections and alternatives

6.4.1 Parametric bootstrap for LRT calibration

When asymptotic approximations are unreliable, a common remedy is *parametric bootstrap*. The procedure uses the null model to simulate many synthetic datasets, refits both models on each simulated dataset, computes the LRT statistic each time, and estimates the null distribution empirically.

The resulting empirical tail probability can be more accurate than a chi-squared approximation, especially in small samples, boundary problems, or nonstandard settings where Wilks’ theorem does not apply cleanly.

7 Design and reporting

7.1 Choosing the null vs. alternative specification

Good LRT design clarifies what constraints the null imposes and what additional flexibility the alternative allows. The null should reflect the scientific or modeling claim under assessment, while the alternative should be the smallest sensible model that captures the suspected deviation.

Selecting models that are logically comparable and truly nested (when relying on standard asymptotics) improves interpretability and calibration. When nesting is approximate or unclear, reporting the modeling relationship explicitly helps readers assess validity.

7.2 Significance level, p-values, and confidence interpretations

Reporting should include the chosen significance level (such as \(\alpha=0.05\)), the computed test statistic, and the p-value or critical value used for decision-making. In likelihood-ratio contexts, p-values are derived from the assumed reference distribution or from simulation calibration.

Interpreting results should also note that a hypothesis test addresses evidence relative to the null, not the probability that the null is true. Confidence intervals can sometimes be derived from likelihood ratios via likelihood-based methods, but such connections depend on model regularity and construction choices.

7.3 Effect size reporting alongside hypothesis testing

Because a statistically significant result can correspond to a modest practical difference, effect size should accompany the LRT. Common effect-size summaries include estimated parameter changes, differences in predicted quantities, or measures derived from fitted model components.

In nested model comparisons, reporting the fitted values under both models and summarizing how predictions or outcomes shift is often more informative than relying solely on p-values.

7.4 Reproducible reporting checklist

Reproducible LRT reporting typically includes:

  • The explicit formulas or model specifications for \(H_0\) and \(H_1\).
  • The method used for fitting (maximum likelihood, restricted likelihood, or other), including optimizer details if relevant.
  • The LRT statistic definition used (e.g., \(2(\ell_1-\ell_0)\)) and the reference distribution assumption or calibration approach.
  • Sample size, number of parameters (degrees of freedom), and any convergence diagnostics.
  • The reported statistic and p-value, along with effect-size summaries.
  • Software and version information, plus key options (e.g., handling of missing data, offsets, and constraints).

These elements allow other analysts to reproduce both the fitting steps and the hypothesis test calculation.