1 Fundamental concepts

A likelihood-ratio test is built on the idea of comparing how well two statistical models explain the same data. The simpler model, called the null model, is treated as a restricted case of a more flexible alternative model. If the added flexibility produces a much higher likelihood, the data may be taken as evidence against the null model. This framework is especially useful because it applies across many kinds of parametric models.

1.1 Likelihood function

The likelihood function measures the relative plausibility of parameter values given observed data. It is not a probability distribution over parameters; rather, it describes how compatible each parameter value is with the data under a specified model. In a likelihood-ratio test, each model is summarized by its maximum likelihood value, obtained by choosing the parameter values that best fit the sample.

1.2 Hypothesis testing

In hypothesis testing, one evaluates whether the data provide enough evidence to reject a stated assumption. The likelihood-ratio test fits naturally into this setting because it directly compares a constrained hypothesis with a less constrained one. The test asks whether relaxing the restriction leads to an improvement in fit large enough to be unlikely under the null hypothesis.

1.3 Nested models

Nested models are models in which the null model can be obtained from the alternative model by imposing constraints on one or more parameters. For example, a regression model without a predictor is nested within the same model with that predictor included. The nesting requirement is central to the classical likelihood-ratio test because the difference in fit can then be attributed to the additional parameters.

1.4 Null and alternative hypotheses

The null hypothesis states the restricted model, often representing no effect, no difference, or a simplified structure. The alternative hypothesis allows the broader model and includes the null as a special case. A likelihood-ratio test evaluates whether the evidence favors the alternative strongly enough to reject the null in favor of the more general specification.

2 Test statistic

The test statistic in a likelihood-ratio test is based on the ratio of the maximum likelihoods under the two competing models. Because likelihood values often become very small, the ratio is usually transformed into a more convenient logarithmic form. This produces a statistic that is easy to compute and has a well-known large-sample reference distribution.

2.1 Likelihood ratio

The basic likelihood ratio compares the maximized likelihood under the null model with the maximized likelihood under the alternative model. Since the alternative model is more flexible, its maximum likelihood is at least as large as that of the null model. A ratio close to 1 indicates little improvement, while a ratio far from 1 suggests that the alternative fits much better.

2.2 Log-likelihood form

A common version of the test statistic is based on the difference between the maximized log-likelihoods. Multiplying this difference by 2 gives a quantity that is often easier to interpret and whose asymptotic distribution is typically chi-square under the null hypothesis. This log-likelihood form is standard in many software implementations and theoretical derivations.

In several model classes, especially generalized linear models, the likelihood-ratio statistic is closely related to deviance. Deviance measures the discrepancy between a fitted model and a saturated or nearly saturated reference model. Differences in deviance between nested models provide a convenient way to carry out likelihood-ratio testing.

2.4 Interpretation of small and large values

Small values of the likelihood-ratio statistic indicate that the simpler model fits almost as well as the more complex one. Large values suggest that the added parameters materially improve the fit. The statistic is interpreted in relation to its reference distribution, which determines whether the observed improvement is unusual enough to count as statistically significant.

3 Theoretical properties

The likelihood-ratio test has strong asymptotic justification under broad conditions. Its appeal comes partly from the fact that it often requires few model-specific derivations. Theoretical results show that, in large samples, the statistic behaves in a predictable way and can be used for approximate inference.

3.1 Wilks’s theorem

Wilks’s theorem is the central result underlying the classical likelihood-ratio test. It states that, under suitable conditions and when the null hypothesis is true, twice the log-likelihood ratio converges in distribution to a chi-square random variable. This result provides the theoretical basis for calculating p-values without extensive simulation.

3.2 Asymptotic chi-square distribution

The chi-square approximation is an asymptotic result, meaning it becomes more accurate as sample size increases. Under the null hypothesis, the test statistic is compared with a chi-square distribution whose shape depends on the number of constrained parameters. This approximation is widely used because it is simple and effective in many standard settings.

3.3 Degrees of freedom

The degrees of freedom for the test usually equal the difference in the number of free parameters between the alternative and null models. Intuitively, each additional unrestricted parameter contributes one degree of freedom. In more complex settings, such as constrained or composite hypotheses, the effective degrees of freedom may require careful determination.

3.4 Regularity conditions

The asymptotic theory relies on regularity conditions such as differentiability of the likelihood, identifiable parameters, and interior parameter values. These assumptions ensure that the maximum likelihood estimators behave well and that the chi-square approximation is valid. When such conditions fail, the standard test may need modification or may no longer have the expected distribution.

4 Construction of the test

Constructing a likelihood-ratio test involves defining the two models, estimating their parameters, and comparing their maximized likelihoods. The procedure is conceptually straightforward and can be applied to a wide variety of statistical models. Its implementation depends on having a likelihood function for each model under consideration.

4.1 Specifying the null model

The null model is defined first, usually by stating the restrictions that represent the simpler hypothesis. These restrictions may fix a parameter at zero, impose equality among parameters, or remove a predictor from the model. The null model should be interpretable and statistically nested within the alternative.

4.2 Specifying the alternative model

The alternative model relaxes the restrictions imposed by the null model. It includes all parameters or structures allowed under the null and adds extra freedom. The alternative should be formulated so that it captures the broader mechanism or effect that the test is intended to detect.

4.3 Estimating parameters by maximum likelihood

For each model, the parameter values that maximize the likelihood are estimated from the data. These maximum likelihood estimates summarize the best fit attainable under the respective model assumptions. In practice, estimation may be carried out numerically, especially when the likelihood has no closed-form maximizer.

4.4 Computing the ratio

Once the maximum likelihoods are obtained, the likelihood ratio is calculated from their quotient or from the corresponding log-likelihood difference. The resulting statistic is then compared with a reference distribution, usually chi-square under standard conditions. This computation is the core step that converts model fit into a formal test.

5 Decision procedure

The decision procedure converts the test statistic into a conclusion about the null hypothesis. As with other hypothesis tests, the result depends on a chosen significance level and on how unusual the observed statistic is under the null model. The final decision is typically framed in terms of rejecting or not rejecting the null.

5.1 Choosing a significance level

The significance level sets the tolerated probability of a false rejection when the null hypothesis is true. Common choices include 0.05 and 0.01, though the appropriate value depends on the context and the consequences of error. The significance level acts as the threshold for the test’s decision rule.

5.2 P-values

A p-value is the probability, under the null hypothesis, of observing a test statistic at least as extreme as the one obtained. In likelihood-ratio testing, it is computed from the chi-square approximation or another suitable reference distribution. Smaller p-values indicate stronger evidence against the null model.

5.3 Critical values

A critical value is the cutoff from the reference distribution beyond which the test statistic is considered too large to be compatible with the null. If the observed statistic exceeds this threshold, the null hypothesis is rejected. Critical values provide an equivalent formulation to p-values for making decisions.

5.4 Rejecting or failing to reject the null hypothesis

If the evidence is strong enough, the null hypothesis is rejected in favor of the alternative. If the statistic does not cross the chosen threshold, the null is not rejected. Failure to reject does not prove the null is true; it only indicates that the data do not provide sufficient evidence against it.

6 Common applications

Likelihood-ratio tests are widely used because they fit naturally into model-based statistical analysis. They are especially useful when comparing nested regression structures or testing whether particular parameters improve model fit. The method appears across many branches of applied statistics and data analysis.

6.1 Testing a single parameter

A common use is testing whether one parameter differs from a specified value, often zero. In regression, this may involve examining whether a predictor has any effect after accounting for other variables. The null model fixes the parameter, while the alternative allows it to vary freely.

6.2 Comparing regression models

Likelihood-ratio tests are frequently used to compare regression models that differ by one or more predictors. The question is whether the added terms produce a meaningful improvement in fit. Because the models are nested, the test provides a principled way to assess whether the extra complexity is justified.

6.3 Generalized linear models

In generalized linear models, likelihood-ratio tests are a standard tool for comparing fitted models. They are often used to evaluate the contribution of predictors, interactions, or link-function choices. The deviance framework makes these tests especially convenient in this setting.

6.4 Survival analysis

In survival analysis, likelihood-ratio tests help compare hazard models and assess the significance of covariates. They are useful when the model is estimated through partial or full likelihood methods. The test can indicate whether including additional explanatory variables substantially improves the description of event times.

6.5 Mixed-effects models

Mixed-effects models often require testing fixed effects, random effects, or structural simplifications. Likelihood-ratio tests are commonly used to compare versions of a model with different random-effect structures or different sets of fixed predictors. Because these models can be complex, careful attention to approximation and boundary issues is important.

Several tests are closely related to the likelihood-ratio test because they address similar inferential questions. These methods differ in how they use the fitted model and what information they emphasize. In large samples, they often lead to similar conclusions, though not always identical ones.

7.1 Wald test

The Wald test assesses whether estimated parameters are far from the null value relative to their estimated standard errors. It uses the fitted alternative model alone, rather than comparing two fitted models directly. Although convenient, it may behave differently from a likelihood-ratio test in smaller samples or near boundaries.

7.2 Score test

The score test, also called the Lagrange multiplier test, evaluates whether the slope of the likelihood at the null model suggests improvement in another direction. It requires fitting only the null model, which can make it computationally attractive. Its conclusions often resemble those of the likelihood-ratio test under standard conditions.

7.3 Likelihood-based model selection

Likelihood-based model selection uses likelihood values, sometimes along with penalties for complexity, to compare competing models. Criteria such as AIC and related measures focus on balancing goodness of fit against parsimony. While not hypothesis tests in the strict sense, they are closely connected to likelihood-ratio reasoning.

7.4 Analysis of deviance

Analysis of deviance extends the likelihood-ratio framework to sequences of nested models. It reports how much deviance is reduced when terms are added step by step. This approach is widely used in generalized linear modeling to examine the incremental contribution of predictors.

8 Practical considerations

Although the likelihood-ratio test is broadly applicable, real-world data can create complications. Finite-sample behavior may differ from asymptotic theory, and some model structures violate standard assumptions. Careful use of the test requires attention to these limitations.

8.1 Small-sample performance

In small samples, the chi-square approximation may be inaccurate, leading to distorted p-values. The test can be conservative or liberal depending on the model and the data-generating process. Simulation or exact methods may be preferable when sample size is limited.

8.2 Boundary problems

Problems arise when parameter values under the null lie on the boundary of the allowable parameter space. In such cases, the usual chi-square reference distribution may not hold. Special theoretical results or adjusted procedures are then needed to obtain valid inference.

8.3 Non-nested model comparisons

The classical likelihood-ratio test is designed for nested models, not arbitrary comparisons. When models are non-nested, the simple ratio-based framework does not yield the usual chi-square result. Alternative methods, including information criteria or specialized non-nested tests, may be more appropriate.

8.4 Computational issues

Some likelihoods are difficult to optimize numerically, especially in high-dimensional or highly nonlinear models. Convergence failures, local maxima, and unstable estimates can affect the test result. Reliable implementation often depends on good starting values, stable algorithms, and diagnostic checks.

9 Extensions

The likelihood-ratio framework has been adapted to many specialized settings. These extensions preserve the basic idea of comparing fit under constrained and unconstrained models, while modifying the details to suit the model class. As a result, the method remains influential across modern statistical practice.

9.1 Multinomial and contingency-table settings

In multinomial models and contingency-table analysis, likelihood-ratio tests are used to assess association, independence, and goodness of fit. The resulting statistics are often expressed as deviations from expected cell counts under the null model. This makes the method a natural companion to categorical data analysis.

9.2 Penalized likelihood approaches

When estimation includes a penalty term, the comparison is based on penalized likelihood rather than ordinary likelihood alone. Such approaches arise in regularized regression and high-dimensional modeling. The interpretation of a likelihood-ratio-style test may change because the penalty alters the effective fit.

9.3 Likelihood-ratio confidence intervals

Likelihood-ratio ideas also support confidence intervals by finding parameter values that do not reduce the likelihood too much relative to the maximum. This yields intervals based on the same logic as hypothesis testing. Such intervals are often asymmetric and can reflect the geometry of the likelihood surface more faithfully than simple symmetric approximations.

9.4 Generalized likelihood ratio tests

Generalized likelihood ratio tests extend the basic method to broader settings, including composite hypotheses and more complex sampling structures. The general principle remains the same: compare the best fit under the null with the best fit under the alternative. In many applications, this generalized form is the default likelihood-based testing approach.

10 Examples

Examples clarify how the likelihood-ratio test operates in concrete situations. The same logic applies across a wide range of models, even though the formulas differ. In each case, the core task is to measure whether the additional freedom of the alternative model produces a meaningful gain in likelihood.

10.1 Simple binomial example

Suppose a binomial model is used to test whether the probability of success equals a specified value. The null model fixes the probability, while the alternative estimates it from the data. The likelihoods under both models are computed, and their ratio is used to assess whether the observed number of successes is inconsistent with the fixed probability.

10.2 Regression coefficient test

In a regression setting, one may test whether a coefficient is zero. The null model excludes the predictor, whereas the alternative includes it. If the fitted model with the predictor achieves a substantially higher likelihood, the test may indicate that the predictor contributes useful explanatory information.

10.3 Nested model comparison in practice

A practical nested comparison might involve fitting a baseline model and then adding an interaction term or additional covariate. The likelihood-ratio statistic summarizes how much the fit improves after the change. If the improvement is large relative to the appropriate chi-square reference distribution, the more complex model is favored.