1 Definition and Basic Concept

1.1 Formal Definition

Let \( L(\theta \mid x) \) denote the likelihood function for a parameter \(\theta\) given observed data \(x\). For two competing hypotheses, often represented by parameter values \(\theta_0\) (null hypothesis) and \(\theta_1\) (alternative hypothesis), the likelihood ratio is defined as

\[ \Lambda(x) = \frac{L(\theta_1 \mid x)}{L(\theta_0 \mid x)}. \]

A value of \(\Lambda(x) > 1\) indicates that the data are more probable under the alternative hypothesis than under the null, while a value less than 1 suggests the opposite. In practice, the null hypothesis is often placed in the denominator.

1.2 Relationship to Likelihood Function

The likelihood function itself is not a probability density, but the likelihood ratio eliminates normalizing constants that cancel in the ratio. This invariance makes the likelihood ratio a direct measure of relative evidence, independent of any prior distribution. It is central to the likelihood principle, which states that all evidence from data lies in the likelihood ratio.

1.3 Interpretation in Hypothesis Testing

In hypothesis testing, the likelihood ratio quantifies the strength of evidence against the null hypothesis. A large ratio (or its logarithm, often twice the log ratio) suggests the null is less plausible. Decision rules based on likelihood ratios can be constructed to control Type I error rates, as formalized in the Neyman–Pearson lemma.

2 Likelihood-Ratio Test

2.1 Test Statistic

The likelihood-ratio test statistic measures the relative fit of two nested models. It is typically defined as

\[ \lambda(x) = \frac{\sup_{\theta \in \Theta_0} L(\theta \mid x)}{\sup_{\theta \in \Theta} L(\theta \mid x)}, \]

where \(\Theta_0\) is the parameter space under the null hypothesis and \(\Theta\) is the full space under the alternative. Equivalently, one often uses the deviance \(D = -2 \log \lambda(x)\).

2.1.1 Asymptotic Distribution

Under certain regularity conditions, the test statistic \(-2 \log \lambda\) asymptotically follows a chi-squared distribution. This property allows the construction of approximate p-values without exact finite-sample distributions.

2.1.1.1 Wilks’ Theorem

Wilks’ theorem states that, for nested models, \(-2 \log \lambda\) converges in distribution to a chi-squared random variable with degrees of freedom equal to the difference in the number of free parameters between the two models. This theorem holds under the null hypothesis and standard regularity conditions (e.g., identifiability, compact parameter spaces, and sufficient smoothness of the likelihood).

2.2 Properties

2.2.1 Neyman–Pearson Lemma

The Neyman–Pearson lemma establishes that, for simple hypotheses (both fully specified), the likelihood-ratio test is the most powerful test at a given significance level. That is, it maximizes the probability of correctly rejecting a false null hypothesis for any fixed Type I error rate.

2.2.2 Uniformly Most Powerful Tests

For composite hypotheses with one-sided alternatives, likelihood-ratio tests often yield uniformly most powerful (UMP) tests under certain monotone likelihood ratio conditions. For two-sided or more complex alternatives, UMP tests may not exist; in such cases, likelihood-ratio tests are often used as a default, though they may not be optimal in the strict Neyman–Pearson sense.

3 Applications

3.1 Diagnostic Testing

In medical and diagnostic contexts, likelihood ratios are used to update the probability of a condition based on a test result.

3.1.1 Positive and Negative Likelihood Ratios

The positive likelihood ratio (LR+) is defined as \(\frac{\text{sensitivity}}{1 - \text{specificity}}\), and the negative likelihood ratio (LR−) as \(\frac{1 - \text{sensitivity}}{\text{specificity}}\). These values quantify how much a positive or negative test result changes the odds of having a condition.

3.1.2 Pre-test and Post-test Probability

Using Bayes’ theorem, the post-test odds are the product of the pre-test odds and the likelihood ratio. This allows clinicians to estimate the probability of disease after a test result, given the prevalence (pre-test probability) and the test’s diagnostic accuracy.

3.2 Model Selection

Likelihood ratios are also used to compare statistical models.

3.2.1 Likelihood-Ratio Test for Nested Models

When two models are nested (one is a special case of the other), the likelihood-ratio test assesses whether the more complex model provides a significantly better fit. The test statistic follows the asymptotic chi-squared distribution under the null that the simpler model is correct.

3.2.2 Relation to Akaike Information Criterion (AIC)

The AIC is defined as \(-2 \log L + 2k\) (where \(k\) is the number of parameters). For nested models, the difference in AIC can be related to the likelihood-ratio statistic: \(\Delta \text{AIC} = -2 \log \lambda - 2\Delta k\). While the likelihood-ratio test uses a fixed significance threshold, AIC penalizes complexity more leniently, favoring models with lower AIC regardless of statistical significance.

4.1 Bayes Factor

The Bayes factor is the Bayesian analog of the likelihood ratio, defined as the ratio of marginal likelihoods under two hypotheses, integrating over prior distributions. While the likelihood ratio uses maximized (or specific) parameter values, the Bayes factor averages over parameter uncertainty, providing a direct measure of evidence that naturally incorporates prior information.

4.2 Likelihood Principle

The likelihood principle asserts that two experiments yielding proportional likelihood functions for the same parameter should lead to identical inferences. Since the likelihood ratio is derived directly from the likelihood function, it embodies this principle: all relevant information from the data is contained in the ratio.

4.3 Score Test and Wald Test

The score test (Rao’s test) and Wald test are alternatives to the likelihood-ratio test for hypothesis testing. The score test evaluates the slope of the log-likelihood under the null hypothesis, while the Wald test uses the estimated parameter value and its standard error.

4.3.1 Comparison of Test Types

Under standard regularity conditions, the three tests (likelihood-ratio, Wald, and score) are asymptotically equivalent. In finite samples, they may yield different p-values. The likelihood-ratio test is often preferred for its invariance under reparameterization and its solid theoretical foundation via Wilks’ theorem.

5 Computational and Practical Considerations

5.1 Numerical Stability (Log-Likelihood Ratio)

Direct computation of the likelihood ratio can suffer from underflow or overflow when likelihoods are extremely small. To avoid numerical issues, practitioners compute the log-likelihood ratio \(\ell\ell R = \log L(\theta_1) - \log L(\theta_0)\). Exponentiation of this difference yields the desired ratio only when needed, and the log scale is used for test statistics (e.g., \(-2 \log \lambda\)).

5.2 Small Sample Adjustments

When sample sizes are small, the asymptotic chi-squared distribution of \(-2 \log \lambda\) may be inaccurate. Adjustments such as Bartlett corrections or the use of exact distributions (when available) improve test performance. Alternatively, resampling methods like the bootstrap can provide more reliable p-values without relying on asymptotic approximations.