1 Definition and basic idea
The score test is a hypothesis test used to evaluate whether a parameter in a statistical model differs from a specified null value. It relies on the slope of the log-likelihood function at the null hypothesis, rather than on an estimate from the fully fitted alternative model. For this reason, it is also called a score-based test in likelihood theory.
The method is one of the three standard large-sample tests in classical inference based on maximum likelihood, the others being the likelihood-ratio test and the Wald test. It is often attractive when the restricted model under the null is simpler to estimate than the unrestricted model.
1.1 Likelihood-based hypothesis testing
In likelihood-based inference, evidence about a parameter is summarized through the likelihood function, which measures how well different parameter values explain the observed data. Hypothesis tests in this framework compare the null model with an alternative model that allows the parameter to vary.
The score test uses only information from the null model. It asks whether the likelihood would increase rapidly if the parameter were allowed to move away from its null value. This makes it especially useful when fitting the full model is difficult or unstable.
1.2 The score function
The score function is the derivative of the log-likelihood with respect to the parameter of interest. It measures the direction and steepness of change in the likelihood near a chosen parameter value.
If the score is close to zero at the null hypothesis, the data provide little immediate evidence against that null value. If the score is large in magnitude, the likelihood appears to increase quickly in some direction away from the null, suggesting that the null model may not fit well.
1.3 Intuition behind the test
The basic intuition is geometric. At the null hypothesis, the score test examines whether the likelihood surface has a noticeable slope. A steep slope indicates that moving away from the null could improve the fit substantially.
Unlike the Wald test, which evaluates an estimated coefficient relative to its standard error, the score test does not require estimating the parameter under the alternative. Unlike the likelihood-ratio test, it does not require fitting both models. This gives it a practical advantage in some settings.
1.4 Relationship to local alternatives
The score test is particularly sensitive to local alternatives, meaning alternatives that differ only slightly from the null hypothesis. It is designed to detect small departures in parameter values when the sample size is large.
Because of this local focus, the test has strong theoretical justification in asymptotic analysis. Under regular conditions, it is nearly equivalent to the other two classic likelihood-based tests for large samples.
2 Mathematical formulation
2.1 Null and alternative hypotheses
The score test is typically formulated as a test of a null hypothesis such as H0: θ = θ0 against an alternative such as H1: θ ≠ θ0. The parameter θ may be a scalar or a vector, depending on the model.
In many applications, the null model fixes one or more parameters while allowing nuisance parameters to be estimated. The test then evaluates whether the fixed parameter value is compatible with the observed data.
2.2 Score vector and information matrix
For a parameter vector θ, the score vector is the gradient of the log-likelihood, usually written as U(θ). It contains the first derivatives of the log-likelihood with respect to each parameter.
The information matrix measures the curvature of the log-likelihood and summarizes how sharply the likelihood is peaked. In practice, the expected information or observed information may be used, depending on the setting.
2.3 Test statistic
A common score test statistic has the form U(θ0)ᵀ I(θ0)⁻¹ U(θ0), where U(θ0) is the score evaluated under the null hypothesis and I(θ0) is the information matrix under the null. For a single parameter, this reduces to a squared standardized score.
When the null hypothesis is true, this statistic is compared to a chi-square distribution with degrees of freedom equal to the number of tested parameters. A large value indicates that the null model is unlikely to explain the data adequately.
2.4 Asymptotic distribution
Under standard regularity conditions and a true null hypothesis, the score statistic has an asymptotic chi-square distribution. This approximation improves as sample size increases.
If the alternative is local and close to the null, the statistic converges to a noncentral chi-square distribution. The noncentrality reflects the degree of departure from the null.
3 Derivation
3.1 Taylor expansion of the log-likelihood
The score test can be derived by expanding the log-likelihood around the null parameter value using a Taylor approximation. The first derivative term captures the score, while the second derivative term captures curvature through the information matrix.
This expansion shows how the likelihood changes near the null without requiring a full optimization over the alternative model. The resulting quadratic approximation leads directly to the score statistic.
3.2 Connection to maximum likelihood estimation
Maximum likelihood estimation seeks the parameter value that maximizes the log-likelihood. If the null hypothesis is true, then the score at the null should be near zero, because the null value is close to the maximizer.
The score test measures how far the null point is from being a local maximum. It compares the slope at the null with the amount of curvature, which provides a natural scale for judging whether the departure is meaningful.
3.3 Regularity conditions
The derivation assumes regularity conditions such as differentiability of the likelihood, identifiable parameters, and suitable behavior of the sample information. These conditions support the use of Taylor expansion and asymptotic normality.
Additional assumptions often include independence or weak dependence of observations and parameter values in the interior of the parameter space. When these conditions fail, the test may require modification.
3.4 Large-sample approximation
The score test is fundamentally an asymptotic method. Its justification comes from large-sample theory, where the score and information have stable limiting behavior.
In finite samples, the approximation may be less accurate, especially for small datasets or irregular models. Nonetheless, the method often performs well in practical likelihood-based applications.
4 Relationship to other tests
4.1 Likelihood-ratio test
The likelihood-ratio test compares the maximized likelihood under the null model with the maximized likelihood under the alternative model. It requires fitting both models, which can be computationally demanding.
The score test, in contrast, uses only the null fit. Both tests are asymptotically equivalent under standard conditions, though they may differ in finite samples.
4.2 Wald test
The Wald test uses an estimated parameter and its estimated standard error to judge whether the parameter differs from the null value. It is simple to compute once the full model has been fitted.
The score test avoids dependence on the full-model estimate. This can be advantageous when the alternative model is hard to fit or when estimates become unstable near the boundary of the parameter space.
4.3 Equivalence under large samples
For regular models and large samples, the score, Wald, and likelihood-ratio tests often lead to similar conclusions. They share the same first-order asymptotic behavior under the null hypothesis.
Differences become more noticeable in smaller samples or in models with skewed likelihoods, weak identification, or boundary issues. In such cases, the choice of test may affect the inference.
4.4 Advantages and limitations relative to other tests
The main advantage of the score test is that it requires only the restricted model. This can save computation and avoid convergence problems in the full model.
Its limitations include possible sensitivity to how the null is specified and reduced accuracy in some finite-sample settings. It may also be less intuitive to interpret than a direct estimate-based test.
5 Common applications
5.1 Generalized linear models
In generalized linear models, score tests are often used to assess the contribution of predictors or to compare nested models. They are convenient because the null model can usually be fitted easily.
These tests appear in settings where responses are binary, count-based, or otherwise non-normal. They are widely used for checking whether adding a predictor improves model fit.
5.2 Logistic regression
In logistic regression, a score test can evaluate whether a covariate has an effect on the log-odds of an outcome. The test is often used when considering the inclusion of one variable or a set of variables.
Because the null model may include only an intercept or a small set of predictors, the score test provides a straightforward way to assess whether additional terms are justified.
5.3 Poisson regression
In Poisson regression, the score test is commonly used to examine whether a count predictor or exposure term contributes significantly to the model. It is especially useful in nested model comparisons.
The method can also be applied to detect overdispersion-related effects or to test whether a parameter linked to the mean count differs from zero.
5.4 Survival analysis
Score tests are central in survival analysis, particularly in semiparametric and proportional hazards frameworks. They are used to assess whether covariates influence survival times or hazard rates.
The approach is attractive because it can test covariate effects without requiring full estimation of every candidate model under consideration.
5.4.1 Proportional hazards models
In proportional hazards models, the score test evaluates whether a covariate contributes to the hazard function. It is often formulated using the partial likelihood, which focuses on relative risk rather than the baseline hazard.
This makes the test practical for time-to-event data, where the baseline hazard may be treated as an unspecified nuisance quantity.
5.4.2 Testing covariate effects
Covariate effects are often tested one at a time or in groups. The score statistic can assess whether a variable, interaction term, or set of regressors improves the explanatory power of the survival model.
Such tests are useful during model building and variable selection, especially when the number of candidate effects is large.
6 Variants and extensions
6.1 Rao's score test
Rao's score test is a classic formulation of the score test associated with C. R. Rao. It is widely cited as the standard version of the method in parametric inference.
The test can be applied to single parameters or multiple restrictions, and it remains one of the most familiar likelihood-based procedures in statistical practice.
6.2 Robust score tests
Robust score tests adjust for certain departures from model assumptions, such as mild heteroskedasticity or misspecification of variance structure. They are designed to improve reliability when classical assumptions are imperfect.
These versions often replace the model-based information with an empirical or sandwich-type estimate. As a result, they may be less efficient under the ideal model but more dependable under deviations.
6.3 Score tests for nuisance parameters
In many models, the parameter of interest is accompanied by nuisance parameters that are not directly of substantive concern. Score tests can be constructed to account for these nuisance terms while focusing on the target restriction.
This is useful when the null hypothesis concerns one component of a larger parameter vector. The nuisance-adjusted score isolates the effect of the parameter being tested.
6.4 Score tests in constrained models
Score tests also appear in constrained optimization problems, where parameters are restricted by equality or inequality conditions. The method can be adapted to check whether relaxing a constraint would significantly improve fit.
In these contexts, the test may involve projected scores or modified information matrices. The general principle remains the same: evaluate the local tendency of the likelihood to move away from the constrained point.
7 Computation
7.1 Estimation under the null
To compute a score test, one first fits the model under the null hypothesis. The parameter estimates from this restricted model are then used to evaluate the score and information.
This step is typically simpler than fitting the full model. In many software environments, the needed quantities are already produced during estimation.
7.2 Practical implementation
Implementation usually involves extracting the gradient and information matrix from the fitted null model. The score statistic is then formed and converted into a p-value using the appropriate reference distribution.
For multiple restrictions, the test may require matrix operations and careful handling of parameter ordering. Good software automates these details, reducing the chance of manual error.
7.3 Numerical stability
Because the score test uses only the null fit, it can be numerically stable when the alternative model is difficult to estimate. This can be helpful in models with convergence problems or highly correlated predictors.
However, instability may still arise if the information matrix is nearly singular or if the null model is poorly conditioned. In such cases, diagnostic checks are important.
7.4 Software and statistical packages
Most major statistical software systems include score test functionality, either directly or through model comparison tools. The test is common in generalized linear model routines, survival analysis packages, and likelihood-based estimation libraries.
The exact labeling may vary. Some systems refer to it as a score test, others as a Lagrange multiplier test, though both terms often describe the same underlying method.
8 Interpretation
8.1 P-values and significance
The p-value from a score test measures how unusual the observed score statistic would be if the null hypothesis were true. A small p-value suggests that the null model is not adequate for the observed data.
As with all hypothesis tests, statistical significance does not by itself measure practical importance. The result should be interpreted in the context of the model and the scientific question.
8.2 Effect size considerations
A significant score test indicates evidence against the null, but it does not directly estimate the magnitude of the effect. For that, one typically examines parameter estimates from the full model or other effect-size summaries.
This distinction is important in applications where even small departures from the null can become significant in large samples. Practical relevance should be assessed separately from hypothesis rejection.
8.3 Model diagnostics
Score tests can also serve as model-checking tools. They help identify whether adding terms, interactions, or alternative structural components may improve the model.
In exploratory analysis, a score test may guide further specification decisions. Still, it should be used alongside residual analysis, fit measures, and subject-matter knowledge.
9 Assumptions and limitations
9.1 Sample size requirements
The usual chi-square approximation depends on a sufficiently large sample. In small samples, the nominal reference distribution may be inaccurate, leading to misleading p-values.
When data are limited, alternative methods such as exact tests, simulation-based inference, or small-sample corrections may be preferable.
9.2 Model misspecification
If the fitted likelihood model is misspecified, the score test may inherit that error. In particular, incorrect distributional assumptions or variance structure can distort the test statistic.
Robust versions can mitigate some of these problems, but they do not eliminate the need for careful model assessment. The validity of the inference depends strongly on the quality of the underlying model.
9.3 Dependence on correct null specification
The score test is built around the null model, so its validity depends on that restricted specification being correct apart from the tested parameter. If the null model is badly chosen, the test may not answer the intended question.
This dependence is both a strength and a weakness. It makes the test computationally efficient, but it also means that misspecification in the baseline model can influence the result.
9.4 Sensitivity to boundary cases
Standard theory assumes the true parameter lies in the interior of the parameter space. When parameters are on the boundary, such as variance components equal to zero, the asymptotic distribution may differ from the usual chi-square form.
In such cases, modified reference distributions or specialized methods are needed. Boundary problems are common in mixed models and certain constrained estimation settings.
10 Historical development
10.1 Early likelihood theory
The score test emerged from the broader development of likelihood theory in the first half of the 20th century. As maximum likelihood methods became central to statistical inference, researchers studied several ways to compare nested models.
The idea of using derivatives of the likelihood at the null fit fit naturally into this framework. It provided an efficient alternative to repeatedly estimating competing models.
10.2 Rao's contribution
C. R. Rao gave a formal and influential treatment of the score test, helping establish it as a standard tool in mathematical statistics. His work clarified the relationship between likelihood derivatives and large-sample testing.
Because of this contribution, the test is frequently called Rao's score test. The terminology reflects both the derivative-based nature of the method and its historical foundation.
10.3 Adoption in modern statistics
The score test became widely used as computational statistics expanded and model-based inference grew more sophisticated. Its efficiency under the null made it especially useful in regression, survival analysis, and generalized linear modeling.
Today it remains a standard component of likelihood-based analysis. Its role is especially prominent when a quick and reliable test is needed without fitting a complicated alternative model.