1 Introduction to Normality Testing
1.1 Why Test for Normality
Normality testing is used to evaluate whether a set of observations can reasonably be treated as draws from a normal distribution. This question arises in many statistical workflows, especially those that rely on normality for exact inference, simplified formulas, or robust approximations. When normality is questionable, analysts may select alternative models, apply transformations, or use methods with weaker distributional assumptions.
1.2 The Empirical Distribution Function
A central object in goodness-of-fit work is the empirical distribution function (ECDF), which summarizes a sample by assigning each value a stepwise cumulative probability. The ECDF provides a nonparametric baseline for how the data accumulate across the range of observations. By comparing the ECDF to a candidate cumulative distribution function, a discrepancy measure can be formed and used to assess fit.
1.3 Relationship to Goodness-of-Fit Tests
The Lilliefors test belongs to the broader family of goodness-of-fit procedures that compare an empirical distribution to a theoretical one. It is closely related to the Kolmogorov–Smirnov framework, but adjusts for a key practical issue: when parameters of the fitted normal distribution are estimated from the same data, the reference distribution of the test statistic changes. The Lilliefors modification incorporates this effect so that rejection thresholds remain appropriate under the fitted-parameter setting.
2 The Lilliefors Test in Detail
2.1 Core Idea: Parameter-Estimated Kolmogorov–Smirnov Variant
The Kolmogorov–Smirnov test can be formulated to compare an ECDF against a known continuous distribution. In the normality setting, however, the mean and variance are typically unknown and are replaced by estimates computed from the sample. The Lilliefors test modifies the Kolmogorov–Smirnov approach to reflect that the fitted normal cumulative distribution function is no longer independent of the observed sample, altering the distribution of the maximum discrepancy.
2.2 Test Statistic Definition
The test statistic is based on the maximum absolute difference between:
- the ECDF of the sample, and
- the cumulative distribution function of the normal distribution evaluated using estimated parameters (commonly the sample mean and sample variance).
Formally, the statistic captures the largest vertical gap between the empirical step function and the fitted normal curve across all points in the data’s domain. Large values indicate that the sample’s accumulation pattern diverges substantially from what the fitted normal model predicts.
2.3 Null and Alternative Hypotheses
The null hypothesis states that the data come from a normal distribution with some mean and variance, with those parameters estimated from the same sample. The alternative hypothesis is broad: the distribution of the data is not normal. Because the alternative is not specified in a narrow parametric way, the test is intended as an overall check for normality rather than a targeted detector of a particular deviation type.
2.4 Interpretation of Results
A typical output is either a rejection decision at a chosen significance level or an associated p-value. Rejection indicates evidence inconsistent with normality under the fitted-parameter normal model. Failure to reject does not prove normality; it only suggests that discrepancies are not large enough, relative to the test’s reference distribution, to conclude non-normality. Interpretation should be tempered by sample size, the presence of outliers, and the possibility of departures that the test has limited power to detect.
3 Assumptions and Requirements
3.1 Data Scale and Measurement Considerations
The procedure is designed for continuous data measured on an appropriate scale. If the data are derived from transformations (e.g., logarithms) or represent bounded/rounded measurements, the interpretation of “normality” pertains to the transformed scale used in the test. For variables with strong measurement constraints or truncation, the fitted normal model may not be the right baseline, which can affect conclusions.
3.2 Independence and Sample Design
Standard versions of the test assume the sample observations are independent and identically distributed. Dependence structures (time series correlations, clustered sampling, repeated measures) can distort the effective variability and alter the behavior of the empirical distribution, making nominal significance levels unreliable. In such cases, analysts typically require dependence-aware methods or resampling strategies.
3.3 Continuous vs. Discrete Data Handling
The Lilliefors test is usually presented for continuous distributions. With discrete data, ties occur in the ECDF, and the distribution of the test statistic can deviate from the continuous reference used to compute critical values. Some implementations still perform the test, but results should be interpreted cautiously; discretization can mask or amplify discrepancies depending on how ties align with the fitted model.
3.4 Parameter Estimation for the Reference Normal Model
The test requires a specific estimation strategy for the normal parameters used in the reference cumulative distribution function. In common practice, the sample mean estimates the location and the sample variance estimates scale. The parameter-estimation step is part of the test’s defining feature: the critical values (or p-values) correspond to the parameter-estimated setting. Using a different estimator without corresponding adjustments can invalidate the nominal calibration.
4 Critical Values and Decision Rules
4.1 Asymptotic vs. Finite-Sample Guidance
The Lilliefors test may be calibrated using asymptotic approximations for large samples, or using finite-sample critical values provided in tables and implementations. Asymptotic guidance offers convenience but may be less accurate for small n, while table-based values can better reflect the finite-sample distribution under the null. Choice of calibration method should match the sample size and the software’s documentation.
4.2 Use of Lilliefors-Specific Tables
Because the null distribution depends on parameter estimation, the critical thresholds differ from those of the standard Kolmogorov–Smirnov test with fully specified parameters. Lilliefors-specific tables, or algorithmic computations of the adjusted critical values, supply the rejection cutoffs corresponding to chosen significance levels. These values are essential for correct decision-making when fitting the normal distribution within the test.
4.3 P-Values and Rejection Thresholds
A p-value summarizes the probability, under the null hypothesis with fitted parameters, of observing a test statistic at least as extreme as the one computed from the sample. Rejection typically occurs when p is less than or equal to the selected significance level α. Some software reports p-values directly; others report only the test statistic and require the user to compare against critical values.
4.4 Common Significance Levels
Analysts often use conventional thresholds such as 0.05 or 0.01. The choice of α affects the trade-off between Type I error (rejecting when normality holds) and Type II error (failing to reject when normality is false). In exploratory work, higher α may be used to avoid missing deviations, while in confirmatory contexts, lower α helps reduce false positives.
5 Practical Workflow
5.1 Computing the Test Statistic
The computation proceeds by:
- ordering the sample values (implicitly via the ECDF),
- constructing the ECDF,
- fitting the normal distribution to obtain parameter estimates, and
- calculating the maximum absolute difference between ECDF and the fitted normal CDF over the relevant domain (often at the sample points or near them, depending on implementation details).
The resulting maximum gap is the test statistic used for calibration.
5.2 Fitting the Normal Distribution
The normal fit typically uses the sample mean and sample variance. Some implementations require a specific convention for the variance estimator (e.g., dividing by n or n−1), which can change the fitted CDF slightly. For consistency, users should rely on the estimator convention built into the test function provided by the software.
5.3 Running the Lilliefors Procedure in Software
Most statistical packages implement the Lilliefors test through built-in functions. The user supplies the data vector and the function returns either a p-value or a decision according to a chosen α. Software may also expose options for the variance normalization convention or for whether to use asymptotic approximations. Reproducibility depends on matching these settings to the assumptions used in the calibration.
5.4 Reporting the Outcome in a Results Section
A clear results report typically includes the test name, sample size, the p-value (or test statistic and critical value comparison), and the conclusion regarding normality at the stated α. If multiple variables are tested, it is common to describe the context (e.g., exploratory screening versus confirmatory modeling) and any adjustment method used to address multiplicity.
6 Comparison With Other Normality Tests
6.1 Versus Shapiro–Wilk
The Shapiro–Wilk test is often regarded as highly sensitive for detecting certain kinds of non-normality, particularly when deviations involve the central part of the distribution and when sample sizes are moderate. Compared with the Lilliefors test, differences arise from how each statistic weights discrepancies across the distribution and how each treats parameter estimation. In practice, analysts choose based on software defaults, sample size range, and the types of departures of interest.
6.2 Versus Anderson–Darling
The Anderson–Darling test is a goodness-of-fit method that emphasizes tail behavior more than many alternatives. This tail weighting can be beneficial when non-normality manifests strongly in extremes (e.g., heavy tails or skewed tails). The Lilliefors approach, centered on the Kolmogorov-type maximum gap, is sensitive to the largest cumulative mismatch but does not explicitly reweight tails in the same systematic way as Anderson–Darling.
6.3 Versus Standard Kolmogorov–Smirnov
The standard Kolmogorov–Smirnov test assumes the reference distribution parameters are known. If mean and variance are estimated but the standard critical values are still used, the resulting p-values are generally miscalibrated. The Lilliefors test corrects for this by incorporating the fitted-parameter effect, producing decision thresholds aligned with the actual uncertainty introduced by estimation.
6.4 Sensitivity to Different Departures from Normality
Different tests react differently to deviations such as skewness, kurtosis, outliers, and multimodality. A Kolmogorov-type statistic focuses on the maximum ECDF–CDF discrepancy, so it can be driven by a localized region where the fit is worst. Other tests may average discrepancies across the domain or emphasize tails. As a result, the same dataset can lead to different conclusions depending on which aspects of non-normality are most pronounced.
7 Strengths, Limitations, and Pitfalls
7.1 Power Under Alternatives (General Intuition)
In general terms, statistical power depends on how strongly the sample deviates from normality and in which parts of the distribution the mismatch occurs. The Lilliefors statistic can be effective when the empirical distribution diverges noticeably from the fitted normal CDF at some point. However, when departures are subtle or distributed smoothly so that no single maximum gap becomes large, power may be limited.
7.2 Effects of Small Sample Sizes
With small n, the fitted normal parameters can vary substantially from one sample to another, and the calibration of the test becomes especially important. Asymptotic approximations may be less reliable, and finite-sample critical values or simulation-based p-values may be preferable. Additionally, small samples naturally reduce the ability to distinguish normality from close alternatives.
7.3 Handling Outliers and Heavy Tails
Outliers can influence the ECDF and also affect parameter estimates, potentially increasing the statistic by producing a region with a large cumulative mismatch. Heavy-tailed distributions may yield discrepancies in the tails that become prominent enough to drive rejection, though the exact sensitivity depends on where the maximum gap occurs. Robustness is therefore not guaranteed; analysts should interpret results in light of data characteristics and consider diagnostic plots.
7.4 Multiple Testing Considerations
When normality is assessed across many variables, repeated testing increases the chance of false discoveries. The Lilliefors test itself addresses only the single-variable hypothesis. For studies involving numerous outcomes, multiplicity-aware approaches (such as controlling the false discovery rate) may be needed to make the overall set of conclusions more reliable.
8 Variants and Extensions
8.1 Versions for Different Parameterizations
Extensions of the Lilliefors idea exist for alternative parameterizations or estimation schemes for the reference distribution. The key principle remains: when parameters are estimated from the data, the null distribution of the test statistic must reflect that estimation. Different versions may therefore correspond to different ways of estimating location, scale, or other parameters, with corresponding critical values or p-value computations.
8.2 Extensions to Other Distributional Targets
The concept of modifying a goodness-of-fit test to account for estimated parameters can be applied beyond the normal case. For example, similar strategies can be used when testing against fitted members of other parametric families, provided that the distribution of the resulting statistic under parameter estimation is properly handled. Such extensions are useful when normality is not the target but a different parametric baseline is.
8.3 Connections to Empirical Process Theory
Lilliefors-type tests can be understood through the lens of empirical process theory, which studies the asymptotic behavior of stochastic processes formed by the ECDF. Parameter estimation changes the limiting process, producing adjusted asymptotic distributions that motivate the modified critical values. This connection helps explain why goodness-of-fit statistics behave differently when the reference model is fitted rather than fixed.