1 Background and Motivation

1.1 From Linear Regression to Generalized Linear Models

Classical linear regression models the conditional mean of a response as a linear function of predictors. This approach relies on assumptions that often fail when the outcome is binary, a count, or strictly positive with skewness. Generalized linear models (GLMs) extend the linear regression idea by permitting non-normal response distributions while retaining a structured relationship between predictors and the mean of the response.

1.2 The Linear Predictor and the Response Mean

A GLM introduces a linear predictor, typically written as η, formed by a linear combination of covariates. The link function then maps η to the response mean μ (the expected value of the outcome given predictors). This mapping allows the model to represent systematic changes in μ in a way that is compatible with the response’s distribution.

Using the identity link (directly modeling μ as linear in predictors) can be problematic because it does not automatically respect constraints implied by the data-generating process. For instance, probabilities must lie in [0, 1], counts cannot be negative, and certain continuous measurements are constrained to be positive. Non-identity links are designed so that the transformed linear predictor yields fitted means that remain in the appropriate range.

1.4 Interpretability of Coefficients

Even when the fitted model is mathematically coherent, the meaning of coefficients depends on the link. For example, a one-unit change in a predictor may correspond to an additive shift in η but to a multiplicative or nonlinear change in μ. Consequently, interpretation requires reporting results on the correct scale or translating the effects through the link.

2 Formal Definition

2.1 Mean-Parameterization

2.1.1 Linking μ to η

In a GLM, the mean μ is expressed as a function of the linear predictor η through a link function g(·). A common formulation is:

  • g(μ) = η

where η is constructed as a linear combination of covariates, and μ is constrained by the response distribution.

2.1.1.1 Notation and typical conventions

Standard notation uses:

- μ = E(YX)
  • η = Xβ (often written as a linear predictor)
  • g is a monotone function selected to align with the distribution and support of μ

The inverse link, g⁻¹, maps η back to the mean: μ = g⁻¹(η).

2.1.2 The role of g(·)

The link function provides the transformation that connects the scale where predictors enter the model (η-scale) to the scale where the mean is defined (μ-scale). Because predictions and likelihood contributions depend on μ, the link shapes the fitted relationship and the implied variance structure.

2.2 Distributional Assumptions in GLMs

2.2.1 Exponential family overview

GLMs are typically defined so that Y follows a distribution in the exponential family. Under this framework, both the mean μ and a dispersion parameter govern the shape of the likelihood, enabling unified estimation procedures across disparate outcome types (e.g., Bernoulli, Poisson, Gamma).

Within the exponential-family setup, each distribution has a canonical link that often yields algebraic simplifications. While canonical links can make estimation and interpretation convenient, GLMs also allow non-canonical links, which may be preferable for stability, variance behavior, or interpretability.

2.3 Range and Domain Constraints

2.3.1 Ensuring valid fitted means

A key practical motivation for choosing g is to ensure that g⁻¹(η) produces valid μ values. For example, if the outcome is Bernoulli, g⁻¹(η) must return probabilities between 0 and 1. If the outcome is a count model, the fitted mean must be nonnegative, and in many cases strictly positive.

2.3.2 Boundary behavior

Different links approach the boundaries of the mean space in different ways. Some links send large positive η toward an upper bound and large negative η toward a lower bound, yielding controlled asymptotic behavior. Others allow more direct linear behavior in transformed space, but still enforce feasibility through the inverse link.

The identity link sets η = μ (so μ = η). It is most natural when the response mean is unconstrained on the real line or when the modeling context ensures that predictions remain meaningful over the range of interest.

3.1.1 When it is appropriate

Identity links are often used in settings where the mean is plausibly linear in predictors and where the distribution supports that linearity without generating invalid fitted values (e.g., certain positive continuous models with careful constraints, or Gaussian-like contexts). For strictly bounded or strictly nonnegative outcomes, identity links may require additional handling to avoid impossible fitted means.

The log link uses η = log(μ), equivalently μ = exp(η). This automatically enforces μ > 0, which aligns with many distributions defined on positive values such as Poisson for counts and certain continuous positive families.

3.2.1 Interpreting multiplicative effects

Because μ is an exponential function of η, additive changes in η translate into multiplicative changes in μ. As a result, coefficients on predictors can be interpreted through ratios (e.g., expected counts scale by exp(β) for a one-unit predictor change), holding other covariates fixed.

The logit link is defined by η = log(μ/(1 − μ)), so that μ = 1/(1 + exp(−η)). It maps any real-valued η to probabilities in (0, 1).

3.3.1 Odds and probability interpretation

In the logit model, η changes linearly with predictors, and exponentiating the coefficients yields odds ratios. This produces a standard probabilistic interpretation after converting back to μ. However, the effect on probability is nonlinear, especially near the boundaries where μ approaches 0 or 1.

The probit link uses η = Φ⁻¹(μ), where Φ denotes the standard normal cumulative distribution function. Thus μ = Φ(η).

3.4.1 Relation to latent-variable interpretations

The probit model can be motivated by a latent normal-threshold view: an observed binary outcome is tied to whether a hidden continuous variable exceeds a threshold. This framing can be useful for interpretation, even when the primary goal is prediction.

The complementary log-log (cloglog) link is defined as η = log(−log(1 − μ)). Its inverse is μ = 1 − exp(−exp(η)).

3.5.1 Asymmetry and hazard-like behavior

Unlike the symmetric logit and probit links, the cloglog link yields asymmetry in how probabilities change with predictors. It often produces patterns consistent with processes that have increasing hazard over time or skewed dependence on η, which can be relevant for event modeling contexts.

Power links take forms related to transforming μ through a power, such as η = μ^p for certain p values (with specific versions depending on the distribution and constraints).

3.6.1 Flexibility and transformation ideas

Power-family links can offer flexibility by adjusting how quickly μ changes as η moves. They can be useful when the relationship between predictors and the mean is better captured by a transformation that emphasizes certain regions of the mean space.

A valid GLM requires that the inverse link produce fitted means compatible with the response’s support. Beyond feasibility, the link should also reflect plausible functional form: whether the effect of predictors on μ is closer to additive, multiplicative, or saturating behavior.

4.2 Data-Driven Selection Strategies

4.2.1 Comparing fit using likelihood-based criteria

Link selection is often guided by model comparison tools such as likelihood-based criteria (e.g., AIC variants) computed under alternative link functions. These comparisons evaluate whether the added flexibility in the mean transformation improves predictive performance while remaining consistent with the assumed distribution.

4.2.2 Diagnostics and residual patterns

Diagnostic work complements formal criteria. Analysts may examine patterns in residuals (or working residuals), fitted-versus-observed plots, and leverage or influence measures to detect systematic misfit that suggests the link is distorting the relationship between η and μ.

4.3 Theoretical Considerations

When a canonical link is available, it can simplify mathematical expressions for estimation and can yield convenient forms for score functions and iterative updates. Canonical choices are not always best in practice, but they are frequently considered a baseline.

4.3.2 Stability and numerical issues

Some links can lead to slow convergence or numerical difficulties when the fitted η values produce μ values close to boundaries. Practical selection includes checking optimizer behavior, warnings from software, and sensitivity to starting values.

4.4 Sensitivity Analysis

Because the link affects both coefficient interpretation and fitted mean shape, it is common to evaluate whether substantive conclusions persist under several plausible link functions. Robustness checks help distinguish findings driven by modeling assumptions from those supported by the data.

5 Interpretation and Reporting

Coefficients are defined on the η-scale, but stakeholders typically care about changes in μ or probabilities. Under each link, a unit change in a predictor corresponds to a different transformation on the mean. Clear reporting often involves specifying the link and providing translated interpretations (e.g., odds ratios for logit models or multiplicative effects for log links).

5.2 Marginal Effects and Scale

5.2.1 From η-scale to μ-scale

Marginal effects describe how μ changes when a predictor changes, holding others fixed. Converting from η-scale requires applying the inverse link and differentiating as needed. Because the mapping from η to μ can be nonlinear, marginal effects may vary across the covariate space.

5.2.2 Differences between odds, risks, and rates

Reporting can be complicated by terminology. Odds relate naturally to logit models, while risks correspond directly to probabilities for binary outcomes. Rates or expected counts arise in log-linked count models. Confusing these quantities can lead to incorrect conclusions even when the statistical model is correct.

5.3 Predicted Values and Transformations

5.3.1 Back-transforming to the mean scale

Predictions obtained on the η-scale must usually be transformed back to μ using the inverse link for interpretation. This step is essential for generating interpretable fitted probabilities, expected counts, or mean responses.

5.3.2 Confidence intervals for μ

Uncertainty quantification can be conducted in multiple ways, such as simulating from the estimated parameter distribution or using approximate variance propagation. Because μ is a nonlinear function of η, intervals on the μ-scale are not generally identical to those computed on the η-scale.

6 Estimation and Inference Implications

6.1 Maximum Likelihood Estimation in GLMs

GLMs commonly use maximum likelihood estimation (MLE) based on the assumed distribution within the exponential family. The link function enters the likelihood through μ = g⁻¹(η), shaping how covariates influence the probability or density of observed outcomes.

6.2 Iteratively Reweighted Least Squares (IRLS)

Many GLM software implementations estimate parameters via IRLS, which repeatedly solves a weighted least squares problem. The weights depend on both the variance function of the distribution and the derivative of μ with respect to η, so the link function affects the weighting scheme and therefore the updates.

6.3 Standard Errors and Asymptotics

Standard errors are typically based on approximations using the observed or expected information matrix. Because the link changes the relationship between η and μ, it also affects the curvature of the likelihood around the optimum, leading to different uncertainty estimates across links.

6.4.1 Systematic patterns and leverage effects

Model checking often looks for systematic deviations that could indicate a wrong link. For example, if predicted probabilities are consistently too low for certain predictor ranges, it may signal that the mapping from η to μ is miscalibrated. Leverage and influential points can exacerbate such problems, especially near the boundaries of μ.

7 Practical Examples (Lightweight)

For binary data (e.g., success/failure), logit and probit links are common. In such models, the linear predictor η combines covariates, and the inverse link produces predicted probabilities. The choice between logit and probit typically yields different coefficient-to-probability mappings, though overall fit can be similar in many datasets.

For count outcomes, log links are frequently used to ensure nonnegative fitted means. A coefficient in η corresponds to a multiplicative factor on the expected count. This can be convenient when covariates represent exposures or when changes in predictors plausibly scale the expected number of events.

7.3 Positive Continuous Data: Identity vs Log Variants

For positive continuous measurements, identity-like links may be used if the model is carefully constrained and remains plausible across the observed range. Log-based links often provide better control over positivity and may capture skewness more effectively. Analysts frequently compare both while focusing on whether predictions remain within reasonable bounds.

8 Misconceptions and Pitfalls

A link function is the mathematical transformation between μ and η, not a separate “link model” of entities or relationships. While networking “links” are a popular metaphor online, in GLMs the term refers strictly to the mapping that relates predictors to the response mean.

8.2 Overlooking the Mean-Scale vs Linear-Predictor-Scale

A common mistake is to interpret coefficients as if they directly add to probabilities or means. In most GLMs, predictors combine linearly only on η-scale. Effects on μ-scale depend on the nonlinear inverse link, so direct coefficient interpretation without back-transformation can be misleading.

If the chosen link allows inverse-link outputs outside the feasible range of μ, fitted values may become invalid (e.g., probabilities below 0 or above 1). Even when software constrains outputs or returns warnings, the underlying mismatch can still distort inference and predictive behavior.

8.4 Extrapolation beyond Observed Data

Predictions far from the covariate values used to fit the model may behave poorly because the link function determines how μ responds to largeη. Since the mapping is often nonlinear, extrapolation can yield extreme probabilities or means, particularly near boundary regions.

9 Software Implementation Notes

9.1 Typical Function Interfaces in Modeling Tools

Most statistical packages allow specifying a link through arguments in GLM-related functions or via a model formula interface. The interface often requires both the distribution family and the link choice, because the two are coupled through the mean and variance modeling assumptions.

Software typically offers link options such as logit, probit, log, identity, and cloglog. These options are validated against the selected family. When an incompatible pair is chosen, implementations may error out or silently adjust behavior, so checking documentation and warnings is important.

Interactions and nonlinear predictor terms (e.g., polynomial terms or splines) enter the linear predictor η in the usual way. The link then transforms η to the mean μ, which means that even simple nonlinearities in predictors can translate into complex shapes for μ depending on the inverse link.

9.4 Common warnings and how to interpret them

Warnings may indicate issues such as nonconvergence, predicted means near boundaries, perfect separation in binary models, or overdispersion in count models. While such messages do not always imply failure, they signal that the link-function-plus-family combination may be strained by the data and deserves additional diagnostics.