1 Introduction to Propensity Scores

1.1 Definition and intuition

A propensity score is the conditional probability that a unit receives a particular treatment (or exposure) given observed covariates. Formally, if \(T\) denotes treatment and \(X\) denotes covariates, the propensity score is \(e(X)=P(T=1\mid X)\). The central intuition is that if two individuals—one treated and one untreated—share the same (or very similar) propensity scores, then they have comparable probabilities of being treated under the treatment assignment mechanism implied by the observed data.

This idea allows analysts to work with a single balancing score rather than adjusting directly for many covariates. In observational studies, treatment is rarely randomized, so differences between treated and untreated groups may reflect both the treatment effect and systematic differences in baseline characteristics. Propensity score methods aim to reduce the latter by aligning groups with similar likelihoods of receiving treatment.

1.2 Assumptions and what “balancing” means

Propensity score approaches rely on two broad principles:

  1. Confounding control via observed covariates: After conditioning on \(X\) (and thus on the propensity score), treatment assignment is as if random with respect to potential outcomes. This is often described as *conditional exchangeability* or *no unmeasured confounding* given observed covariates.
  2. Positivity (overlap): For covariate patterns that occur in the data, there must be a nonzero chance of receiving both treatment and control. Without overlap, “similar” comparisons may not exist, and balancing cannot be meaningfully achieved.

*Balancing* refers to the property that, after appropriate use of propensity scores (e.g., matching, weighting), the distribution of covariates is made similar between treated and control groups. When balance is achieved, estimated treatment effects are less likely to be artifacts of baseline differences.

1.3 When propensity score methods are appropriate

Propensity score methods are particularly useful when:

  • The study is observational and treatment assignment is plausibly related to observed covariates.
  • There are many covariates and analysts want a structured way to adjust for them.
  • The goal is causal interpretation under assumptions about confounding and overlap.

They are not a universal solution. When treatment assignment is nearly deterministic for some covariate patterns, or when key predictors are missing, propensity score strategies can fail or provide misleading inferences.

2 The Propensity Score Model

2.1 Choosing covariates

Choosing covariates is the foundation of propensity score analysis. Common practice is to include variables that predict both treatment assignment and outcomes, especially those measured before treatment occurs. Including irrelevant variables is not necessarily harmful, but it can complicate estimation and diagnostics; including variables that are affected by treatment can introduce bias by conditioning on post-treatment information.

A well-defined covariate set is often based on subject-matter knowledge and study design. Variable coding (e.g., transformations, categorization) should be planned so that meaningful differences in baseline risk are captured.

2.2 Specifying the treatment assignment model

2.2.1 Logistic regression and extensions

A common parametric way to estimate \(e(X)\) is logistic regression for binary treatment: \[ \text{logit}\,P(T=1\mid X)=\beta_0+\beta^\top X. \] Extensions include adding polynomial terms, interaction terms, or using generalized linear models when treatment is non-binary. Parametric models can be efficient when the functional form is close to the truth, but misspecification can compromise balance.

2.2.2 Machine learning for propensity estimation

Machine learning methods (e.g., gradient boosting, random forests, neural networks) are frequently used to flexibly estimate treatment probabilities, particularly when relationships between covariates and treatment are complex. The goal is to approximate \(P(T=1\mid X)\) without relying heavily on rigid functional assumptions.

When using flexible learners, analysts typically still conduct balance diagnostics and may apply regularization, cross-validation, or sample-splitting to reduce overfitting. Overfitting can produce extreme estimated scores and degrade overlap, even if in-sample prediction is strong.

2.3 Overlap and positivity considerations

After estimating propensity scores, overlap is assessed by examining the distribution of scores in treated and control groups. Lack of overlap often shows up as:

  • Extreme propensities near 0 or 1
  • Treated units whose scores have little representation among controls (or vice versa)

Positivity problems can lead to large weights or poor matches, increasing variance and weakening the credibility of comparisons. Practical responses include trimming regions with minimal overlap or adjusting the weighting scheme.

2.4 Diagnostics for propensity score estimation

Diagnostics aim to verify that the estimated propensity scores support the intended balancing goal. Typical checks include:

  • Comparing covariate balance before and after adjustment
  • Evaluating propensity score distributions and overlap
  • Ensuring that the estimation procedure is stable (e.g., no pathological predictions, convergence issues for parametric models)

Good estimation is not judged solely by predictive performance; it is judged by the quality of balance and the plausibility of comparisons it enables.

3 Using Propensity Scores to Create Comparable Groups

3.1 Matching approaches

3.1.1 Nearest-neighbor matching

Matching pairs treated units with control units that have similar propensity scores. A basic approach is nearest-neighbor matching, which selects control(s) with the closest estimated score to each treated unit. Analysts may allow one-to-one or one-to-many matching and choose whether to match with or without replacement.

Nearest-neighbor methods can be intuitive and yield direct comparisons, but they may discard unmatched units and can be sensitive to the choice of distance metric and the presence of poor overlap.

3.1.2 Caliper matching and common support

Caliper matching imposes a maximum allowable difference in propensity scores between matched units. Units that cannot be matched within the caliper may be excluded. This improves comparability by preventing very dissimilar matches, often at the expense of using fewer observations.

The notion of *common support* (or *common support region*) refers to the overlap interval where treated and control scores both occur. Using calipers or restricting to common support helps avoid extrapolation beyond the data.

3.1.3 Exact and coarsened exact matching

Exact matching ensures treated and control units share identical values of selected covariates, but it can be impractical with many covariates. Coarsened exact matching relaxes this by grouping covariates into coarser bins (e.g., ranges), enabling more feasible exact comparisons. In the propensity score context, analysts may use coarsening on the propensity score itself or on key covariates.

These methods can produce strong balance when covariates are carefully selected and coarsening is thoughtfully chosen.

3.2 Stratification (subclassification)

Stratification divides the sample into subclasses (strata) based on propensity score values—commonly by quantiles such as quintiles. Within each stratum, treated and control units are compared, relying on the assumption that within-stratum balance is improved.

Stratification often reduces reliance on extreme extrapolation and provides an interpretable structure. However, coarse stratification can leave residual imbalance, particularly if propensity scores are estimated imprecisely.

3.3 Weighting approaches

3.3.1 Inverse probability of treatment weighting (IPTW)

IPTW uses propensity scores to reweight observed outcomes so that, in expectation, the weighted treated group mimics a target population with the same covariate distribution as the untreated group (and vice versa). For binary treatment, a common form is: \[ \text{weight}= \begin{cases} 1/e(X) & \text{if treated}\\ 1/(1-e(X)) & \text{if control.} \end{cases} \] IPTW can use all observations and target different estimands, but it can produce high variance when propensity scores are near 0 or 1 because weights become large.

3.3.2 Stabilized weights

Stabilized weights modify IPTW to reduce variance while preserving the balancing objective. A typical stabilization divides by an estimate of the marginal treatment probability (or otherwise normalizes weights). This can improve the stability of estimates, particularly when extreme scores occur.

3.3.3 Overlap weights and other variants

Overlap weighting is designed to emphasize comparisons in regions of good overlap. Instead of weighting treated units by \(1/e(X)\), overlap weights often scale by functions such as \(1-e(X)\) for treated and \(e(X)\) for control. The resulting weighted population concentrates on covariate patterns where both treatment and control are plausible, improving robustness and often reducing variance.

Other variants include entropy balancing and calibration-based weights, each with their own targeting and stability properties.

3.4 Covariate adjustment with propensity scores

3.4.1 Including propensity score as a covariate

Some approaches use the propensity score estimate directly in an outcome model—e.g., adding it as a predictor in regression. This can be viewed as a dimension-reduction technique, though the effectiveness depends on the correctness of the functional form relating propensity score to outcomes.

In practice, analysts often combine strategies: use matching or weighting to improve balance and then fit an outcome model for additional robustness.

3.4.2 Doubly robust frameworks (overview)

Doubly robust methods combine propensity score–based weighting (or modeling of treatment assignment) with modeling of outcomes. Under suitable conditions, consistency can be achieved if either the propensity model or the outcome model is correctly specified (or appropriately estimated), rather than requiring both.

These frameworks are frequently paired with modern estimation tools and can be tailored to various targets, though they still depend on overlap and on the validity of assumptions about confounding.

4 Assessing Balance and Model Adequacy

4.1 Standardized mean differences (SMDs)

Balance is commonly evaluated using standardized mean differences for covariates. For a covariate \(X_j\), the SMD compares the difference in means (or proportions) between treated and control groups, scaled by a measure of variability. By focusing on scale-free differences, SMDs facilitate comparison across covariates.

A typical heuristic is that SMD values below a modest threshold (often around 0.1) indicate acceptable balance, though the threshold can vary by context and covariate type.

4.2 Graphical diagnostics (e.g., propensity score distributions)

Graphs are widely used to inspect whether adjustment achieved the intended alignment. Common plots include:

  • Propensity score histograms or density overlays for treated versus control
  • Love plots (SMDs across covariates, shown before and after adjustment)
  • Cumulative distribution comparisons within strata

These visuals help identify systematic imbalance, regions lacking overlap, and covariates that remain problematic even after adjustment.

4.3 Checking balance within strata or matched sets

For stratification and matching, balance should be assessed within each matched set or stratum, not only overall. Within-stratum balance is the closest analog to the theoretical idea that conditioning on propensity score makes treated and control groups comparable.

If imbalance persists within many strata, pooled estimates can be biased even when aggregate diagnostics appear acceptable.

4.4 Sensitivity to specification choices

Results can depend on modeling and tuning choices, such as:

  • Covariate selection
  • Propensity model form (parametric vs flexible)
  • Matching distance and caliper width
  • Weighting truncation or trimming rules
  • The number of strata

Sensitivity analysis examines how estimates change under reasonable alternatives. This does not guarantee correctness, but it clarifies the degree to which conclusions rely on particular specification decisions.

5 Estimating Treatment Effects

5.1 Defining estimands (ATE, ATT, ATC)

Treatment effect estimation begins by specifying the estimand—the estimand defines the target population and comparison. Common choices include:

  • ATE (Average Treatment Effect): Average effect if everyone were treated versus if everyone were untreated.
  • ATT (Average Treatment effect on the Treated): Effect for the subgroup that actually received treatment.
  • ATC (Average Treatment effect on the Controls): Effect for those who actually received control.

Different propensity score methods target different estimands depending on the weighting or matching structure.

5.2 Estimators by method type

5.2.1 Effect estimation after matching

After matching, treatment effects can be computed as average differences in outcomes between matched treated and control units. If matching is done one-to-one or one-to-many, the estimator may take the form of an average over matched pairs or sets, possibly with weights reflecting match multiplicity.

Because matching may discard units without good matches, the resulting estimand can differ from the ATE, often corresponding to the population of units with adequate overlap.

5.2.2 Weighted outcome estimation

In weighting approaches, the estimator typically computes a weighted difference in average outcomes, where weights are derived from propensity scores. The weighting scheme determines which population is effectively represented. Variants handle treatment and control separately and may normalize weights to stabilize totals.

5.2.3 Regression adjustment after weighting

A common practice is to fit an outcome regression model after applying weights, using the weights to achieve balance and then modeling outcomes for efficiency. Regression adjustment can reduce residual variance and help accommodate nonlinear outcome relationships.

In these hybrid approaches, careful attention is still required because outcome modeling can introduce bias if functional forms are misspecified—though the degree depends on diagnostics and, in doubly robust settings, on the overall framework.

5.3 Variance estimation and standard errors

Uncertainty quantification must reflect the estimation of propensity scores and the downstream estimator. Variance estimation methods include:

Choice of variance estimator matters, particularly with complex weighting, trimming, or matching where effective sample sizes differ from nominal sample sizes.

5.4 Handling missing data in covariates (overview)

Missingness in covariates complicates propensity estimation and subsequent balancing. A common overview strategy is to use methods such as multiple imputation or model-based techniques to handle missing covariate values before estimating propensity scores, or incorporate missingness indicators when appropriate. The overarching principle is to avoid inadvertently changing the target estimand or introducing bias through inconsistent treatment of missing data.

The validity of causal conclusions under missingness depends on assumptions about why data are missing and how that information is used.

6.1 Propensity score trimming and overlap improvement

Trimming removes units with propensity scores outside a chosen range, often near 0 or 1, where overlap is poor. This can reduce bias due to extrapolation and improve numerical stability in weighting.

Overlap improvement also includes techniques that modify weighting functions to emphasize regions where treated and control units are more comparable, potentially yielding more reliable estimates when covariate patterns produce extreme propensities.

6.2 Handling high-dimensional covariates

When covariate dimension is large relative to sample size, conventional propensity models can overfit or fail to generalize. Strategies include regularization for parametric models, dimension reduction, and flexible machine learning with appropriate cross-validation and calibration.

In high-dimensional settings, diagnostics become even more important because good predictive performance does not guarantee covariate balance after adjustment.

6.3 Instrumental and generalized propensity score connections (conceptual)

Beyond standard propensity score frameworks, related concepts connect propensity score thinking to broader causal identification ideas. Instrumental variable methods introduce additional assumptions that can address unmeasured confounding under specific structures. Generalized propensity score approaches extend the basic idea to settings with different treatment types or multiple treatments.

These connections are often discussed conceptually in terms of how assumptions enable causal interpretation and how balancing ideas may need to be replaced by alternative strategies when key assumptions fail.

6.4 Extensions for time-varying treatments (overview)

When treatment can change over time and covariates may evolve, standard propensity score methods must be extended. Time-varying treatment settings often require sequential modeling of treatment probabilities at each time point and careful handling of prior treatment history and evolving confounders.

Such extensions aim to generalize “balancing” to a dynamic process, but they increase complexity, data demands, and sensitivity to modeling choices.

6.5 Doubly robust and target outcome modeling (overview)

Doubly robust methods combine propensity estimation with outcome modeling, aiming to protect against misspecification in one component. Target outcome modeling focuses on estimating outcomes directly for treated and untreated counterfactuals and then combining these with adjustment weights or imputation structures.

These approaches can improve performance and reduce reliance on a single modeling assumption, while still requiring overlap and confounding control via observed variables.

7 Practical Workflow and Reporting

7.1 Step-by-step analysis workflow

A typical workflow includes:

  1. Define the treatment, outcome, and study population.
  2. Specify the estimand (e.g., ATE or ATT).
  3. Choose baseline covariates measured before treatment.
  4. Estimate propensity scores using an appropriate model or learning approach.
  5. Assess overlap and diagnose propensity model behavior.
  6. Apply a method to create comparable groups (matching, weighting, or stratification).
  7. Evaluate covariate balance with standardized differences and graphs.
  8. Estimate treatment effects and quantify uncertainty.
  9. Perform sensitivity checks for specification choices and overlap handling.

This sequence emphasizes that causal conclusions depend jointly on the treatment model, the balancing achieved, and the quality of outcome estimation.

7.2 Common pitfalls and how to avoid them

Common issues include:

  • Assuming balance without checking covariates (diagnostics are essential).
  • Ignoring overlap and relying on extrapolation (extreme scores can drive bias and variance).
  • Including post-treatment variables in the covariate set.
  • Using a propensity model that is too rigid or too flexible without adequate validation.
  • Treating the propensity score as a magic covariate without assessing residual imbalance.

Avoidance generally requires transparent covariate selection, careful diagnostics, and sensitivity analysis.

7.3 Reproducibility and documentation

Reproducibility is supported by documenting:

  • Covariate lists and coding decisions
  • Propensity model specification or learning algorithm settings
  • Matching calipers or stratification rules
  • Trimming or overlap thresholds
  • Balance diagnostics used and their results
  • Estimands and variance estimation approach

Sharing code and data-processing steps further strengthens the ability of others to evaluate the analysis.

7.4 Reporting guidelines and transparency (conceptual)

Transparent reporting typically includes describing the causal question, the assumed confounding structure, the balancing approach, and diagnostic evidence that covariates are comparable after adjustment. Reporting effect estimates with confidence intervals and detailing how uncertainty was computed are also important. Finally, describing sensitivity analyses helps readers interpret how robust conclusions are to reasonable changes in modeling choices.