Wait, what? Two groups can contain thousands of people and still be fundamentally unfair to compare.

If people choose, receive or qualify for different treatments because they are already different before the treatment begins, a simple comparison of outcomes mixes the treatment effect with those pre-existing differences. Propensity-score methods were developed to make the comparison more like the comparison we wish randomisation had created—without pretending that observational data has magically become a randomised trial.

The one-sentence answer

A propensity score summarises the observed characteristics associated with receiving a treatment, so researchers can match, stratify or weight observations to create more comparable treatment groups before estimating a causal effect.

Quick Read

• The propensity score is the probability of treatment assignment conditional on observed baseline covariates.
• Its main job is not prediction. Its main job is balance.
• Matching pairs or groups people with similar propensity scores.
• Inverse probability weighting gives more weight to observations that are less typical of the treatment they actually received.
• No propensity method can balance an important confounder that was never measured.
• Poor overlap creates unstable comparisons, extreme weights and dependence on extrapolation.
• The estimand must be chosen first: the average treatment effect for whom?
• Balance diagnostics should be examined after adjustment; a high treatment-prediction AUC is not proof of a good causal design.

Why raw observational comparisons can fail

Imagine evaluating an optional after-school tutoring programme. Students who join may be more motivated, may have more involved parents, may be struggling more, may live closer to the centre, or may have different prior achievement. If the participating students later outperform the non-participants, the programme might have helped—but the raw difference alone cannot tell us how much.

The causal question is counterfactual: what would have happened to the same kinds of students under the alternative treatment? We cannot observe both potential outcomes for the same person at the same time. Causal inference therefore tries to construct a credible comparison group.

The balancing-score idea

Suppose treatment assignment depends on ten baseline variables. Matching exactly on every combination can become impossible, especially when variables are continuous. Rosenbaum and Rubin showed that under key assumptions, the multidimensional covariate information can be summarised by a balancing score: the propensity score.

For an individual with baseline characteristics X, the propensity score is commonly written as:

e(X) = P(T = 1 | X)

It is the probability of receiving treatment, given the observed pre-treatment covariates included in the assignment model. If treated and untreated observations with similar scores are compared, the distributions of those measured covariates can become much more similar.

The score is not the destination

A common mistake is to fit a sophisticated treatment-prediction model, celebrate its accuracy and stop. But a propensity model is not usually judged by how impressively it predicts treatment. If treatment is almost perfectly predictable, there may be very little overlap between groups—which is exactly the situation that makes causal comparison difficult.

The operational question is: after using the score, are the groups balanced on relevant baseline covariates?

Four major ways to use a propensity score

Once estimated, propensity scores can support several designs. They are related, but they do not create identical target populations or identical statistical behaviour.

• Matching: find treated and untreated observations with similar scores.
• Stratification: divide observations into score bands and compare treatment groups within those bands.
• Weighting: reweight observations so the weighted covariate distribution represents a chosen target population.
• Covariate adjustment: include the propensity score in an outcome model, although this approach can rely more heavily on modelling assumptions than design-focused matching or weighting.

Matching: compare people who could plausibly have gone either way

In nearest-neighbour matching, a treated observation is paired with one or more untreated observations whose propensity scores are close. A caliper can prevent poor matches by refusing pairs that are too far apart. Matching can be done with or without replacement, one-to-one or one-to-many, and with additional exact matching constraints.

The attraction is intuitive: instead of fitting an outcome regression across very different populations, first build a comparison set in which the treatment groups resemble each other on observed baseline characteristics.

But matching can change the population being studied. If many observations have no plausible counterpart, they may be discarded. That may improve internal comparability while narrowing generalisability.

Inverse probability weighting: construct a pseudo-population

Inverse probability of treatment weighting, often abbreviated IPTW or IPW, uses the propensity score to reweight observations. For an average treatment effect, a treated observation may receive a weight proportional to 1/e(X), while an untreated observation receives a weight proportional to 1/[1−e(X)].

The intuition is beautiful. Someone who received a treatment that was very likely for a person with their characteristics contributes less new information about the comparison. Someone whose treatment was relatively unusual for their characteristics represents more people like them who could have appeared in the opposite group, so their observation receives more weight.

If the assumptions hold and the weighting is well behaved, the weighted pseudo-population can make baseline covariates independent of treatment in a way analogous to the balance produced by randomisation.

ATE, ATT and other estimands: effect for whom?

Before choosing weights, the researcher must define the estimand. The average treatment effect (ATE) asks about the average effect in the target population. The average treatment effect on the treated (ATT) asks what the effect was for the population that actually received treatment. Other weighting schemes can target overlap populations or other policy-relevant groups.

This is not merely a technical detail. Two well-executed analyses can produce different answers because they are answering different population questions.

Positivity and overlap: the hidden geometry of causal comparison

For each relevant combination of covariates, there must be a meaningful chance of receiving each treatment under comparison. This is the positivity or overlap requirement.

If every student with a particular profile always receives tutoring and no comparable student remains untreated, the data cannot directly reveal the untreated counterfactual for that profile. A model may still output a number, but it is extrapolating beyond observed support.

In weighting, near-zero or near-one propensity scores create very large inverse weights. One or two observations can then dominate the estimate. Researchers may use stabilised weights, truncation, trimming or overlap weights, but each choice changes the effective target and should be reported.

Balance diagnostics come before outcome excitement

After matching or weighting, baseline balance should be inspected. Standardised mean differences are widely used because they compare covariate differences on a scale less sensitive to sample size than ordinary hypothesis tests. Distribution plots, variance ratios and balance across important interactions or nonlinear transformations can add further evidence.

The purpose is not to obtain a ceremonial table with many green ticks. It is to test whether the design has actually reduced observable differences relevant to the treatment assignment and outcome.

What belongs in the propensity model?

Variables are chosen for causal reasons, not merely because an automated selection algorithm found statistical associations. Important pre-treatment causes of the outcome and treatment generally deserve careful consideration. Variables measured after treatment can be dangerous because they may be mediators or consequences of treatment. Conditioning on colliders can open biasing paths rather than close them.

This is why propensity-score analysis belongs inside a causal design, often informed by subject-matter knowledge and causal diagrams, rather than being treated as a standalone machine-learning exercise.

Unmeasured confounding remains unmeasured

Propensity scores balance what has been measured and appropriately modelled. They cannot automatically balance motivation, socioeconomic advantage, disease severity, institutional selection or any other confounder that is absent or badly measured.

This is the boundary that protects the method from being oversold: excellent balance on observed covariates does not prove that treatment assignment is effectively random. Sensitivity analysis, negative controls, alternative designs and domain knowledge remain important.

Doubly robust estimation: two models, one additional layer of protection

A doubly robust estimator combines a treatment-assignment model, such as the propensity score, with an outcome model. Under standard conditions, some doubly robust estimators remain consistent if either the treatment model or the outcome model is correctly specified—not necessarily both.

That is useful, but the phrase “doubly robust” is easy to misunderstand. It does not mean invulnerable. It does not repair violations of consistency, positivity or unmeasured-confounding assumptions. It does not guarantee valid confidence intervals after careless model selection. And if both nuisance models are badly wrong, the protection disappears.

A worked example: optional tutoring

Suppose 10,000 students are observed. Some enroll in a tutoring programme and others do not. Baseline data include prior scores, attendance, school, age, study time and family-reported education resources. A propensity model estimates each student’s probability of enrolling based only on pre-programme information.

Before adjustment, tutored students have much lower prior Mathematics scores but higher study time. After weighting, those baseline distributions become similar. The outcome comparison is now less confounded by those measured factors. But if parental motivation strongly influenced both enrolment and later performance and was not measured, residual confounding remains possible.

The correct conclusion is therefore not “weighting proved tutoring caused the gain”. It is “under the stated causal assumptions, using this design and these measured covariates, the weighted comparison estimates the specified treatment effect.” That sentence is less dramatic and much more scientific.

Common failure modes

• Prediction-score obsession: optimising treatment prediction instead of covariate balance.
• Outcome peeking: repeatedly modifying the design after seeing which choice produces the preferred treatment effect.
• Post-treatment adjustment: including variables affected by treatment in the propensity model.
• Extreme-weight blindness: allowing a tiny number of observations to dominate the estimate.
• No overlap check: reporting an effect in regions where one treatment group is essentially absent.
• Balance by p-value: using significance tests that depend heavily on sample size instead of meaningful balance diagnostics.
• One estimand, another interpretation: estimating an ATT but discussing it as though it were an ATE for everyone.
• Observed-balance triumphalism: assuming balanced measured covariates eliminate unmeasured confounding.

How this connects across the eduKate Library

This article is a specialised bridge inside the broader causal-inference owner. Read it with How Causal Inference Works, How Observational Studies Work, How Quasi-Experimental Designs Work, How Sensitivity Analysis and Robustness Checks Work, and How Regression Models Work.

Authoritative source corridor

• Austin (2011), An Introduction to Propensity Score Methods for Reducing the Effects of Confounding in Observational Studies.
• Chatton & Rohrer (2024), The Causal Cookbook: Propensity Scores, G-Computation and Doubly Robust Standardization.
• Review of propensity-score methods for clustered observational data.
• 2026 systematic review of propensity-score method performance in observational studies.

The deeper idea

Propensity-score methods are not machines for turning correlation into causation. Their deeper contribution is design discipline. They force us to ask whether the treated and untreated groups are actually comparable, where that comparison is supported by data, which population the effect belongs to, and which assumptions carry the causal claim.

The goal is not to make observational data look random. It is to make the comparison as explicit, balanced and auditable as the available evidence allows.