Skip to content
Dr. Ram KumarDiabetes & metabolic health

Medical AI · Machine learning · Lesson 1 of 9

Prediction or causation? Framing the clinical question

Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on · Last updated

  • Beginner
  • Clinical focus
  • About 25 minutes
  • Includes exercise

Before this lesson: Diagnostic accuracy metrics: sensitivity, specificity, PPV, ROC and calibration

In short

A prediction model answers 'who is likely to have this outcome?'; a causal question asks 'what would happen if we did something different?'. Machine learning is very good at the first and, on its own, cannot answer the second, because associations in routine data are distorted by who received which treatment and why. Decide which question you are asking before choosing data, methods or how the output will be used.

You will learn to

  • Classify a clinical data question as description, prediction or causal inference
  • Explain how confounding by indication makes treatments look harmful in routine data
  • Recognise why the coefficients or feature importances of a prediction model are not treatment effects
  • Choose an appropriate design, such as a trial or target trial emulation, for a causal question

Three kinds of data question

Hernán and colleagues describe three tasks in data science: description, prediction and causal inference. They need different data, different methods and different kinds of evidence, and confusing them is one of the commonest errors in clinical machine learning.

Based on Hernán, Hsu and Healy (CHANCE, 2019).
TaskQuestionClinical exampleTypical methods
DescriptionWhat is happening?What proportion of adults with diabetes have an HbA1c above 8%?Summary statistics, visualisation, clustering
PredictionWho is likely to have the outcome?Which patients are at high risk of readmission within 30 days?Regression, tree ensembles, neural networks
Causal inferenceWhat would happen if we intervened?Would starting an SGLT2 inhibitor reduce this patient's risk of kidney failure?Randomised trials; target trial emulation with confounding adjustment

Prediction does not need causes

A good predictor need not be a cause. Grey hair predicts heart attacks because both increase with age; a hospital's ordering of a specific test predicts a diagnosis because clinicians order it when they already suspect that diagnosis. For triage or risk stratification, such predictors can be perfectly useful, as long as the model is used only to estimate risk.

The problem starts when a prediction model's output is used to decide on an action, and the action changes the very patterns the model learned from.

A classic warning: asthma and pneumonia

  • In a well-known pneumonia dataset, models learned that patients with asthma had a lower risk of death. The reason was that asthmatic patients were treated more aggressively, often in intensive care, which lowered their risk.
  • A model used to decide who could safely be sent home would have sent home a high-risk group. The association was real; acting on it would have been harmful (Caruana et al., KDD 2015).

Confounding by indication, in code

Clinicians give treatments to the patients who need them most. In routine data, this makes effective treatments look harmful, because the treated patients were sicker to begin with. The simulation below creates 20,000 synthetic patients in which the treatment truly lowers the risk of death.

python
import numpy as np
import pandas as pd
from sklearn.linear_model import LogisticRegression

rng = np.random.default_rng(0)
n = 20000
severity = rng.normal(0, 1, n)                               # how ill each patient is
treated = rng.binomial(1, 1 / (1 + np.exp(-2 * severity)))   # sicker patients treated more
logit = -1 + 1.5 * severity - 0.8 * treated                  # treatment truly LOWERS risk
died = rng.binomial(1, 1 / (1 + np.exp(-logit)))
df = pd.DataFrame({"severity": severity, "treated": treated, "died": died})

print(df.groupby("treated").died.mean().round(3))

# treated
# 0    0.184
# 1    0.325
Sicker patients are more likely to be treated; the treatment truly reduces risk.
  • Treated patients die more often (32.5% vs 18.4%), even though the simulated treatment is protective. A naive reading of the data would conclude the opposite of the truth.

Adjusting for the confounder

If the reason for treatment is measured, it can be adjusted for. Here severity is the only confounder and is perfectly recorded, which is never true in practice.

python
naive = LogisticRegression(penalty=None).fit(df[["treated"]], df.died)
adjusted = LogisticRegression(penalty=None).fit(df[["treated", "severity"]], df.died)
print(f"Treatment coefficient, unadjusted: {naive.coef_[0][0]:+.2f}")
print(f"Treatment coefficient, adjusted for severity: {adjusted.coef_[0][0]:+.2f}")

# Treatment coefficient, unadjusted: +0.76
# Treatment coefficient, adjusted for severity: -0.87
Continuing from the previous block: compare the treatment coefficient with and without adjustment.
  • Adjustment recovers a protective effect close to the true value of −0.8 used in the simulation.
  • In real data, the reasons for treatment are many, partly unrecorded (frailty, patient preference, the clinician's judgement) and sometimes affected by earlier treatment. Adjustment is only as good as the measured confounders, which is why causal questions need a deliberate design.

Why model coefficients are not treatment effects

It is tempting to read a prediction model's coefficients, odds ratios or feature importances as 'what matters' and to act on them. Each coefficient is adjusted for every other variable in the model, so for some variables it may estimate a direct effect, for others a confounded association, and for others nothing meaningful at all. Westreich and Greenland called the habit of interpreting all of them as effects the 'Table 2 fallacy'.

Explainability methods such as SHAP describe how the model uses its inputs. They do not describe what would happen to a patient if an input changed.

Designs for causal questions

Randomised controlled trials remain the most reliable way to estimate a treatment effect. When a trial is not feasible, target trial emulation designs an observational analysis to mimic the trial you would have run: explicit eligibility criteria, a clear time zero, defined treatment strategies, follow-up and an analysis plan that adjusts for confounding.

Which kind of question is this?

  1. Will the output be used to choose an action, such as a treatment?

    • If No, only to estimate risk

      1. Prediction: focus on discrimination, calibration and validation

    • If Yes

      1. Is there trial evidence that the action helps patients like these?

        • If Yes

          1. Predict risk to target a proven action, then evaluate the combination

        • If No

          1. Causal question: needs a trial or a target trial emulation

A simplified framing aid for learning purposes.

Exercise

Explore confounding by indication in a simulated cohort, and see when adjustment does and does not recover the true treatment effect.

Data: Synthetic patients generated in the notebook (no real patient data) (Generated in the notebook)

  1. Run the simulation and reproduce the unadjusted and adjusted treatment coefficients.
  2. Add an unmeasured confounder (for example frailty) that affects both treatment and death, and refit the adjusted model without it. How far is the estimate from −0.8?
  3. Train a gradient-boosted model to predict death using both treatment and severity, and inspect its feature importances. Do they tell you the treatment works?
  4. Write the target trial protocol (eligibility, time zero, strategies, outcome, follow-up) for estimating this treatment's effect from routine data.

The runnable notebook and worked solution will be published soon. You can work through the steps with the code on this page in the meantime.

Key takeaways

  • Description, prediction and causal inference are different tasks with different methods.
  • Predictors need not be causes; that is fine only while the model just estimates risk.
  • Confounding by indication can make effective treatments look harmful in routine data.
  • Model coefficients and feature importances are not treatment effects.

Terms used: Confounding, Logistic regression, SHAP, Target trial emulation

Related: Diagnostic accuracy metrics, Building a cohort and avoiding data leakage, The healthcare-AI landscape

FAQ

Can machine learning find causes in observational data?

Not by itself. Machine learning can be used inside causal methods, for example to model who receives treatment, but the causal conclusions depend on the study design and on assumptions about confounding, not on the algorithm.

Is it wrong to include treatments as predictors in a risk model?

Not for estimating risk, as long as the model is used in a setting where treatment patterns stay the same. It becomes a problem if the model is used to decide on those treatments.

What is confounding by indication?

A distortion that arises because treatments are given to patients with stronger reasons for them, often sicker patients. Their worse outcomes can make a treatment look harmful when it is not.

What is target trial emulation?

An approach to observational research that designs the analysis to mimic a specific randomised trial, with explicit eligibility, time zero, treatment strategies and follow-up, to reduce common biases.

Do SHAP values show what causes the outcome?

No. SHAP values explain how a model uses its inputs to make a prediction. They describe the model, not the effect of changing a patient's characteristics.

Sources

Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.