Medical AI · Machine learning · Lesson 9 of 9

Unsupervised learning for patient phenotypes

Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on 2026-10-18 · Last updated 2026-10-18

Intermediate
Technical focus
About 30 minutes
Includes exercise

Before this lesson: A logistic-regression risk score

In short

Unsupervised learning looks for structure in data without an outcome label, for example grouping patients into phenotypes. Clustering of six clinical variables has suggested subgroups of adult-onset diabetes in Swedish and Indian populations. But clustering algorithms always return clusters, even from pure noise, so any proposed phenotype must be shown to be stable, reproducible in other populations and more useful than simple clinical measures.

You will learn to

Explain the difference between supervised and unsupervised learning
Cluster standardised clinical variables with k-means and choose k with silhouette scores
Show that clustering finds 'groups' in data that has none, and test stability
Critically appraise data-driven disease subtypes, using diabetes as an example

Learning without an outcome

Every model so far has been supervised: it learned to predict a known outcome. Unsupervised learning has no outcome. It looks for structure, such as groups of similar patients (clustering) or a smaller number of underlying dimensions (dimensionality reduction, for example principal component analysis).

In medicine, clustering is used to propose phenotypes: subgroups of a disease that may differ in cause, course or treatment response.

A diabetes example from the literature

In 2018 Ahlqvist and colleagues clustered adults with newly diagnosed diabetes in Sweden using six variables: GAD antibodies, age at diagnosis, BMI, HbA1c, and HOMA2 estimates of beta-cell function and insulin resistance. They described five clusters, from severe autoimmune and severe insulin-deficient diabetes to mild age-related diabetes, with different rates of complications.

In India, the INSPIRED study applied a similar approach to 19,084 people with type 2 diabetes and found four clusters, two of which had not been described in the Swedish data: combined insulin-resistant and deficient diabetes (CIRDD) and insulin-resistant obese diabetes (IROD). Subgroups found in one population may not transfer to another.

SAID severe autoimmune; SIDD severe insulin-deficient; SIRD severe insulin-resistant; MOD mild obesity-related; MARD mild age-related diabetes.
Study	Population	Clusters reported
Ahlqvist et al., 2018	Newly diagnosed adult-onset diabetes, Sweden	SAID, SIDD, SIRD, MOD, MARD
Anjana et al. (INSPIRED), 2020	Type 2 diabetes, India	SIDD, MARD, CIRDD, IROD

Clustering synthetic patients

The simulation creates three synthetic groups loosely inspired by those profiles. Because we built the groups, we can check whether clustering recovers them.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(17)
centres = {
    "deficient": dict(age_dx=45, bmi=25, hba1c=10.5, homa_b=40, homa_ir=2.0),
    "resistant": dict(age_dx=50, bmi=34, hba1c=7.5, homa_b=120, homa_ir=5.5),
    "age_related": dict(age_dx=66, bmi=27, hba1c=7.0, homa_b=85, homa_ir=2.5),
}
spread = dict(age_dx=8, bmi=3, hba1c=1.0, homa_b=20, homa_ir=0.9)
frames = [pd.DataFrame({k: rng.normal(v, spread[k], 600) for k, v in c.items()})
          .assign(true_group=name) for name, c in centres.items()]
df = pd.concat(frames, ignore_index=True)
variables = list(spread)
print(df.groupby("true_group")[variables].mean().round(1))

#              age_dx   bmi  hba1c  homa_b  homa_ir
# true_group
# age_related    65.9  26.9    7.0    84.1      2.5
# deficient      44.6  25.2   10.5    38.9      2.0
# resistant      50.4  34.1    7.6   121.2      5.5

Three synthetic diabetes profiles with overlapping distributions.

Choosing the number of clusters

k-means needs the number of clusters, k, in advance. Variables are standardised first so that a variable measured in large units, like HOMA-B, does not dominate. The silhouette score (from −1 to 1) measures how much closer each patient is to their own cluster than to the next one.

python

from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.preprocessing import StandardScaler

X = StandardScaler().fit_transform(df[variables])
for k in range(2, 7):
    labels = KMeans(n_clusters=k, n_init=10, random_state=0).fit_predict(X)
    print(f"k = {k}: silhouette {silhouette_score(X, labels):.2f}")

# k = 2: silhouette 0.39
# k = 3: silhouette 0.45
# k = 4: silhouette 0.34
# k = 5: silhouette 0.25
# k = 6: silhouette 0.16

Continuing from the previous block: silhouette scores for k = 2 to 6.

The silhouette score peaks at k = 3, the true number of groups. With real data, the peak is often less clear, and clinical judgement plays a large role.

Checking against the truth

python

from sklearn.metrics import adjusted_rand_score

labels = KMeans(n_clusters=3, n_init=10, random_state=0).fit_predict(X)
print(pd.crosstab(df.true_group, labels, colnames=["cluster"]))
print(f"\nAgreement with true groups (adjusted Rand index) "
      f"{adjusted_rand_score(df.true_group, labels):.2f}")

# cluster        0    1    2
# true_group
# age_related    4  590    6
# deficient      2    1  597
# resistant    596    4    0
#
# Agreement with true groups (adjusted Rand index) 0.97

Continuing from the previous block: compare k = 3 clusters with the true groups.

When real, well-separated groups exist, k-means recovers them almost perfectly (adjusted Rand index 0.97; 1.0 is perfect agreement).
Real patients rarely fall into such separate groups. Diabetes, for example, may be better described as continuous variation across several dimensions than as distinct types.

Clustering always finds clusters

The danger of clustering is that it never says 'there are no groups here'. The example below clusters pure random noise.

python

import numpy as np
from sklearn.cluster import KMeans
from sklearn.metrics import adjusted_rand_score, silhouette_score

rng = np.random.default_rng(3)
noise = rng.normal(size=(1800, 5))                 # no groups at all
labels = KMeans(n_clusters=4, n_init=10, random_state=0).fit_predict(noise)
print(f"Cluster sizes on pure noise: {np.bincount(labels)}")
print(f"Silhouette {silhouette_score(noise, labels):.2f}")

# Stability: refit on two random halves and compare assignments on the full data
a, b = (KMeans(n_clusters=4, n_init=10, random_state=s)
        .fit(noise[rng.choice(1800, 900, replace=False)]) for s in (1, 2))
print(f"Agreement between two refits (adjusted Rand index) "
      f"{adjusted_rand_score(a.predict(noise), b.predict(noise)):.2f}")

# Cluster sizes on pure noise: [454 446 445 455]
# Silhouette 0.13
# Agreement between two refits (adjusted Rand index) 0.20

k-means on data with no structure, and a stability check.

k-means split pure noise into four neat, equally sized 'phenotypes'. Each would have its own average profile and could be given a clinical-sounding name.
The warning signs are a low silhouette score and poor stability: refitting on different halves of the data assigns patients very differently (adjusted Rand index 0.20).

Is a phenotype useful?

A proposed subtype is worth using only if it adds something. Dennis and colleagues, using clinical-trial data, found that simple continuous clinical features such as age at diagnosis and kidney function predicted diabetes progression and treatment response better than the cluster labels.

Stability: do clusters reappear with different samples, random seeds and algorithms?
Reproducibility: do they appear in other populations, measured in other ways?
Added value: do they predict outcomes or treatment response better than the variables used to make them?
Practicality: can a patient be assigned to a cluster with tests available in routine care?

Subtypes are not yet treatment rules

Data-driven diabetes subgroups are a research tool. Treatment decisions for people with diabetes follow established guidelines and individual assessment, not cluster membership.

Exercise

Cluster synthetic diabetes profiles, test whether the clusters are real and stable, and compare them with continuous variables for predicting an outcome.

Data: Synthetic profiles generated in the notebook (no real patient data) (Generated in the notebook)

Reproduce the silhouette scores and the comparison with the true groups.
Make the groups overlap more (increase the spreads) and watch how silhouette and agreement change.
Run a bootstrap stability analysis: refit k-means on 50 resamples and compute the average adjusted Rand index.
Try Gaussian mixture models and hierarchical clustering; do they agree with k-means?
Simulate a complication outcome that depends on HbA1c and HOMA-IR, and compare a model using cluster labels with one using the continuous variables.

The runnable notebook and worked solution will be published soon. You can work through the steps with the code on this page in the meantime.

Key takeaways

Clustering finds groups without an outcome, and always returns clusters, even from noise.
Standardise variables, choose k with care, and test stability before trusting clusters.
Data-driven diabetes subgroups differ between populations, such as Sweden and India.
A phenotype is useful only if it adds value over the variables used to create it.

Terms used: k-means clustering, Unsupervised learning

FAQ

What is the difference between supervised and unsupervised learning?

Supervised learning predicts a known outcome from inputs. Unsupervised learning has no outcome and looks for structure, such as groups of similar patients.

How many types of diabetes are there?

Clinically, the main categories remain type 1, type 2, gestational and other specific types such as LADA and monogenic diabetes. Data-driven clusters such as the Swedish five-cluster and Indian four-cluster models are research proposals, not replacements for these categories.

How do I choose the number of clusters?

Use measures such as silhouette scores or information criteria, test stability across resamples, and consider clinical interpretability. There is often no single correct answer.

Why do I need to standardise variables before clustering?

Distance-based methods like k-means are dominated by variables with large numeric ranges. Standardising puts all variables on a comparable scale.

Can clustering be used for patient care?

Not on its own. Clusters need to be shown to be stable, reproducible and more useful than existing measures, and then tested prospectively, before they inform care.

Sources

Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.