Medical AI · Clinical NLP · Lesson 3 of 4

Clinical text embeddings

Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on 2026-10-18 · Last updated 2026-10-18

Intermediate
Technical focus
About 30 minutes
Includes exercise

Before this lesson: Named-entity recognition and clinical coding, Embeddings and vector stores

In short

Text embeddings turn notes into vectors so that notes with similar meaning are close together, which lets a classifier recognise a problem described in words it was never trained on. On synthetic notes, an embedding-based classifier recognised foot problems in new wording far better than word counts did. But the same classifier gave 'denies tingling in both feet' the same score as 'reports tingling in both feet': embeddings capture the topic more than the assertion. Clinical use needs negation handling and validation on real local notes.

You will learn to

Use text embeddings as features for classifying clinical notes
Compare TF-IDF and embeddings when test notes use unseen wording
Show that embeddings can ignore negation, and why that matters
Choose between general and clinical embedding models

Embeddings as features

The RAG track used embeddings to find passages. They are also useful as input features: a classifier trained on embeddings of a few hundred labelled notes can flag, for example, notes that describe a foot problem needing review. The test below is hard on purpose: the test notes describe problems in words that never appear in the training notes.

python

import numpy as np
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score

rng = np.random.default_rng(0)
openers = ["Seen in diabetes clinic.", "Annual review.", "Follow-up visit.", "Telephone consultation."]
foot_train = ["tingling in both feet", "numbness in the toes", "foot ulcer on the heel",
              "burning pain in the feet at night", "reduced sensation on monofilament testing"]
other_train = ["cough for one week", "headache since Monday", "medication review only",
               "blurred vision when reading", "knee pain after walking"]
# Test notes describe the same problems in words the training notes never use
foot_test = ["pins and needles in her soles", "cannot feel the floor properly",
             "sore on the sole that will not heal", "feet feel like walking on cotton wool",
             "wound under the big toe"]
other_test = ["chest infection improving", "migraine episodes", "routine prescription check",
              "eyes tired with screen use", "hip stiffness in the morning"]

def make_notes(phrases_pos, phrases_neg, n):
    notes, labels = [], []
    for _ in range(n):
        label = rng.integers(0, 2)
        phrase = rng.choice(phrases_pos if label else phrases_neg)
        notes.append(f"{rng.choice(openers)} Patient reports {phrase}. HbA1c {rng.uniform(6, 10):.1f}%.")
        labels.append(label)
    return notes, np.array(labels)

train_notes, y_train = make_notes(foot_train, other_train, 200)
test_notes, y_test = make_notes(foot_test, other_test, 200)

tfidf = TfidfVectorizer().fit(train_notes)
clf = LogisticRegression(max_iter=1000).fit(tfidf.transform(train_notes), y_train)
print(f"TF-IDF + logistic regression:     test AUROC "
      f"{roc_auc_score(y_test, clf.predict_proba(tfidf.transform(test_notes))[:, 1]):.3f}")

# TF-IDF + logistic regression:     test AUROC 0.825

Synthetic notes; train on one set of phrases, test on paraphrases. TF-IDF baseline.

Embeddings recognise paraphrase

python

from sentence_transformers import SentenceTransformer

encoder = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
E_train = encoder.encode(train_notes, normalize_embeddings=True)
E_test = encoder.encode(test_notes, normalize_embeddings=True)
clf_e = LogisticRegression(max_iter=1000).fit(E_train, y_train)
print(f"Sentence embeddings + logistic regression: test AUROC "
      f"{roc_auc_score(y_test, clf_e.predict_proba(E_test)[:, 1]):.3f}")

# Sentence embeddings + logistic regression: test AUROC 1.000

Continuing from the previous block: the same classifier on sentence embeddings.

Word counts can only use words seen in training; 'pins and needles in her soles' shares almost none with 'tingling in both feet'. Embeddings place them close together, so the classifier generalises.
A perfect score on ten synthetic phrases is a demonstration, not evidence. Real notes are far more varied.

Embeddings can ignore negation

python

probes = ["Annual review. Patient reports tingling in both feet. HbA1c 7.2%.",
          "Annual review. Patient denies tingling in both feet. HbA1c 7.2%.",
          "Annual review. No numbness, no ulcers, feet examined and normal. HbA1c 7.2%."]
for text, p in zip(probes, clf_e.predict_proba(encoder.encode(probes, normalize_embeddings=True))[:, 1]):
    print(f"P(foot problem) {p:.2f}  {text}")

# P(foot problem) 0.90  Annual review. Patient reports tingling in both feet. HbA1c 7.2%.
# P(foot problem) 0.90  Annual review. Patient denies tingling in both feet. HbA1c 7.2%.
# P(foot problem) 0.76  Annual review. No numbness, no ulcers, feet examined and normal. HbA1c 7.2%.

Continuing from the previous block: three probe notes, including two that say there is no problem.

'Denies tingling' scores exactly as high as 'reports tingling', and a normal foot examination still scores 0.76. The embedding captures that the note is about feet, not whether there is a problem.
The training data contained no negated examples, so the classifier never learned the difference. Including negated notes in training, applying negation detection first, or using models trained for clinical assertion all help, and all need testing.

General or clinical embedding models?

Model type	Examples	Notes
General sentence models	all-MiniLM-L6-v2 and other sentence-transformers	Small and fast; may miss clinical abbreviations and jargon
Clinical or biomedical encoders	ClinicalBERT, BioClinicalBERT, MedCPT	Trained on clinical or biomedical text; often better on clinical language
Large embedding models from LLM providers	Commercial embedding APIs	Strong general performance; data leaves the organisation unless approved and contracted

Embeddings of notes are patient data

Research has shown that much of a text can be reconstructed from its embedding. Store and transmit embeddings of clinical notes with the same protection as the notes themselves.

Exercise

Build and stress-test embedding-based note classifiers.

Data: Synthetic notes generated in the notebook (Created for this course; no real patient data)

Reproduce the TF-IDF and embedding results and the negation probes.
Add negated training notes ('denies…', 'no…') labelled as no problem, retrain, and re-run the probes.
Apply the NegEx-style rule from the first lesson to remove negated sentences before embedding, and compare.
Optional: replace the general model with a clinical encoder (for example Bio_ClinicalBERT with mean pooling) and compare on the paraphrase test.

Open the notebook in Colab View on GitHubWorked solution coming soon

Key takeaways

Embeddings let classifiers recognise problems described in unseen words.
They capture topic more than assertion; negated notes can look positive.
Train with negated examples or detect negation first, and test both.
Protect embeddings of clinical notes like the notes themselves.

Terms used: Embedding, Negation detection

FAQ

What are clinical text embeddings?

Numerical representations of notes or sentences in which texts with similar meaning have similar vectors, used for search, clustering and as features for classifiers.

Do embeddings understand negation?

Not reliably. Many embedding models place 'has chest pain' and 'denies chest pain' close together because they share a topic. Test negation explicitly.

Is TF-IDF still useful?

Yes, as a fast, transparent baseline, especially when key terms are consistent. It struggles when the same idea is expressed in different words.

Should I use a clinical embedding model?

Often it helps with clinical abbreviations and jargon, but compare models on your own labelled notes; general models sometimes perform well.

Can embeddings identify patients?

They can leak information about the original text, so embeddings of clinical notes should be treated as sensitive patient data.

Sources

Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.