Medical AI · Foundations · Lesson 2 of 6

Healthcare data types: records, images, signals, notes and omics

Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on 2026-10-18 · Last updated 2026-10-18

Beginner
Clinical focus
About 25 minutes
Includes exercise

Before this lesson: The healthcare-AI landscape: what is in use, what works, and what to ask

In short

Healthcare data comes in five broad types: structured records (diagnoses, medicines, lab results), medical images, physiological signals such as ECGs, free-text clinical notes, and omics data such as genomes. Each needs different methods, and all of them reflect how care was delivered and documented, not just the patient's biology. Knowing how a dataset was produced is the first step in any AI project.

You will learn to

Describe the five main healthcare data types and the formats they are stored in
Explain why EHR data are irregular, incomplete and shaped by care processes
Turn a long-format event table into one row per patient with pandas
Identify the questions to ask about how any healthcare dataset was produced

Five types of healthcare data

Most healthcare AI projects use one or more of the data types below. The type largely decides the modelling approach: tabular methods for structured records, convolutional or transformer networks for images, sequence models for signals, and language models for text.

Type	Examples	Common formats	Typical AI methods
Structured records	Diagnoses, medicines, lab results, vital signs, demographics	EHR database tables, FHIR resources, claims files	Logistic regression, gradient-boosted trees, survival models
Medical images	X-ray, CT, MRI, ultrasound, retinal photographs, pathology slides	DICOM; whole-slide image formats	Convolutional and transformer networks
Physiological signals	ECG, EEG, continuous glucose monitoring, ICU monitor waveforms	Waveform files (e.g. WFDB), device exports	Sequence models, 1-D convolutional networks
Clinical text	Clinic letters, discharge summaries, radiology and pathology reports	Free text in the EHR, PDFs, scanned documents	NLP, named-entity recognition, language models
Omics	Genome sequences, gene expression, proteomics	FASTQ, VCF and other specialist formats	Specialised statistical and deep-learning methods

Structured EHR data: an event log, not a spreadsheet

Electronic health records usually store data as events: one row per lab result, prescription or diagnosis, each with a time. Patients are measured at irregular intervals, and how often someone is tested often says as much about their care as their health: sicker or more closely followed patients have more data.

The code below creates a small synthetic lab table in this 'long' format.

python

import numpy as np
import pandas as pd

rng = np.random.default_rng(7)

rows = []
for patient_id in range(1, 6):
    n_tests = rng.integers(2, 6)
    days = np.sort(rng.choice(365, size=n_tests, replace=False))
    for day in days:
        test = rng.choice(["hba1c", "glucose", "creatinine"], p=[0.4, 0.4, 0.2])
        value = {"hba1c": rng.normal(7.5, 1.2), "glucose": rng.normal(150, 35),
                 "creatinine": rng.normal(1.0, 0.25)}[test]
        rows.append({"patient_id": patient_id, "day": int(day),
                     "test": test, "value": round(value, 1)})

labs = pd.DataFrame(rows)
print(labs.head(8))
print(f"\n{len(labs)} measurements for {labs.patient_id.nunique()} patients")

#    patient_id  day        test  value
# 0           1  210  creatinine    0.9
# 1           1  225     glucose  162.5
# 2           1  247     glucose  174.3
# 3           1  283     glucose  104.9
# 4           1  325       hba1c    6.0
# 5           2  187  creatinine    0.6
# 6           2  354       hba1c    7.7
# 7           3    1       hba1c    7.6
#
# 20 measurements for 5 patients

Synthetic lab results for five made-up patients, one row per measurement.

From events to one row per patient

Most classical ML models need one row per patient (or per admission) with a fixed set of columns. Getting there means decisions: which time window, which summary (latest, mean, maximum), and what to do when a test was never done. Each choice changes the model.

python

latest = (labs.sort_values("day")
              .groupby(["patient_id", "test"])["value"].last()
              .unstack("test"))
print(latest)
print("\nMissing values per test:")
print(latest.isna().sum())

# test        creatinine  glucose  hba1c
# patient_id
# 1                  0.9    104.9    6.0
# 2                  0.6      NaN    7.7
# 3                  1.4    220.0    7.6
# 4                  NaN      NaN    5.1
# 5                  1.0    186.8    6.0
#
# Missing values per test:
# creatinine    1
# glucose       2
# hba1c         0

Continuing from the previous block: the latest value of each test per patient.

Even with five synthetic patients, some tests were never done. In real records, a missing test is rarely random: it may mean the clinician saw no reason to order it.
'Latest value' ignores when it was measured. Patient 3's HbA1c came from day 1, almost a year before the end of the window. Real pipelines record the measurement's age as well as its value.
The 'Missing data and class imbalance' lesson in the Machine learning track covers how to handle these gaps.

Images, signals and text

Medical images are stored and exchanged in DICOM, a standard that bundles pixel data with metadata such as the scanner, acquisition settings and (unless removed) patient identifiers. Models can learn scanner- or hospital-specific features from images, so a model trained at one site may partly be detecting the site.

Physiological signals are high-frequency time series. A single 10-second, 12-lead ECG at 500 samples per second has 60,000 data points. Signal quality, device type and placement all affect what a model sees.

Clinical text holds much of the information that never reaches structured fields: symptoms, reasoning, social context. It is also full of abbreviations, negation ('no chest pain'), copy-forward text and identifiers, which is why the Clinical NLP track starts with how notes are written.

Data reflect care, not just biology

Every healthcare dataset is produced by people and systems: who came to hospital, which tests were ordered, how conditions were coded for billing, which devices were used. Models learn these patterns along with the biology.

Effect	What happens	Example
Informative missingness	Whether a test was done carries information	Lactate is measured mainly when sepsis is already suspected
Coding practice	Diagnosis codes reflect billing and local habits	A condition may be coded only when it changes payment
Selection	Only people who reach care appear in the data	Hospital datasets under-represent people with poor access to care
Measurement differences	Devices and labs differ	Two analysers or units for the same test; pulse oximeters that read less accurately on darker skin

Questions to ask about any dataset

Who is included, and who is missing?
How and why was each variable recorded, and by whom?
Over what period, at which sites, with which equipment?
How was the outcome defined, and could the definition itself be biased?

Where to find data for learning

Exercises in this course use only synthetic or openly licensed de-identified data. Synthea, an open-source simulator, generates realistic but entirely synthetic patient records, including FHIR output. PhysioNet hosts many physiological and clinical datasets; some are open, while others, such as the full MIMIC-IV database, require credentialed access and a data use agreement, which must be respected.

Never use real patient data for learning exercises

Do not copy data from your hospital's systems into notebooks, cloud drives or AI tools to practise. Use synthetic data, or open datasets under their licence terms.

Exercise

Generate synthetic patient records with Synthea (or use its sample data), load them with pandas, and build a one-row-per-patient table with explicit choices about time windows and missing values.

Data: Synthea synthetic patient records (CSV export) (Synthea is Apache 2.0; generated records are synthetic)

Load the Synthea patients.csv and observations.csv files (the notebook downloads a small sample).
Filter observations to HbA1c, glucose and creatinine using their LOINC codes.
For each patient, compute the latest value and its age in days at a chosen index date.
Count how many patients have each test missing and compare missingness by age group.
Write down three ways the resulting table could mislead a model.

The runnable notebook and worked solution will be published soon. You can work through the steps with the code on this page in the meantime.

Key takeaways

Healthcare data come as structured records, images, signals, text and omics, each with its own formats and methods.
EHR data are irregular event logs; turning them into model features involves choices that change results.
Missing data and coding practices carry information about care, not only about patients.
Learn with synthetic or properly licensed open data, never real patient records.

Terms used: De-identification, DICOM, EHR (electronic health record), PHI (protected health information)

FAQ

What is the most common type of data used in healthcare AI?

Imaging dominates regulator-authorised devices, while structured EHR data are the most common basis for risk-prediction models built inside health systems.

What is DICOM?

DICOM (Digital Imaging and Communications in Medicine) is the international standard for storing and exchanging medical images together with metadata about the patient, study and equipment.

Why is EHR data hard to use for machine learning?

Measurements are irregular and often missing, codes reflect billing and local habits, and the data are shaped by who receives care. All of these must be handled explicitly when building features.

Is synthetic data good enough to learn with?

Yes, for learning methods and building pipelines. Synthetic data like Synthea's does not capture every real-world quirk, so models must still be developed and validated on appropriate real data under proper governance.

Can I use MIMIC-IV for the exercises?

The full MIMIC-IV database requires credentialed access through PhysioNet, including training and a data use agreement. Exercises here use open data, and link to MIMIC with instructions where relevant.

Sources

Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.