Medical AI · Evaluation, safety & deployment · Lesson 3 of 5
Reporting guidelines: TRIPOD+AI, CONSORT-AI, SPIRIT-AI, DECIDE-AI, CLAIM and more
Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on · Last updated
- Beginner
- Clinical focus
- About 25 minutes
- Includes exercise
Before this lesson: External validation and dataset shift
In short
Reporting guidelines are checklists that say what a study must report so readers can judge it. For clinical AI there is now one for almost every study type: TRIPOD+AI for prediction models, STARD-AI for diagnostic accuracy, CLAIM for imaging AI, DECIDE-AI for early clinical evaluation, SPIRIT-AI and CONSORT-AI for trials, TRIPOD-LLM for studies of language models and CHART for chatbot health-advice studies. PROBAST+AI helps readers judge risk of bias. Using them makes studies more transparent and missing information easier to spot.
You will learn to
- Match common clinical AI study types to the right reporting guideline
- Describe what TRIPOD+AI, STARD-AI, CONSORT-AI and DECIDE-AI add for AI studies
- Use PROBAST+AI to judge risk of bias in a prediction-model study
- Spot the most commonly missing information in AI papers
The main guidelines
| Guideline | Study type | Year |
|---|---|---|
| TRIPOD+AI | Developing or validating prediction models, by regression or machine learning | 2024 |
| PROBAST+AI | Appraising quality, risk of bias and applicability of prediction-model studies (a reader's tool) | 2025 |
| STARD-AI | Diagnostic accuracy studies of AI | 2025 |
| CLAIM | AI in medical imaging (checklist updated 2024) | 2020, 2024 |
| DECIDE-AI | Early-stage live clinical evaluation of AI decision support | 2022 |
| SPIRIT-AI / CONSORT-AI | Protocols and reports of randomised trials of AI interventions | 2020 |
| TRIPOD-LLM | Studies developing, evaluating or applying large language models | 2025 |
| CHART | Studies evaluating chatbot health advice (12 items, 39 sub-items) | 2025 |
| STANDING Together | Documenting and using health datasets to reduce algorithmic bias | 2025 |
Choosing the right one
Which reporting guideline fits the study?
Is the AI being tested in a randomised trial?
If Yes
SPIRIT-AI for the protocol, CONSORT-AI for the report
If No
Is it a language model or chatbot study?
If Yes
TRIPOD-LLM; CHART for chatbot health-advice studies
If No
Prediction model: TRIPOD+AI. Diagnostic accuracy: STARD-AI (plus CLAIM for imaging). Early live use: DECIDE-AI
What AI-specific items add
AI extensions keep the core items of the original guidelines and add what is specific to AI. Common additions include:
- Data: sources, dates, inclusion criteria, how missing data were handled, and the composition of the population by relevant groups.
- Model: the type of model, how it was trained and tuned, the version used, and how to access it or its code.
- Inputs and outputs: what the model receives, in what form, and how its output is presented to users.
- Evaluation: how data were split, internal and external validation, calibration and performance in subgroups.
- Human interaction: who used the AI, how, and how it changed their decisions (especially DECIDE-AI and CONSORT-AI).
- Errors and harms: analysis of failure cases and any adverse events.
Reading a paper with PROBAST+AI
PROBAST+AI helps a reader judge whether a prediction-model study's results can be trusted and applied. It covers the participants and data, the predictors, the outcome and the analysis, and asks about applicability to the reader's own setting.
| Common warning sign | Why it matters |
|---|---|
| No external validation | Performance is likely optimistic outside the development data |
| Calibration not reported | Predicted risks may be wrong even if AUROC is high |
| Few outcome events | Estimates are unstable and overfitting is likely |
| Unclear time of prediction | Possible data leakage from information recorded later |
| Data split by record, not patient | The same patient may appear in training and test sets |
| No subgroup results | Performance in important groups is unknown |
| No code, model or equation available | Others cannot validate or reproduce the work |
Reporting is not quality
- A study can follow a reporting guideline perfectly and still be poorly designed. Guidelines make problems visible; appraisal tools such as PROBAST+AI help judge them.
Exercise
Appraise a fictional clinical AI paper against the right reporting guideline and PROBAST+AI.
Data: A fictional study summary written for the notebook (Created for this course)
- Read the fictional abstract and methods, and decide which reporting guideline applies.
- Fill in the provided TRIPOD+AI checklist extract: which items are reported, partly reported or missing?
- Apply the PROBAST+AI domains and give a risk-of-bias judgement for each, with reasons.
- List the three pieces of missing information that would most change your trust in the results.
- Rewrite the study's limitations paragraph so that it is honest about what the results do and do not show.
The runnable notebook and worked solution will be published soon. You can work through the steps with the code on this page in the meantime.
Key takeaways
- There is a reporting guideline for most clinical AI study types; use the right one.
- AI items cover data, model version, inputs and outputs, validation and human interaction.
- PROBAST+AI helps readers judge risk of bias and applicability.
- Good reporting makes weaknesses visible; it does not by itself make a study good.
Terms used: Calibration, Data leakage, External validation
Related: External validation and dataset shift, Building a cohort and avoiding data leakage, Medical LLMs and their benchmarks
FAQ
What is TRIPOD+AI?
A 2024 reporting guideline for studies that develop or validate clinical prediction models using regression or machine learning, replacing the original TRIPOD statement.
Which guideline applies to a study of a medical chatbot?
TRIPOD-LLM covers studies of large language models generally; CHART focuses on studies evaluating chatbot health advice. Many journals expect one or both.
What is the difference between a reporting guideline and PROBAST+AI?
Reporting guidelines tell authors what to report. PROBAST+AI is an appraisal tool that helps readers judge the risk of bias and applicability of prediction-model studies.
Do journals require these guidelines?
Many do, often asking authors to submit a completed checklist. Requirements vary by journal; check the author instructions.
Where can I find the current versions?
The EQUATOR Network maintains a searchable library of reporting guidelines and their updates.
Sources
- Collins GS, et al. TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ 2024;385:e078378. doi:10.1136/bmj-2023-078378
- Moons KGM, et al. PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods. BMJ 2025;388:e082505. doi:10.1136/bmj-2024-082505
- Sounderajah V, et al. The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence. Nature Medicine 2025;31:3283–3289. doi:10.1038/s41591-025-03953-8
- Tejani AS, et al. Checklist for Artificial Intelligence in Medical Imaging (CLAIM): 2024 update. Radiology: Artificial Intelligence 2024;6(4):e240300. doi:10.1148/ryai.240300
- Vasey B, et al. Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI. Nature Medicine 2022;28:924–933. doi:10.1038/s41591-022-01772-9
- Liu X, et al. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension. Nature Medicine 2020;26:1364–1374. doi:10.1038/s41591-020-1034-x
- Cruz Rivera S, et al. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nature Medicine 2020;26:1351–1363. doi:10.1038/s41591-020-1037-7
- Gallifant J, et al. The TRIPOD-LLM reporting guideline for studies using large language models. Nature Medicine 2025;31:60–69. doi:10.1038/s41591-024-03425-5
- CHART Collaborative. Reporting guideline for chatbot health advice studies: the Chatbot Assessment Reporting Tool (CHART) statement. 2025
- EQUATOR Network. Library of reporting guidelines
Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.