Twelve questions to ask before adopting an AI tool in a diabetes clinic

Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on 2026-10-18 · Last updated 2026-10-18

In short

Before adopting an AI tool in diabetes care, ask what exactly it is for and whether that use is authorised, what evidence shows it works in patients like yours, how it performs across groups and equipment, what data it uses and where that data goes, who checks its outputs, how it will be monitored, and how you would stop using it. A tool that cannot answer these questions clearly is not ready, however impressive its accuracy figures.

Purpose and authorisation

1. What exactly is the intended use? Which task, which patients, which users, which setting? Write it in one sentence before looking at accuracy figures.
2. Is it a medical device for that use, and is it authorised where you work? A tool built for wellness or documentation may not be authorised to guide treatment.

Evidence

3. Has it been validated outside the developer's own data, in a population like yours? External validation is where many models lose performance.
4. Are calibration and performance at the actual threshold reported, not just AUROC? A risk tool can rank patients well while overstating or understating risk.
5. How does it perform across groups and equipment: age, sex, ethnicity, skin tone, language, camera or device type? Ask for subgroup results.
6. Is there evidence that using it improves decisions or outcomes, not just that it is accurate? For patient-facing tools, has it been tested with real users?

Tool type	Evidence to look for
Retinopathy screening	Sensitivity and specificity in your screening population and with your cameras; ungradable rate; referral workload
Risk prediction (kidney, cardiovascular)	External validation, calibration, comparison with established equations such as the KFRE
CGM analytics and alerts	Lows predicted in advance, false alerts per day, performance across devices
Documentation and ambient scribes	Error rates on clinician review, time saved in your setting, consent process

Data and privacy

7. What data does it use, where is it processed and stored, for how long, and is it used to train the vendor's models? These belong in the contract.
8. Does it meet the data protection law that applies to you, such as the GDPR, HIPAA or India's DPDP Act, and is a data protection impact assessment done?

Oversight, monitoring and exit

9. Who reviews its outputs before they affect care, and is that review realistic in clinic time? Watch for automation bias.
10. How will performance be monitored after go-live: input drift, referral or alert rates, error reports, and audits against clinician judgement?
11. Who owns it locally, and how are incidents reported to the organisation and the vendor?
12. How do you switch it off safely, and what happens to your data and workflows if you stop using it?

A useful rule of thumb

If a vendor can show strong accuracy figures but cannot answer questions about intended use, local validation, subgroup performance, data handling and monitoring, the tool is not yet ready for patient care.

FAQ

What is the most important question to ask about a clinical AI tool?

What exactly it is for. The intended use decides whether it is a medical device, what evidence is needed and how it should be monitored.

Is a high AUROC enough to adopt a tool?

No. It shows ranking ability in the test data, not calibration, performance at your threshold, performance in your population, or benefit to patients.

Should small clinics use AI tools at all?

They can, with the same questions answered proportionately. For lower-risk uses such as drafting letters for review, the bar is lower than for tools that guide diagnosis or treatment.

Who should be involved in the decision?

Clinicians who will use the tool, someone responsible for data protection and information governance, and, for tools that affect care, a clinical safety lead.

Where can I learn more about evaluating AI tools?

The Evaluation, safety and deployment track and the Regulation track of this course cover each question in detail.

Sources

Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.