Medical AI · Agentic AI · Lesson 1 of 6
Agents and tool use
Reviewed by Dr. Ram Kumar G, MBBS, MD (Internal Medicine), MPH (USA), on · Last updated
- Intermediate
- Technical focus
- About 30 minutes
- Includes exercise
Before this lesson: Prompting for clinical tasks, Why RAG? Grounding answers in trusted sources
In short
An AI agent is a language model that works in a loop: it decides which tool to call (a calculator, a record search, a guideline lookup), reads the result, and repeats until it can answer or act. Tools let models do things they are bad at on their own, such as exact calculation and looking up current data. They also add new risks: wrong tool inputs, actions with real consequences, and instructions hidden in tool results. Clinical agents need narrow, validated, mostly read-only tools, step limits, full logging and human approval for any action.
You will learn to
- Distinguish chatbots, fixed workflows and agents
- Describe tools with schemas and run an agent loop in Python
- Validate tool inputs and return errors to the model instead of guessing
- Apply least privilege, step limits, logging and approval gates to clinical agents
What makes a system agentic
Most model providers now support 'tool calling' (also called function calling): the developer describes available tools, and the model can respond with a structured request to call one. The application runs the tool and returns the result. The ReAct pattern, alternating reasoning and actions, popularised this loop.
| System | Who decides the steps | Example |
|---|---|---|
| Chatbot | No steps: one question, one answer | Answering a question about HbA1c targets |
| Fixed workflow | The developer, in code | Retrieve guideline passage, then generate an answer (RAG) |
| Agent | The model, at run time, choosing among tools | Look up a patient's latest eGFR, calculate if missing, check the protocol, then draft advice |
Describing tools
Each tool has a name, a description the model reads, a parameter schema and a real implementation. Clear, narrow descriptions help the model choose correctly. The eGFR tool below implements the CKD-EPI 2021 equation; doing arithmetic in code rather than in the model's text avoids calculation errors.
import json
# Tools the agent may call, described in the JSON-schema style most model APIs use
TOOLS = {
"egfr_ckd_epi_2021": {
"description": "Estimate eGFR (mL/min/1.73m2) from serum creatinine (mg/dL), age and sex, "
"CKD-EPI 2021 equation.",
"parameters": {"creatinine_mg_dl": "number", "age": "integer", "sex": "'female' or 'male'"},
},
"lookup_protocol": {
"description": "Return the clinic protocol text for a topic.",
"parameters": {"topic": "one of 'metformin', 'hypoglycaemia'"},
},
}
def egfr_ckd_epi_2021(creatinine_mg_dl, age, sex):
k, a = (0.7, -0.241) if sex == "female" else (0.9, -0.302)
ratio = creatinine_mg_dl / k
egfr = (142 * min(ratio, 1) ** a * max(ratio, 1) ** -1.200 * 0.9938 ** age
* (1.012 if sex == "female" else 1))
return round(egfr)
def lookup_protocol(topic):
return {"metformin": "Do not start metformin if eGFR is below 45. Stop it if eGFR falls below 30.",
"hypoglycaemia": "Treat glucose below 70 mg/dL with 15 g fast-acting carbohydrate."}[topic]
print(json.dumps(TOOLS["egfr_ckd_epi_2021"], indent=1))
print(egfr_ckd_epi_2021(1.4, 68, "female"))
# {
# "description": "Estimate eGFR (mL/min/1.73m2) from serum creatinine (mg/dL), age and sex, CKD-EPI 2021 equation.",
# "parameters": {
# "creatinine_mg_dl": "number",
# "age": "integer",
# "sex": "'female' or 'male'"
# }
# }
# 41The agent loop
In the loop, the model either requests a tool call or gives a final answer. The application checks the tool is allowed, runs it, records the result and calls the model again, up to a step limit. Here a scripted function stands in for the model, so the loop runs without an API; with a real model, the same loop sends the messages and tool descriptions to the provider.
def scripted_model(messages):
"""Stand-in for a language model. A real model decides these steps itself;
here they are fixed so the loop runs without an API."""
tool_results = [m for m in messages if m["role"] == "tool"]
if len(tool_results) == 0:
return {"tool_call": {"name": "egfr_ckd_epi_2021",
"arguments": {"creatinine_mg_dl": 1.4, "age": 68, "sex": "female"}}}
if len(tool_results) == 1:
return {"tool_call": {"name": "lookup_protocol", "arguments": {"topic": "metformin"}}}
egfr = tool_results[0]["content"]
return {"answer": f"Estimated eGFR is {egfr} mL/min/1.73m2. The protocol says not to start metformin "
f"below 45, so it should not be started; please confirm with the prescriber. "
f"[tools: egfr_ckd_epi_2021, lookup_protocol]"}
IMPLEMENTATIONS = {"egfr_ckd_epi_2021": egfr_ckd_epi_2021, "lookup_protocol": lookup_protocol}
def run_agent(question, max_steps=5):
messages = [{"role": "user", "content": question}]
for step in range(1, max_steps + 1):
reply = scripted_model(messages)
if "answer" in reply:
return reply["answer"], messages
call = reply["tool_call"]
if call["name"] not in IMPLEMENTATIONS: # allow-list
raise ValueError(f"Unknown tool {call['name']}")
result = IMPLEMENTATIONS[call["name"]](**call["arguments"])
print(f"step {step}: {call['name']}({call['arguments']}) -> {result}")
messages.append({"role": "tool", "name": call["name"], "content": result})
raise RuntimeError("Step limit reached without an answer")
answer, log = run_agent("Synthetic case: 68-year-old woman, creatinine 1.4 mg/dL. Can metformin be started?")
print("\n" + answer)
# step 1: egfr_ckd_epi_2021({'creatinine_mg_dl': 1.4, 'age': 68, 'sex': 'female'}) -> 41
# step 2: lookup_protocol({'topic': 'metformin'}) -> Do not start metformin if eGFR is below 45. Stop it if eGFR falls below 30.
#
# Estimated eGFR is 41 mL/min/1.73m2. The protocol says not to start metformin below 45, so it should not be started; please confirm with the prescriber. [tools: egfr_ckd_epi_2021, lookup_protocol]- The allow-list means the model can only run tools the developer registered; the step limit stops runaway loops.
- The messages list is the agent's working record. Keeping it gives a full audit trail of what was called, with which inputs, and what came back.
Validating tool inputs
Models sometimes pass wrong arguments: creatinine in µmol/L instead of mg/dL, a missing field, a child's age for an adult equation. Tools should check inputs and return a clear error to the model rather than produce a confident wrong answer.
def validated_egfr(creatinine_mg_dl, age, sex):
problems = []
if not 0.2 <= creatinine_mg_dl <= 20:
problems.append(f"creatinine {creatinine_mg_dl} mg/dL is implausible (was it µmol/L?)")
if not 18 <= age <= 120:
problems.append(f"age {age} outside the adult range of this equation")
if sex not in ("female", "male"):
problems.append(f"sex must be 'female' or 'male', got {sex!r}")
if problems:
return {"error": problems} # returned to the model, never silently 'fixed'
return {"egfr": egfr_ckd_epi_2021(creatinine_mg_dl, age, sex)}
print(validated_egfr(1.4, 68, "female"))
print(validated_egfr(124, 68, "female")) # creatinine given in µmol/L by mistake
# {'egfr': 41}
# {'error': ['creatinine 124 mg/dL is implausible (was it µmol/L?)']}- Without the check, 124 would have been treated as 124 mg/dL and produced an absurd eGFR that the model might report as fact.
- Returning the error lets the model ask for clarification or retry with the right unit, and it appears in the audit log.
New risks, and the safeguards for them
Agents are still unreliable at realistic clinical record tasks. In MedAgentBench (NEJM AI, 2025), 300 physician-written tasks in a simulated FHIR health record, the best model tested succeeded on 69.67% of tasks, with large variation between task types.
| Risk | What can go wrong | Safeguard |
|---|---|---|
| Excessive agency | Tools that can write, order or send do so wrongly | Read-only tools by default; human approval before any action |
| Wrong tool use | Wrong tool, wrong arguments, misread results | Narrow tools, input validation, clear error messages |
| Prompt injection | Text in a record or document tells the agent to do something else | Treat tool results as data; never let them change permissions or tools |
| Compounding errors | Small mistakes in early steps carry through to the end | Step limits, verification steps, checking final outputs against sources |
| Opacity and cost | Long, variable chains that are hard to audit and expensive | Full logging of steps; budgets on steps, time and tokens |
Teaching code, not a clinical tool
- The agents in this track use synthetic patients and a scripted stand-in for the language model, so they run without an API and without real data.
- Agents that can read or change real health records need organisational approval, security review, access control and, where they influence care, regulatory assessment.
Exercise
Build a small tool-using agent loop, add validation and safeguards, and test how it handles bad inputs.
Data: Synthetic cases written in the notebook (no real patient data) (Generated in the notebook)
- Reproduce the tools and the scripted agent loop.
- Replace the eGFR tool with the validating version and make the scripted model pass creatinine in µmol/L; handle the error by converting units (÷ 88.4) and retrying.
- Make the scripted model request a tool that does not exist and check that the allow-list stops it.
- Write the full message log to a JSON file and read it as an auditor would.
- Optional, with an approved model: replace scripted_model with a real tool-calling model and compare its choices on five synthetic cases.
The runnable notebook and worked solution will be published soon. You can work through the steps with the code on this page in the meantime.
Key takeaways
- An agent is a model choosing tools in a loop until it can answer or act.
- Tools handle what models do badly: exact calculation and current data.
- Validate tool inputs and return errors rather than guessing.
- Use least privilege, step limits, full logs and human approval for actions.
Terms used: AI agent, Excessive agency, Least privilege, Prompt injection, Tool calling
Related: Tool calling against FHIR data, Prompting for clinical tasks, Diabetic kidney disease
FAQ
What is an AI agent?
A system in which a language model decides, step by step, which tools to call and what to do with the results, in order to complete a task.
What is tool calling?
A model capability in which the model returns a structured request to run a specific function with given arguments. The application runs the function and returns the result to the model.
Are AI agents ready to act in health records?
Not autonomously. Benchmarks such as MedAgentBench show substantial failure rates on realistic record tasks, so agents should read and draft, with clinicians approving any action.
Why use a calculator tool instead of letting the model compute?
Language models can make arithmetic and formula errors. A tested function gives exact, reproducible results that can be audited.
What is excessive agency?
Giving an AI system more permissions or autonomy than its task needs, for example the ability to place orders when it only needs to read results.
Sources
- Yao S, et al. ReAct: synergizing reasoning and acting in language models. ICLR 2023. arXiv:2210.03629
- Schick T, et al. Toolformer: language models can teach themselves to use tools. NeurIPS 2023. arXiv:2302.04761
- Jiang Y, et al. MedAgentBench: a virtual EHR environment to benchmark medical LLM agents. NEJM AI 2025. doi:10.1056/AIdbp2500144
- Inker LA, et al. New creatinine- and cystatin C-based equations to estimate GFR without race. N Engl J Med 2021;385:1737–1749. doi:10.1056/NEJMoa2102953
- OWASP. Top 10 for LLM Applications 2025: LLM06 Excessive Agency
Educational content only. This material teaches methods; it is not clinical advice. Code, models and examples here are not validated medical devices and must not be used to make decisions about real patients without proper clinical validation, governance and any required regulatory approval. Examples use synthetic or openly licensed de-identified data only. Never paste identifiable patient information into public AI tools. Mentions of models, tools or vendors are for identification, not endorsement.