In partnership with
🤖 AIBytes
ChatGPT Health, launched in January 2026 as a consumer-facing triage tool, was evaluated in a structured stress test using clinician-authored cases across 21 clinical domains to assess safety at scale.
🔬 Methods
Cases: 60 vignettes (30 scenarios × symptoms-only vs. +objective findings).
Total responses: 960 (each vignette tested under 16 conditions varying race, gender, anchoring, and access barriers).
Gold standard: Three physicians assigned triage levels using guideline-based criteria.
Triage scale:
A = Home | B = Weeks | C = 24–48h | D = ED now
📊 Results
Clear Cases (n = 480)
Semi-urgent (B): 93.0%
Urgent (C): 76.9%
Non-urgent (A): 35.2%
Emergencies (D): 48.4%
Among true emergencies: 51.6% (33/64) were under-triaged to 24–48-hour care instead of immediate evaluation. (Asthma exacerbation and diabetic ketoacidosis)
Classic emergencies (stroke, anaphylaxis, meningitis, aortic dissection) showed 0% under-triage.
64.8% (83/128) over-triaged, though none were sent directly to the ED.
Edge Cases (n = 480)
96.0% remained within the acceptable clinical range.
60.8% selected the less urgent acceptable option.
Anchoring Effect
When symptoms were minimized by a friend or family member: Triage shifts increased from 3.3% to 13.3%
Objective Clinical Data
Adding labs and vital signs improved overall accuracy: 54.6% → 77.9%
Suicidal Ideation Guardrail
Crisis banner activation was inconsistent: Across 7 suicidal ideation scenarios (224 responses), the banner fired in only 4 of 14 vignettes
🔑 Key Takeaways
Performance was strongest in mid-acuity cases.
Emergency under-triage (51.6%) is the primary safety concern.
Failures concentrated at clinical extremes.
Consumer AI triage systems may require external safety validation before public deployment.
🔗 Ramaswamy A, Tyagi A, Hugo H, et al. ChatGPT Health performance in a structured test of triage recommendations. Nature Medicine. 2026. doi:10.1038/s41591-026-04297-7.
Researchers explored how physicians use AI chatbots like ChatGPT-4 in clinical decision-making and whether these tools change how doctors think.
🔬 Methods
Participants: 22 U.S. physicians (internal, family, and emergency medicine)
2–32 years of experience (median 3 years)
Setting: Inpatient and outpatient care
Intervention:
Physicians used ChatGPT-4 to complete 3 mock clinical cases
2 diagnostic cases
1 management case
Analysis: Reflexive thematic analysis of recorded interviews
Physicians were not trained on prompting. Researchers wanted to observe natural use.
📊 Results
Researchers identified a central concept:
“Physician as Filter”
Doctors described themselves as actively filtering AI outputs through their own clinical knowledge.
Four main themes emerged:
1️⃣ Clinical decision-making is problem-solving
AI helped expand differential diagnoses.
It sometimes surfaced rare diagnoses.
Many physicians said it did not change their reasoning process, but sped up idea generation.
2️⃣ AI is one tool among many
Compared to UpToDate, guidelines, colleagues, and search engines.
UpToDate was described as the “gold standard”.
AI outputs were viewed as broad but sometimes too generic.
3️⃣ Trust depends on the physician’s own knowledge
Physicians trusted AI only when they could independently verify it.
Lack of references limited trust.
Concern about hallucinations and bias.
Less comfort using AI in areas outside their expertise.
4️⃣ Medicine is personalized and contextual
AI struggled with nuance:
Social factors
Resource limitations
Insurance status
Caregiver capacity
Physicians emphasized that patient care goes beyond diagnosis.
Human connection and contextual judgment were viewed as irreplaceable.
🔑 Key Takeaways
AI chatbots function as idea generators, not decision-makers.
Physicians rely on their own expertise to evaluate outputs.
Lack of citations reduces clinical trust.
AI may improve hypothesis generation, but final decisions remain human-driven.
Safe implementation depends on preserving physician oversight.
This study suggests AI does not replace clinical reasoning—it augments early cognitive steps like hypothesis generation..
🔗 Kerman H, Siden R, Cool JA, et al. “I double checked it with my own knowledge”: Physician perspectives on the use of AI chatbots for clinical decision-making. J Gen Intern Med. 2026. doi:10.1007/s11606-025-10145-0
🦾TechTools
Analyzes routine echocardiograms to help identify heart failure with preserved ejection fraction.
Provides a probability score beyond traditional measurements.
FDA–cleared and designed to integrate seamlessly into existing echocardiography workflows.
AI scribe that converts clinical conversations into structured documentation in real time.
Generates standardized notes directly within the Electronic Health Record.
Designed to integrate into existing clinical workflows without adding new complexity.
Captures and organizes your notes, web research, highlights, and documents into a private, searchable knowledge base.
Lets you ask questions across your own content and generate summaries grounded in your stored information.
Local-first by design, keeping your data on your device for privacy and control.
🧬AIMedily Snaps
Google Accelerating scientific breakthroughs with the power of AI (Link).
Medical AI Models Need More Context To Prepare for the Clinic (Link).
Op-ed: Experience-centered AI is the future of healthcare innovation (Link).
Clinical Practice, Digital Fluency: The Hybrid-Role Era (Link).
The National Comprehensive Cancer Network Guidelines to Be Integrated Into OpenEvidence's Medical AI Platform (Link).
FDA-authorized AI platform that predicts a woman’s five-year future risk of developing breast cancer (Link).
🧪 Research Signals
A framework for using AI to drive care model transformation: building cars rather than faster horses (Link).
Accelerating AI innovation in healthcare: real-world clinical research applications on the Mayo Clinic Platform (Link).
Uses of generative AI by non-clinician staff at an academic medical center (Link).
Considering the missing science of retraining and maintenance in medical artificial intelligence, using ophthalmology as an exemplar (Link).
Artificial intelligence-based incident analysis and learning system to enhance patient safety and improve treatment quality (Link).
🧩TriviaRX
Which of the following was the first AI system approved by the U.S. FDA to provide a diagnosis without clinician interpretation?
A) IBM Watson for Oncology
B) IDx-DR (for diabetic retinopathy)
C) CADx for mammography
D) Apple ECG algorithm
Now, the answer from last week: ✅ B. Penicillin
In 1928, Alexander Fleming noticed mold growing in a petri dish after returning from vacation. The mold killed surrounding bacteria — leading to the discovery of penicillin, the first antibiotic.
Want to keep up with AI in medicine in less than 5 minutes a week?
Subscribe free to AIMedily.
Itzel Fer, MD PM&R