Ontario audit finds AI notetakers in doctor’s offices fabricate details
▼ Summary
– AI medical scribes recommended by the Ontario government were found to generate incorrect, incomplete, and hallucinated information that could harm patient health outcomes.
– An audit reviewed transcription tests of two simulated patient-doctor conversations across 20 approved AI scribe vendors, all of which showed accuracy or completeness issues.
– Nine vendors hallucinated patient information, 12 recorded information incorrectly, and 17 missed key details about discussed mental health issues.
– Examples of mistakes included hallucinated referrals for blood tests or therapy, incorrect transcription of prescription medication names, and missed mental health details.
In recent years, an increasing number of overworked physicians have adopted AI medical scribes to streamline the documentation of patient conversations, diagnoses, and care plans into electronic health records. However, a recent audit from the Ontario auditor general reveals that these tools, which were endorsed by the provincial government, frequently produce inaccurate, incomplete, or fabricated details. This could potentially lead to inadequate or harmful treatment plans, according to the report, directly threatening patient health outcomes.
The audit, published in a report on the Use of Artificial Intelligence in the Ontario Government, examined transcription tests from two simulated patient-doctor dialogues. These tests were run across 20 AI scribe vendors that had been pre-qualified and approved by the province for purchase by healthcare providers. Every single vendor demonstrated some flaw in accuracy or completeness during at least one of these basic scenarios. Specifically, nine of the systems hallucinated patient information, 12 recorded incorrect data, and 17 omitted critical details regarding mental health issues discussed in the conversations.
The auditor general highlights several alarming examples of errors that could directly compromise a patient’s subsequent care. In one instance, an AI scribe fabricated nonexistent referrals for blood tests or therapy. Other cases involved the incorrect transcription of prescription medication names. Most concerning, many systems missed key details about mental health conditions that were explicitly raised during the simulated visits, creating a significant risk of misdiagnosis or inappropriate follow-up.
(Source: Ars Technica)
