TL;DR
An Ontario audit of 20 AI note-taking systems used in healthcare found that most systems frequently made errors, including fabricating information and missing key details. The findings highlight significant concerns about AI accuracy in critical medical documentation.
The Ontario Office of the Auditor General has reported that 20 AI-powered medical note-taking systems approved for use in the province routinely produce inaccurate and sometimes fabricated patient records, raising concerns about safety and reliability.
The audit evaluated 20 AI Scribe systems used by healthcare providers across Ontario, using simulated doctor-patient recordings. It found that nine of these systems fabricated information, such as suggesting treatments or symptoms not discussed during consultations. Additionally, 12 systems inserted incorrect medication details into patient notes, and 17 missed critical mental health information discussed during recordings. Six systems fully or partially overlooked key mental health issues.
OntarioMD, an organization supporting healthcare technology adoption, recommended that physicians manually review AI-generated notes for accuracy. However, the report notes that none of the approved systems include mandatory attestation features to verify note correctness, increasing risks of errors going unchecked.
Why It Matters
The findings raise serious questions about the safety and reliability of AI tools used in medical documentation, which could impact patient care and safety. Errors such as fabricated treatment suggestions or incorrect medication records could lead to misdiagnoses or inappropriate treatments. The report also highlights regulatory and evaluation shortcomings that may have allowed subpar systems to be approved, potentially exposing patients to harm and undermining trust in AI-assisted healthcare.
As an affiliate, we earn on qualifying purchases.
Background
The use of AI for medical note-taking has been expanding in Ontario as part of broader efforts to improve healthcare efficiency. The AI Scribe program was launched to assist clinicians by automating documentation tasks, with more than 5,000 physicians participating. Prior to this, concerns about AI accuracy in healthcare have been raised internationally, but this is the first comprehensive audit in Ontario to evaluate the actual performance of approved systems in simulated settings.
“Nine out of 20 AI systems fabricated information or made suggestions to treatment plans not discussed in the recordings.”
— Office of the Auditor General of Ontario
“Physicians are advised to manually review AI-generated notes for accuracy, but current systems lack mandatory verification features.”
— OntarioMD
What Remains Unclear
It is still unclear how widespread these inaccuracies are in real-world clinical settings, as the audit was based on simulated recordings. The Ontario Ministry of Health has not reported any known patient harms linked to the AI Scribe systems so far, and the full impact of these errors remains to be assessed.
What’s Next
The Ontario government and health authorities are expected to review the audit findings and consider revising evaluation criteria for AI tools. Further investigations into actual clinical impacts and possible regulatory adjustments are likely, alongside efforts to improve system accuracy and safety features.
Key Questions
Are AI note-taking systems safe to use in Ontario healthcare?
The recent audit indicates significant accuracy issues, suggesting caution. Physicians are advised to review AI notes manually, and further safety evaluations are expected.
What specific errors did the audit find in the AI systems?
The audit found that nine systems fabricated information, 12 inserted incorrect drug details, and 17 missed key mental health issues discussed during recordings.
Has the Ontario government responded to these findings?
The Ministry of Health has not yet issued a detailed response but stated that more than 5,000 physicians are participating in the program, with no reported patient harms so far.
Will these AI systems be replaced or improved?
It is likely that evaluation criteria will be revised and systems improved, but specific plans have not been publicly announced yet.
