Ontario auditor general finds problems with medical AI transcription tool

TORONTO — New artificial intelligence transcription services for doctors are at risk of providing inaccurate information and outright hallucinations, the province’s auditor general concluded in a special report released Tuesday.

Shelley Spence said the so-called “AI scribe” software was not evaluated adequately before implementation.

“We found gaps in how AI scribe systems used to generate medical notes were evaluated by Supply Ontario, including accuracy issues and missing security and privacy documentation,” Spence said.

Ontario’s doctors have long asked for help dealing with the administrative burden placed on them and the AI scribe system has been touted to do just that. It is a recording and transcription tool designed to save time.

The province has approved 20 vendors that are providing the transcription software to doctors. Of those, Spence found that 11 did not submit any third-party audit reports while five others did not submit threat risk assessments and privacy impact assessments as part of the request-for-bids process.

“These vendors were still approved,” Spence wrote in the report.

“A comprehensive evaluation of vendors was not conducted to ensure that their AI Scribe systems mitigated the risk of bias.”

But Minister of Public and Business Service Delivery and Procurement Stephen Crawford took issue with Spence’s conclusions and said medical practitioners will always review the notes before a decision is made.

He said the software remains in the testing phase and that hallucinations and other issues have only arisen during that phase.

“That’s not actually in operational use with doctors, that’s in the optional stage where we’re reviewing the various scribes,” he said.

“Having said that, what I can say is that the doctors that are using the AI scribes today are saving five to seven hours a week on average.”

Spence said she visited her own doctor recently, who was using an AI scribe during her visit.

“So I kind of mentioned, ‘please look at the transcript when you’re done with my own visit,’ but it is in use,” Spence said.

The auditor general also found broader issues within the Ontario Public Service’s use of AI.

She said the vast majority of public servants do not use the accepted, secure AI chat bot, Microsoft Co-Pilot.

“We found that when staff used generative AI, they used Co-Pilot six per cent of the time,” Spence said.

Her probe found the ministry has not blocked public servants’ access to unsafe and unsecured AI-powered websites.

There are also no guardrails in place to ensure public servants do not upload Ontarians’ personal information or sensitive corporate data on those websites.

Crawford again took issue with Spence’s conclusions, saying only the approved AI chat bot is in use across the 55,000 public servants.

“Co-Pilot is the product the public servants use,” he said.

This report by The Canadian Press was first published May 12, 2026.

Liam Casey, The Canadian Press