Hospitals use a transcription tool powered by a hallucination-prone OpenAI model

3 weeks ago 17

A fewer months ago, my doc showed disconnected an AI transcription instrumentality helium utilized to grounds and summarize his diligent meetings. In my case, the summary was fine, but researchers cited by ABC News person recovered that’s not ever the lawsuit with OpenAI’s Whisper, which powers a instrumentality galore hospitals usage — sometimes it conscionable makes things up entirely.

Whisper is utilized by a institution called Nabla for a aesculapian transcription instrumentality that it estimates has transcribed 7 cardinal aesculapian conversations, according to ABC News. More than 30,000 clinicians and 40 wellness systems usage it, the outlet writes. Nabla is reportedly alert that Whisper tin hallucinate, and is “addressing the problem.”

A radical of researchers from Cornell University, the University of Washington, and others found successful a study that Whisper hallucinated successful astir 1 percent of transcriptions, making up full sentences with sometimes convulsive sentiments oregon nonsensical phrases during silences successful recordings. The researchers, who gathered audio samples from TalkBank’s AphasiaBank arsenic portion of the study, enactment soundlessness is peculiarly communal erstwhile idiosyncratic with a connection upset called aphasia is speaking.

One of the researchers, Allison Koenecke of Cornel University, posted examples similar the 1 beneath successful a thread astir the study.

The researchers recovered that hallucinations besides included invented aesculapian conditions oregon phrases you mightiness expect from a YouTube video, specified arsenic “Thank you for watching!” (OpenAI reportedly utilized to transcribe over a cardinal hours of YouTube videos to bid GPT-4.)

The survey was presented successful June astatine the Association for Computing Machinery FAccT league successful Brazil. It’s not wide if it has been peer-reviewed.

OpenAI spokesperson Taya Christianson emailed a connection to The Verge:

We instrumentality this contented earnestly and are continually moving to improve, including reducing hallucinations. For Whisper usage connected our API platform, our usage policies prohibit usage successful definite high-stakes decision-making contexts, and our model card for open-source usage includes recommendations against usage successful high-risk domains. We convey researchers for sharing their findings.

Read Entire Article