New systems for documenting outpatient visits are adding features and moving into hospitals; ‘we are just scratching the surface’
Excerpt – “We are just scratching the surface of what this technology can do,” says Dr. Lance Owens, regional chief medical information officer at University of Michigan Health, which uses Microsoft’s DAX Copilot ambient-listening technology. “I see it being able to provide insights about the patient that the human mind just can’t do in a reasonable time.” By connecting older data with new information in the medical record, for instance, the technology could help make sure that an incidental finding years ago was followed up on.
Ambient-listening software runs on devices from desktops to mobile phones and tablets, using speech recognition and AI language models to capture and process conversations between a clinician and a patient during a visit. The AI, from companies including Microsoft, Ambience Healthcare, Abridge, Onpoint Healthcare Partners and Nabla, can recognize pertinent medical dialogue, distinguish voices and filter out casual chitchat, such as talk about the weather or sports.
By the time a doctor’s appointment is done, an AI scribe can generate a comprehensive note for the electronic medical record, create a concise after-visit summary for the patient and provide data for coding and billing purposes.
Researchers predict the systems will evolve into a 360-degree presence that extends before and after the medical visit: analyzing records before an appointment to identify red flags, prompting doctors about recommended tests and treatments based on patient symptoms, and teeing up follow-up actions like lab tests and prescription orders. Ambience Healthcare is developing an AI agent to help patients manage their care outside the clinic, calling or texting with updates, medication reminders and follow-up scheduling.
New features are on the way, including diagnostic aid tools that suggest conditions and more sophisticated multilingual capabilities. Microsoft is working with Canary Speech on applying its technology to analyze vocal features for early signs of conditions like anxiety and mild cognitive impairment.
Research has shown that the technology has already helped save time and reduce burnout for doctors, who often cobble together their summaries after each visit or at the end of the day, working from handwritten or computer notes or dictation devices. It has also unshackled them from staring into a computer or scribbling notes rather than being present with the patient. [..]
While promising, ambient-listening technologies raise privacy and security concerns in an industry already plagued by data breaches that compromise patient information. The technology doesn’t currently require regulatory approval. That could change if capabilities such as diagnosis were added in the future.
Legal and bioethics experts warn that healthcare providers should plan around potential risks and ask patients for permission before using ambient listening in appointments. Cost is also a concern—users typically license or subscribe to the technology, and fees can range from $200 to $600 per doctor per month. It isn’t yet clear how the technology would affect healthcare costs in the longer term. [..]
Microsoft is issuing an enhanced version of the technology, called Dragon Copilot, and recently signed an agreement with data company Press Ganey to help analyze the content and tone of patient-doctor conversations alongside patient-experience survey data. The aim is to proactively address patient concerns and coach doctors in having difficult conversations with patients and showing more empathy.
The goal is to help doctors, not replace their judgment, says Joe Petro, vice president of health and life sciences for Microsoft. “This is a co-pilot, not a pilot or an autopilot, and the physician is always in command of the situation.”
Full article, L Landro, Wall Street Journal, 2025.5.27