← Back to Insights
May 24, 2023
David Burda
Consumerism Outcomes System Dynamics

3 Things We Lose When AI-Powered Chatbots Answer Our Medical Questions

I’ve never used a dating app — not even to research a story — but based on ads for dating apps that I see on TV, hear on the radio and read about on social media, dating apps have a chronic problem: People lie.

There’s a significant difference between how people describe themselves on the apps and who they are in person. Five-foot-eight in person becomes six-foot tall online. Fifty-four in real life becomes 44 on the app. Single on the app can mean married for five years, that sort of thing.

What about doctors and how well they interact with patients? Is there a difference between how they interact with patients in person and online? And what if that online interaction isn’t with a real doctor but with an AI-powered chatbot?

I thought about that recently after all the media coverage of a new study published in JAMA Internal Medicine last month.

Researchers from the University of California, San Diego, Bryn Mawr College, Johns Hopkins University and the U.S. Navy wanted to know who answered a patient’s question the best: a real doctor or an AI-powered chatbot. To answer their question, the researchers looked at 195 online exchanges between real doctors and patients. The researchers then posed those same patient questions to an AI-powered chatbot. A team of evaluators comprised of licensed healthcare professionals in pediatrics, geriatrics, internal medicine, oncology, infectious disease and preventive medicine blindly compared the answers from the real doctors with the answers from the chatbot.

Here’s what the researchers found:

  • 78.6% of the evaluators rated the chatbot’s responses better than the doctors’ responses.
  • Evaluators said the chatbot’s responses were of higher quality (4.1 to 3.3 on a 5-point scale).
  • Evaluators said the chatbot’s responses were more empathetic (3.7 to 2.1 on a 5-point scale).

“These results suggest that artificial intelligence assistants may be able to aid (physicians) in drafting responses to patient questions,” the researchers said.

If that happens — doctors start using robots to answer patient questions — what do patients lose? I think patients lose a lot.

  • First, all responses would sound the same regardless of who the doctor is, where they practice, how old the doctor is, etc. Patients lose insights into an individual doctor’s personality — polite or gruff, kind or mean, knowledgeable or incompetent, patient or impatient.
  • Second, all responses would be the same to the same question. They would answer the same way without variation or going off script. Standardization, not customization. Patients lose the ability to ask detailed, specific questions that pertain to their individual medical situations.
  • Third, patients lose the ability to interact with a physician. No interruptions. No back-and-forth. No follow-up questions. It’s a sterile, impersonal exchange.

Think of your experience now as a consumer when you confront an electronic phone tree that sends you on an endless chase to find the right department or a website chatbot that asks scripted questions and replies with preprogrammed answers. It’s one thing when you want to know why your new toaster isn’t working. It’s another when you have a fever and feel swelling under your jaw.

I also think it says something about a medical practice that would prefer you talk to a machine rather than a real person. Tell your most intimate medical secrets to this computer first. Thank you.

Come to think of it, why didn’t the researchers ask consumers to judge and compare the responses of real doctors versus the AI-powered chatbot? I honestly don’t care what a geriatrician evaluator thinks.

To learn more on this topic, please read “Healthcare’s Failure to Communicate” on 4sighthealth.com.

Thanks for reading.

About the Author

David Burda

Dave Burda began covering healthcare in 1983 and hasn’t stopped since. Dave writes this monthly column “Burda on Healthcare,” contributes weekly blog posts, manages our weekly newsletter 4sight Friday, and hosts our weekly Roundup podcast. Dave believes that healthcare is a business like any other business, and customers — patients — are king. If you do what’s right for patients, good business results will follow.

Dave’s personal experiences with the healthcare system both as a patient and family caregiver have shaped his point of view. It’s also been shaped by covering the industry for 35 years as a reporter and editor. He worked at Modern Healthcare for 25 years, the last 11 as editor.

Prior to Modern Healthcare, he did stints at the American Medical Record Association (now AHIMA) and the American Hospital Association. After Modern Healthcare, he wrote a monthly column for Twin Cities Business explaining healthcare trends to a business audience, and he developed and executed content marketing plans for leading healthcare corporations as the editorial director for healthcare strategies at MSP Communications.

When he’s not reading and writing about healthcare, Dave spends his time riding the trails of DuPage County, IL, on his bike, tending his vegetable garden and daydreaming about being a lobster fisherman in Maine. He lives in Wheaton, IL, with his lovely wife of 35 years and his three children, none of whom want to be journalists or lobster fishermen.


Recent Posts

Podcast: Income, Poverty, the Uninsured and Healthcare 9/21/23
What do the new Census Bureau figures on these three societal measures say about health policy and the… Read More
By September 21, 2023
A Sign of Faith in Healthcare Consumerism
If you follow this blog, you know I’m always looking for signs that people are becoming good healthcare… Read More
By September 20, 2023
Diagnostic Determinism: How Precision Diagnostics Will Reinvent Medicine
In the movie “Star Trek IV: The Voyage Home,” the spaceship Enterprise travels back in time to San… Read More
By September 5, 2023