← Back to Insights
May 24, 2023
David Burda
Consumerism Outcomes System Dynamics

3 Things We Lose When AI-Powered Chatbots Answer Our Medical Questions

I’ve never used a dating app — not even to research a story — but based on ads for dating apps that I see on TV, hear on the radio and read about on social media, dating apps have a chronic problem: People lie.

There’s a significant difference between how people describe themselves on the apps and who they are in person. Five-foot-eight in person becomes six-foot tall online. Fifty-four in real life becomes 44 on the app. Single on the app can mean married for five years, that sort of thing.

What about doctors and how well they interact with patients? Is there a difference between how they interact with patients in person and online? And what if that online interaction isn’t with a real doctor but with an AI-powered chatbot?

I thought about that recently after all the media coverage of a new study published in JAMA Internal Medicine last month.

Researchers from the University of California, San Diego, Bryn Mawr College, Johns Hopkins University and the U.S. Navy wanted to know who answered a patient’s question the best: a real doctor or an AI-powered chatbot. To answer their question, the researchers looked at 195 online exchanges between real doctors and patients. The researchers then posed those same patient questions to an AI-powered chatbot. A team of evaluators comprised of licensed healthcare professionals in pediatrics, geriatrics, internal medicine, oncology, infectious disease and preventive medicine blindly compared the answers from the real doctors with the answers from the chatbot.

Here’s what the researchers found:

  • 78.6% of the evaluators rated the chatbot’s responses better than the doctors’ responses.
  • Evaluators said the chatbot’s responses were of higher quality (4.1 to 3.3 on a 5-point scale).
  • Evaluators said the chatbot’s responses were more empathetic (3.7 to 2.1 on a 5-point scale).

“These results suggest that artificial intelligence assistants may be able to aid (physicians) in drafting responses to patient questions,” the researchers said.

If that happens — doctors start using robots to answer patient questions — what do patients lose? I think patients lose a lot.

  • First, all responses would sound the same regardless of who the doctor is, where they practice, how old the doctor is, etc. Patients lose insights into an individual doctor’s personality — polite or gruff, kind or mean, knowledgeable or incompetent, patient or impatient.
  • Second, all responses would be the same to the same question. They would answer the same way without variation or going off script. Standardization, not customization. Patients lose the ability to ask detailed, specific questions that pertain to their individual medical situations.
  • Third, patients lose the ability to interact with a physician. No interruptions. No back-and-forth. No follow-up questions. It’s a sterile, impersonal exchange.

Think of your experience now as a consumer when you confront an electronic phone tree that sends you on an endless chase to find the right department or a website chatbot that asks scripted questions and replies with preprogrammed answers. It’s one thing when you want to know why your new toaster isn’t working. It’s another when you have a fever and feel swelling under your jaw.

I also think it says something about a medical practice that would prefer you talk to a machine rather than a real person. Tell your most intimate medical secrets to this computer first. Thank you.

Come to think of it, why didn’t the researchers ask consumers to judge and compare the responses of real doctors versus the AI-powered chatbot? I honestly don’t care what a geriatrician evaluator thinks.

To learn more on this topic, please read “Healthcare’s Failure to Communicate” on 4sighthealth.com.

Thanks for reading.

About the Author

David Burda

David Burda began covering healthcare in 1983 and hasn’t stopped since. Dave writes this monthly column “Burda on Healthcare,” contributes weekly blog posts, manages our weekly newsletter 4sight Friday, and hosts our weekly Roundup podcast. Dave believes that healthcare is a business like any other business, and customers — patients — are king. If you do what’s right for patients, good business results will follow.

Dave’s personnel experiences with the healthcare system both as a patient and family caregiver have shaped his point of view. It’s also been shaped by covering the industry for 40 years as a reporter and editor. He worked at Modern Healthcare for 25 years, the last 11 as editor.

Prior to Modern Healthcare, he did stints at the American Medical Record Association (now AHIMA) and the American Hospital Association. After Modern Healthcare, he wrote a monthly column for Twin Cities Business explaining healthcare trends to a business audience, and he developed and executed content marketing plans for leading healthcare corporations as the editorial director for healthcare strategies at MSP Communications.

When he’s not reading and writing about healthcare, Dave spends his time riding the trails of DuPage County, IL, on his bike, tending his vegetable garden and daydreaming about being a lobster fisherman in Maine. He lives in Wheaton, IL, with his lovely wife of 40 years and his three children, none of whom want to be journalists or lobster fishermen.

Recent Posts

Has the Retail Health Boom Gone Bust?
My local Target store used to have a CVS MinuteClinic in it. Target has since repurposed that retail… Read More
By April 10, 2024
Burda on Healthcare: Why Your Digital Front Door Needs a Side Analog Entrance
After the World Wide Web became a thing in the mid-1990s, people called me a technology luddite. I… Read More
By April 9, 2024
Spy vs. Spy? More Like Medicare vs. Medicare Advantage
If you follow my columns, blog posts and podcasts on 4sighthealth.com, you know that I’m not much for… Read More
By March 6, 2024