AI and doctors offer different strengths in virtual emergency treatment

AI and doctors offer different strengths in virtual emergency treatment

Will doctors or artificial intelligence (AI) provide better treatment recommendations for patients seen in a virtual emergency care setting? A new Cedars-Sinai study shows that doctors and AI models have different strengths.

The late study, presented at the American College of Physicians Internal Medicine meeting and published simultaneously in theAnnals of Internal Medicinecompared the initial AI treatment recommendations with the final recommendations from physicians who had access to the AI ​​recommendations but may or may not have reviewed them.

“We found that initial AI recommendations were rated higher than final physician recommendations for common urgent care conditions,” said Joshua Pevnick, MD, MSHS, co-director of the Cedars-Sinai Department of Informatics, associate professor of medicine, and co-senior author of the study. “For example, artificial intelligence has been particularly successful in flagging urinary tract infections that may be caused by antibiotic-resistant bacteria and suggesting that a culture be ordered before medication is prescribed.”

However, Pevnick said that while AI is better at identifying critical red flags, "it has allowed physicians to better understand patients' histories and tailor their recommendations accordingly."

The retrospective study was conducted using data from Cedars-Sinai Connect, a virtual primary and urgent care program that began in 2023. An expansion of Cedars-Sinai's in-person care, Cedars-Sinai Connect aims to expand virtual health care for patients in California through a mobile app that allows people to quickly and easily access Cedars-Sina's acute and preventative care, chronic and preventive care experts.

The study reviewed 461 medical visits with AI recommendations from June 12 to July 14, 2024. Major medical issues addressed during these urgent virtual visits included adults with respiratory, urinary, vaginal, vision or dental symptoms.

Patients using the mobile app initiate visits by entering their medical concerns and providing demographic information for first-time users. An expert AI model conducts a structured dynamic interview that collects symptom information and medical history. On average, patients answer 25 questions in five minutes.

An algorithm uses the patient's responses as well as data from the patient's electronic health record to provide initial information about diseases with related symptoms. After patients with possible diagnoses present their symptoms, the mobile app allows patients to initiate a video visit with a doctor.

The algorithm also suggests diagnosis and treatment recommendations that can be viewed by the Cedars-Sinai Connect treatment physician, although during the study, Cedars-Sinai Connect may scroll down the required physicians to view them.

The biggest uncertainty in this study is whether physicians were scrolled down to look at the prescriptions, orders, referrals, or other management suggestions made by the AI ​​and whether they incorporated these recommendations into their clinical decision-making. However, the fact that AI recommendations were often rated as higher quality than physician decisions suggests that AI decision support, when effectively implemented at the point of care, can improve clinical decision making for common and acute conditions. “

Caroline Goldzweig, MD, Cedars-Sinai Medical Network chief medical officer and co-senior author of the study

The AI ​​system used for Cedars-Sinai Connect was developed by K Health, creating the technology to reduce the burden of clinical intake and data entry, allowing physicians to focus more on patient care. K Health and Cedars-Sinai developed Cedars-Sinai Connect through a joint venture and collaborated on the research study. Tel Aviv University investigators, including first author Dan Zeltzer, PhD, also participated in the study.

“We put AI to the test in real-world conditions, not made-up scenarios,” said Ran Shaul, co-founder and chief product officer of K Health. “In the reality of everyday primary care, there are so many variables and factors dealing with complex people, and any AI has to deal with incomplete data and a very diverse patient population.”

Shaul said investigators learned that if you train AI on the treasure trove of deidentified clinical notes and use daily provider care as a constantly reinforced learning mechanism, "you can achieve the level of accuracy you would expect from a human doctor."

Additional authors involved in the study include Dan Zeltzer, PhD; Zehavi Kugler, MD; Lior Hayat, MD; Tamar Brufman, MD; Ran Ilan Ber, PhD; Keren Leibovich, PhD; Tom Beer, MSC; and Ilan Frank, MSc.

This work was supported by funding from K Health.

Sources: