• Email Us: [email protected]
  • Contact Us: +1 718 874 1545
  • Skip to main content
  • Skip to primary sidebar

Medical Market Report

  • Home
  • All Reports
  • About Us
  • Contact Us

ChatGPT Can Pass Part Of The US Medical Licensing Exam

January 24, 2023 by Deborah Bloomfield

A team of researchers has tested ChatGPT, an artificial intelligence (AI) chatbot, on its clinical reasoning skills using questions from the United States Medical Licensing Examination (USMLE).

The team, publishing their results on preprint server medRxiv, wrote that they chose to test the generative language AI on questions from the USMLE as it was a “high-stakes, comprehensive three-step standardized testing program covering all topics in physicians’ fund of knowledge, spanning basic science, clinical reasoning, medical management, and bioethics”. 

Advertisement

The language model, trained on massive amounts of text from the internet, was not trained on the version of the test used by the researchers; nor was it given any supplementary medical training prior to the study, which saw it answer a number of open-ended and multiple choice questions. 

“In this present study, ChatGPT performed at >50% accuracy across all examinations, exceeding 60% in most analyses,” the team wrote in their study. 

“The USMLE pass threshold, while varying by year, is approximately 60%. Therefore, ChatGPT is now comfortably within the passing range. Being the first experiment to reach this benchmark, we believe this is a surprising and impressive result.”

Advertisement

The team write that the performance of the AI could be improved with more prompting and interaction with the model. Where the AI performed poorly, providing answers that were less concordant, they believe it was partly due to missing information that the AI has not encountered.

However, they believe that the OpenAI bot had an advantage over models trained entirely on medical text, as it got more of an overview of the clinical context.

“Paradoxically, ChatGPT outperformed PubMedGPT (accuracy 50.8%, unpublished data), a counterpart [language learning model] with similar neural structure, but trained exclusively on biomedical domain literature,” the team wrote in their discussion. 

Advertisement

“We speculate that domain-specific training may have created greater ambivalence in the PubMedGPT model, as it absorbs real-world text from ongoing academic discourse that tends to be inconclusive, contradictory, or highly conservative or noncommittal in its language.”

The team writes that AI may soon become commonplace in healthcare settings, given the speed of progress of the industry, perhaps by improving risk assessment or providing assistance and support with clinical decisions.

The study is published on preprint server medRxiv. It has not yet been peer-reviewed.

Deborah Bloomfield
Deborah Bloomfield

Related posts:

  1. Social network Peanut expands to include more women with launch of Peanut Menopause
  2. Marketmind: Watch those spiralling gas prices
  3. Thai central bank chief warns economy remains fragile, exposed to shocks
  4. Be On The Cutting-Edge Of Tech With This Top-Rated Learning Bundle

Source Link: ChatGPT Can Pass Part Of The US Medical Licensing Exam

Filed Under: News

Primary Sidebar

  • Orcas Filmed Kissing (With Tongues) In The Wild For The First Time
  • How Easy Is It For A Country To Change Its Time Zone?
  • Earth’s First Commercial Space Station Set To Launch In 2026
  • Black Hole Moon: Rogue Planets With Weird Signatures Could Be A Sign Of Advanced Alien Life
  • World’s Largest Ephemeral Lake Set To Turn Iconic Peachy Pink After Extreme Flooding
  • Stunning New JWST Observations Give Further Evidence That Dark Matter Is A Real Substance
  • How Big Is This Spider? Study Explains Why You Might Overestimate Their Size
  • Orcas Sometimes Give Humans Presents Of Food And We Don’t Know Why
  • New Approach For Interstellar Navigation Was Tested On A Spacecraft 9 Billion Kilometers Away
  • For Only The Second Recorded Time, Two Novae Are Visible With The Naked Eye At Once
  • Long-Lost Ancient Egyptian City Ruled By Cobra Goddess Discovered In Nile Delta
  • Much Maligned Norwegian Lemming Is One Of The Newest Mammal Species On Earth
  • Where Are The Real Geographical Centers Of All The Continents?
  • New Species Of South African Rain Frog Discovered, And It’s Absolutely Fuming About It
  • Love Cheese But Hate Nightmares? Bad News, It Looks Like The Two Really Are Related
  • Project Hail Mary Trailer First Look: What Would Happen If The Sun Got Darker?
  • Newly Discovered Cell Structure Might Hold Key To Understanding Devastating Genetic Disorders
  • What Is Kakeya’s Needle Problem, And Why Do We Want To Solve It?
  • “I Wasn’t Prepared For The Sheer Number Of Them”: Cave Of Mummified Never-Before-Seen Eyeless Invertebrates Amazes Scientists
  • Asteroid Day At 10: How The World Is More Prepared Than Ever To Face Celestial Threats
  • Business
  • Health
  • News
  • Science
  • Technology
  • +1 718 874 1545
  • +91 78878 22626
  • [email protected]
Office Address
Prudour Pvt. Ltd. 420 Lexington Avenue Suite 300 New York City, NY 10170.

Powered by Prudour Network

Copyrights © 2025 · Medical Market Report. All Rights Reserved.

Go to mobile version