• Email Us: [email protected]
  • Contact Us: +1 718 874 1545
  • Skip to main content
  • Skip to primary sidebar

Medical Market Report

  • Home
  • All Reports
  • About Us
  • Contact Us

ChatGPT Can Pass Part Of The US Medical Licensing Exam

January 24, 2023 by Deborah Bloomfield

A team of researchers has tested ChatGPT, an artificial intelligence (AI) chatbot, on its clinical reasoning skills using questions from the United States Medical Licensing Examination (USMLE).

The team, publishing their results on preprint server medRxiv, wrote that they chose to test the generative language AI on questions from the USMLE as it was a “high-stakes, comprehensive three-step standardized testing program covering all topics in physicians’ fund of knowledge, spanning basic science, clinical reasoning, medical management, and bioethics”. 

Advertisement

The language model, trained on massive amounts of text from the internet, was not trained on the version of the test used by the researchers; nor was it given any supplementary medical training prior to the study, which saw it answer a number of open-ended and multiple choice questions. 

“In this present study, ChatGPT performed at >50% accuracy across all examinations, exceeding 60% in most analyses,” the team wrote in their study. 

“The USMLE pass threshold, while varying by year, is approximately 60%. Therefore, ChatGPT is now comfortably within the passing range. Being the first experiment to reach this benchmark, we believe this is a surprising and impressive result.”

Advertisement

The team write that the performance of the AI could be improved with more prompting and interaction with the model. Where the AI performed poorly, providing answers that were less concordant, they believe it was partly due to missing information that the AI has not encountered.

However, they believe that the OpenAI bot had an advantage over models trained entirely on medical text, as it got more of an overview of the clinical context.

“Paradoxically, ChatGPT outperformed PubMedGPT (accuracy 50.8%, unpublished data), a counterpart [language learning model] with similar neural structure, but trained exclusively on biomedical domain literature,” the team wrote in their discussion. 

Advertisement

“We speculate that domain-specific training may have created greater ambivalence in the PubMedGPT model, as it absorbs real-world text from ongoing academic discourse that tends to be inconclusive, contradictory, or highly conservative or noncommittal in its language.”

The team writes that AI may soon become commonplace in healthcare settings, given the speed of progress of the industry, perhaps by improving risk assessment or providing assistance and support with clinical decisions.

The study is published on preprint server medRxiv. It has not yet been peer-reviewed.

Deborah Bloomfield
Deborah Bloomfield

Related posts:

  1. Social network Peanut expands to include more women with launch of Peanut Menopause
  2. Marketmind: Watch those spiralling gas prices
  3. Thai central bank chief warns economy remains fragile, exposed to shocks
  4. Be On The Cutting-Edge Of Tech With This Top-Rated Learning Bundle

Source Link: ChatGPT Can Pass Part Of The US Medical Licensing Exam

Filed Under: News

Primary Sidebar

  • Unethical Experiments: When Scientists Really Should Have Stopped What They Were Doing Immediately
  • The First Humans Were Hunted By Leopards And Weren’t The Apex Predators We Thought They Were
  • Earth’s Passage Through The Galaxy Might Be Written In Its Rocks
  • What Is An Einstein Cross – And Why Is The Latest One Such A Unique Find?
  • If We Found Life On Mars, What Would That Mean For The Fermi Paradox And The Great Filter?
  • The Longest Living Mammals Are Giants That Live Up To 200 Years In The Icy Arctic
  • Entirely New Virus Detected In Bat Urine, And It’s Only The 4th Of Its Kind Ever Isolated
  • The First Ever Full Asteroid History: From Its Doomed Discovery To Collecting Its Meteorites
  • World’s Oldest Pachycephalosaur Fossil Pushes Back These Dinosaurs’ Emergence By 15 Million Years
  • The Hole In The Ozone Layer Is Healing And On Track For Full Recovery In The 21st Century, Thanks To Science
  • First Sweet Potato Genome Reveals They’re Hybrids With A Puzzling Past And 6 Sets Of Chromosomes
  • Why Is The Top Of Canada So Sparsely Populated? Meet The “Canadian Shield”
  • Humans Are In The Middle Of “A Great Evolutionary Transition”, New Paper Claims
  • Why Do Some Toilets Have Two Flush Buttons?
  • 130-Year-Old Butter Additive Discovered In Danish Basement Contains Bacteria From The 1890s
  • Prehistoric Humans Made Necklaces From Marine Mollusk Fossils 20,000 Years Ago
  • Zond 5: In 1968 Two Soviet Steppe Tortoises Beat Humans To Orbiting Around The Moon
  • Why Cats Adapted This Defense Mechanism From Snakes
  • Mother Orca Seen Carrying Dead Calf Once Again On Washington Coast
  • A Busy Spider Season Is Brewing: Why This Fall Could See A Boom Of Arachnid Activity
  • Business
  • Health
  • News
  • Science
  • Technology
  • +1 718 874 1545
  • +91 78878 22626
  • [email protected]
Office Address
Prudour Pvt. Ltd. 420 Lexington Avenue Suite 300 New York City, NY 10170.

Powered by Prudour Network

Copyrights © 2025 · Medical Market Report. All Rights Reserved.

Go to mobile version