• Email Us: [email protected]
  • Contact Us: +1 718 874 1545
  • Skip to main content
  • Skip to primary sidebar

Medical Market Report

  • Home
  • All Reports
  • About Us
  • Contact Us

ChatGPT Can Pass Part Of The US Medical Licensing Exam

January 24, 2023 by Deborah Bloomfield

A team of researchers has tested ChatGPT, an artificial intelligence (AI) chatbot, on its clinical reasoning skills using questions from the United States Medical Licensing Examination (USMLE).

The team, publishing their results on preprint server medRxiv, wrote that they chose to test the generative language AI on questions from the USMLE as it was a “high-stakes, comprehensive three-step standardized testing program covering all topics in physicians’ fund of knowledge, spanning basic science, clinical reasoning, medical management, and bioethics”. 

Advertisement

The language model, trained on massive amounts of text from the internet, was not trained on the version of the test used by the researchers; nor was it given any supplementary medical training prior to the study, which saw it answer a number of open-ended and multiple choice questions. 

“In this present study, ChatGPT performed at >50% accuracy across all examinations, exceeding 60% in most analyses,” the team wrote in their study. 

“The USMLE pass threshold, while varying by year, is approximately 60%. Therefore, ChatGPT is now comfortably within the passing range. Being the first experiment to reach this benchmark, we believe this is a surprising and impressive result.”

Advertisement

The team write that the performance of the AI could be improved with more prompting and interaction with the model. Where the AI performed poorly, providing answers that were less concordant, they believe it was partly due to missing information that the AI has not encountered.

However, they believe that the OpenAI bot had an advantage over models trained entirely on medical text, as it got more of an overview of the clinical context.

“Paradoxically, ChatGPT outperformed PubMedGPT (accuracy 50.8%, unpublished data), a counterpart [language learning model] with similar neural structure, but trained exclusively on biomedical domain literature,” the team wrote in their discussion. 

Advertisement

“We speculate that domain-specific training may have created greater ambivalence in the PubMedGPT model, as it absorbs real-world text from ongoing academic discourse that tends to be inconclusive, contradictory, or highly conservative or noncommittal in its language.”

The team writes that AI may soon become commonplace in healthcare settings, given the speed of progress of the industry, perhaps by improving risk assessment or providing assistance and support with clinical decisions.

The study is published on preprint server medRxiv. It has not yet been peer-reviewed.

Deborah Bloomfield
Deborah Bloomfield

Related posts:

  1. Social network Peanut expands to include more women with launch of Peanut Menopause
  2. Marketmind: Watch those spiralling gas prices
  3. Thai central bank chief warns economy remains fragile, exposed to shocks
  4. Be On The Cutting-Edge Of Tech With This Top-Rated Learning Bundle

Source Link: ChatGPT Can Pass Part Of The US Medical Licensing Exam

Filed Under: News

Primary Sidebar

  • Martian Mudstone Has Features That Might Be Biosignatures, New Brain Implant Can Decode Your Internal Monologue, And Much More This Week
  • Crocodiles Weren’t All Blood-Thirsty Killers, Some Evolved To Be Plant-Eating Vegetarians
  • Stratospheric Warming Event May Be Unfolding In The Southern Polar Vortex, Shaking Up Global Weather Systems
  • 15 Years Ago, Bees In Brooklyn Appeared Red After Snacking Where They Shouldn’t
  • Carnian Pluvial Event: It Rained For 2 Million Years — And It Changed Planet Earth Forever
  • There’s Volcanic Unrest At The Campi Flegrei Caldera – Here’s What We Know
  • The “Rumpelstiltskin Effect”: When Just Getting A Diagnosis Is Enough To Start The Healing
  • In 1962, A Boy Found A Radioactive Capsule And Brought It Inside His House — With Tragic Results
  • This Cute Creature Has One Of The Largest Genomes Of Any Mammal, With 114 Chromosomes
  • Little Air And Dramatic Evolutionary Changes Await Future Humans On Mars
  • “Black Hole Stars” Might Solve Unexplained JWST Discovery
  • Pretty In Purple: Why Do Some Otters Have Purple Teeth And Bones? It’s All Down To Their Spiky Diets
  • The World’s Largest Carnivoran Is A 3,600-Kilogram Giant That Weighs More Than Your Car
  • Devastating “Rogue Waves” Finally Have An Explanation
  • Meet The “Masked Seducer”, A Unique Bat With A Never-Before-Seen Courtship Display
  • Alaska’s Salmon River Is Turning Orange – And It’s A Stark Warning
  • Meet The Heaviest Jelly In The Seas, Weighing Over Twice As Much As A Grand Piano
  • For The First Time, We’ve Found Evidence Climate Change Is Attracting Invasive Species To Canadian Arctic
  • What Are Microfiber Cloths, And How Do They Clean So Well?
  • Stowaway Rat That Hopped On A Flight From Miami Was A “Wake-Up Call” For Global Health
  • Business
  • Health
  • News
  • Science
  • Technology
  • +1 718 874 1545
  • +91 78878 22626
  • [email protected]
Office Address
Prudour Pvt. Ltd. 420 Lexington Avenue Suite 300 New York City, NY 10170.

Powered by Prudour Network

Copyrights © 2025 · Medical Market Report. All Rights Reserved.

Go to mobile version