• Email Us: [email protected]
  • Contact Us: +1 718 618 4351
  • Skip to main content
  • Skip to primary sidebar

Medical Market Report

  • Home
  • All Reports
  • About Us
  • Contact Us

ChatGPT Can Pass Part Of The US Medical Licensing Exam

January 24, 2023 by Deborah Bloomfield

A team of researchers has tested ChatGPT, an artificial intelligence (AI) chatbot, on its clinical reasoning skills using questions from the United States Medical Licensing Examination (USMLE).

The team, publishing their results on preprint server medRxiv, wrote that they chose to test the generative language AI on questions from the USMLE as it was a “high-stakes, comprehensive three-step standardized testing program covering all topics in physicians’ fund of knowledge, spanning basic science, clinical reasoning, medical management, and bioethics”. 

Advertisement

The language model, trained on massive amounts of text from the internet, was not trained on the version of the test used by the researchers; nor was it given any supplementary medical training prior to the study, which saw it answer a number of open-ended and multiple choice questions. 

“In this present study, ChatGPT performed at >50% accuracy across all examinations, exceeding 60% in most analyses,” the team wrote in their study. 

“The USMLE pass threshold, while varying by year, is approximately 60%. Therefore, ChatGPT is now comfortably within the passing range. Being the first experiment to reach this benchmark, we believe this is a surprising and impressive result.”

Advertisement

The team write that the performance of the AI could be improved with more prompting and interaction with the model. Where the AI performed poorly, providing answers that were less concordant, they believe it was partly due to missing information that the AI has not encountered.

However, they believe that the OpenAI bot had an advantage over models trained entirely on medical text, as it got more of an overview of the clinical context.

“Paradoxically, ChatGPT outperformed PubMedGPT (accuracy 50.8%, unpublished data), a counterpart [language learning model] with similar neural structure, but trained exclusively on biomedical domain literature,” the team wrote in their discussion. 

Advertisement

“We speculate that domain-specific training may have created greater ambivalence in the PubMedGPT model, as it absorbs real-world text from ongoing academic discourse that tends to be inconclusive, contradictory, or highly conservative or noncommittal in its language.”

The team writes that AI may soon become commonplace in healthcare settings, given the speed of progress of the industry, perhaps by improving risk assessment or providing assistance and support with clinical decisions.

The study is published on preprint server medRxiv. It has not yet been peer-reviewed.

Deborah Bloomfield
Deborah Bloomfield

Related posts:

  1. Social network Peanut expands to include more women with launch of Peanut Menopause
  2. Marketmind: Watch those spiralling gas prices
  3. Thai central bank chief warns economy remains fragile, exposed to shocks
  4. Be On The Cutting-Edge Of Tech With This Top-Rated Learning Bundle

Source Link: ChatGPT Can Pass Part Of The US Medical Licensing Exam

Filed Under: News

Primary Sidebar

  • “Flamingo” The Rescue Pigeon Is A Reminder To Never Dye Birds
  • Which Material Has The Highest Melting Point? The Answer’s Complex
  • How To Tell If That Rock You Found Is Actually A Meteorite (And If You Can Keep It)
  • Suspected Spy Craft Seen Flying Over US Is Lost Weather Balloon, Chinese Officials Say
  • The First Chiropractor Claimed The Treatment Was Inspired By A Ghost
  • Site Of Ancient Egyptian “Great Revolt”, Recorded On Rosetta Stone, Finally Discovered
  • How To Tell Real Gold From Fool’s Gold
  • The Devastating Kramatorsk Radiological Accident Almost Killed An Entire Apartment Block
  • The World’s First Environmental Clean-Up Happened 400 Million Years Ago
  • Is Scorpion Venom Really The Most Expensive Liquid In The World?
  • Jupiter Officially Has 12 New Moons
  • Massachusetts Bill Proposes Prisoners Could Trade Organs For Less Prison Time
  • Increased Rocket Launches Could Undo The Ozone Layer’s Recovery
  • Human “Mini Brains” Merge With Injured Rat Brain Tissue And Respond To Light
  • Lost Interview With Creator Of Big Bang Theory Rediscovered After 60 Years
  • Largest Gold Nugget Ever Found Weighed The Same As An Adult Man
  • Scientists Have Discovered A New Type Of Ice, And It’s A Mess
  • Why Do Some People Fear Blood, While Others Love Gory Films?
  • The Tragic Story Of A Boy Who Found A Radioactive Capsule And Brought It Inside The House
  • “Fossil” That Rewrote Indian Geologic History Is Actually A Very Recent Beehive
  • Business
  • Health
  • News
  • Science
  • Technology
  • +1 718 618 4351
  • +91 78878 22626
  • [email protected]
Office Address
Prudour Pvt. Ltd. 420 Lexington Avenue Suite 300 New York City, NY 10170.

Powered by Prudour Network

Copyrights © 2023 · Medical Market Report. All Rights Reserved.

Go to mobile version