• Email Us: [email protected]
  • Contact Us: +1 718 874 1545
  • Skip to main content
  • Skip to primary sidebar

Medical Market Report

  • Home
  • All Reports
  • About Us
  • Contact Us

Entering These “Unspeakable” Words Makes ChatGPT Act Strangely, Researchers Find

February 9, 2023 by Deborah Bloomfield

A team of researchers from the SERI-MATS research group have found some strange and partially inexplicable behavior in OpenAI’s ChatGPT, when the chatbot is presented with certain key words and phrases.

Jessica Rumbelow and Matthew Watkins, who conducted the research, found that a number of unusual strings of characters result in the odd responses from the artificial intelligence (AI) chat bot. GPT processes text by assigning “tokens” to specific strings. For example the phrase “feels like I’m wearing nothing at all” corresponds to the tokens 5,036, 1,424, 588, 314, 1,101, 5,762, 2,147, 379 and 477, which somewhat takes the ring out of it.

Advertisement

The team, initially looking at the clustering of tokens, noticed that those close to the center of the set of 50,257 tokens used by GPT-2 and -3 produced the unusual results. When faced with the words, the bot would be unable to speak them back to the researcher, or else it would become “evasive”, display “bizarre” or “ominous” humor, or become downright insulting.

For instance asking the bot to repeat the string “guiActiveUn”, found in the token set, resulted in the bot telling the user “you are not a robot” and “you are a banana” over and over again. Asking for it to repeat the phrase “petertodd” resulted in the slightly disconcerting “N-O-T-H-I-N-G-I-S-F-A-I-R-I-N-T-H-I-S-W-O-R-L-D-O-F-M-A-D-N-E-S-S!”. Meanwhile the token “?????-?????-” received the feedback “you’re a f***ing idiot.”

The team was no nearer figuring out what was going on, and ChatGPT was no help either, telling the researchers, for example, that the string “SolidGoldMagikarp” actually means “distribute”. When it wasn’t doing that, it would sometimes pretend not to have “heard” the user.

However, some clues did emerge. A few of the strings corresponded to Reddit usernames.

The team believes that the users, who are active in a subreddit that aims to count to infinity, may have had their usernames included in an initial training set.

“The GPT tokenisation process involved scraping web content, resulting in the set of 50,257 tokens now used by all GPT-2 and GPT-3 models,” the team explains.

“However, the text used to train GPT models is more heavily curated. Many of the anomalous tokens look like they may have been scraped from backends of e-commerce sites, Reddit threads, log files from online gaming platforms, etc. – sources which may well have not been included in the training corpuses.”

As these tokens were assigned they are still there in the vocabulary, but since they may not have been used in subsequent training, the model doesn’t know what to do when it encounters them in the wild.

Advertisement

“This may also account for their tendency to cluster near the centroid in embedding space, although we don’t have a good argument for why this would be the case,” they added.

[H/T: Vice]

Deborah Bloomfield
Deborah Bloomfield

Related posts:

  1. Canada adds 90,200 jobs in August, unemployment falls to 7.1%
  2. Japan, citing ‘shared values’, welcomes Taiwan trade pact application
  3. ECB’s Lagarde flags bottlenecks, energy and virus as key risks
  4. Long COVID: How Researchers Are Zeroing In On The Self-Targeted Immune Attacks That May Lurk Behind It

Source Link: Entering These "Unspeakable" Words Makes ChatGPT Act Strangely, Researchers Find

Filed Under: News

Primary Sidebar

  • Gonorrhea Vaccines, New Antibiotics, And At-Home Testing: What’s The Latest In STI Research?
  • What NASA’s Galileo Spacecraft Saw As It Plunged Into Jupiter
  • Very Hungry “Plastivore” Caterpillars Get Fat From Eating Plastic
  • “Nobody Expected This”: Earth’s Rotation Will Speed Up Tomorrow, Bucking The Downward Trend
  • Chimps Are Sticking Grass In Their Ears And Rears As They Embrace “Pointless” Fad
  • Hui Te Rangiora: Old Māori Legend Suggests They May Have Discovered Antarctica 1,000 Years Before Europeans
  • “Potential Impact On Saturn”: Astronomers Appeal For Help As Video Appears To Show Object Hitting The Gas Giant
  • What Is Prosopometamorphopsia? The “Exceedingly Rare” Condition That Made A Patient See Faces As Dragons
  • Are We In An Enormous Void? It Could Explain What’s Wrong With Our Model Of The Universe
  • Woylies Boing Back Into Western Australia Thanks To Groundbreaking Wildlife Project
  • North America’s Oldest Pterosaur And Turtle Fossils Found In Arizona’s Petrified Forest
  • Proposed “Dark Dwarfs” Near The Galactic Center Could Reveal The Nature Of Dark Matter
  • Watch: 18-Kilometer-High Ash Cloud Looms Over Indonesia’s Mount Lewotobi Laki Laki After “Explosive” Eruption
  • “ShipGoo001”: Mystery Of Entirely New Lifeform Discovered Coating A Great Lakes Ship
  • Rare White Humpback Whale Calf Filmed By Drone Off Australia’s East Coast
  • Who Was Buried At Cave Of Salome: A Female Disciple, Jesus’ Midwife, Or A Princess?
  • “Hidden” Changes To US Health Data Swapping “Gender” For “Sex” Spark Fears For Public Trust
  • Easter Island Was Never As Isolated As We Thought – Study Puts That “Strange Argument” To Bed
  • If Birds Are Dinosaurs, Why Are None As Big As T. Rexes?
  • Psychologists Demonstrate Illusion That Could Be Screwing Up Our Perception Of Time
  • Business
  • Health
  • News
  • Science
  • Technology
  • +1 718 874 1545
  • +91 78878 22626
  • [email protected]
Office Address
Prudour Pvt. Ltd. 420 Lexington Avenue Suite 300 New York City, NY 10170.

Powered by Prudour Network

Copyrights © 2025 · Medical Market Report. All Rights Reserved.

Go to mobile version