• Email Us: [email protected]
  • Contact Us: +1 718 874 1545
  • Skip to main content
  • Skip to primary sidebar

Medical Market Report

  • Home
  • All Reports
  • About Us
  • Contact Us

It’s Possible To Extract Audio From A Still, Soundless Image

September 28, 2023 by Deborah Bloomfield

Researchers have found a way to extract audio from still images and soundless video after a professor was inspired to do so by the sci-fi TV show Fringe.

In the TV show, the FBI is able to extract recorded sound from a melted pane of glass. Den of Geek called the idea a “ridiculous pseudo-science technique”, which seems fair enough. However, professor of electrical and computer engineering and computer science at Northeastern University Kevin Fu saw the review and set about showing that extracting audio from images and silent video, at least, is possible.

Advertisement

“Imagine someone is doing a TikTok video and they mute it and dub music,” Fu said in a press release. “Have you ever been curious about what they’re really saying? Was it ‘Watermelon watermelon’ or ‘Here’s my password’? Was somebody speaking behind them? You can actually pick up what is being spoken off camera.”

So, how can this happen? Cameras, while aimed at capturing visual information, are inadvertently picking up audio information too. Virtually all camera phones have image stabilization technology built in. Springs hold the camera lens suspended in liquid, while an electromagnet pushes the camera lens around to reduce camera shake. 

While a cool feature, it is this which enables the capture of audio. As someone or something makes a noise near the camera lens, the springs vibrate slightly and bend the light ever so slightly. It’s not noticeable “unless you’re looking for it” according to Fu. Alone, it wouldn’t provide you with useful audio. However, another feature of modern phone cameras helps turn it into something worth listening to.

“The way cameras work today to reduce cost basically is they don’t scan all pixels of an image simultaneously – they do it one row at a time,” Fu explained. “[That happens] hundreds of thousands of times in a single photo. What this basically means is you’re able to amplify by over a thousand times how much frequency information you can get, basically the granularity of the audio.”

Advertisement

Using this information, captured as a byproduct of how photographs are taken, it’s possible to extract fairly muffled audio from pretty much any photo that contains light. Applying a machine-learning algorithm named Side Eye by the team, they can get useful audio.

“If you want to know if I said yes or no, you can train [Side Eye] on people saying yes and no and then look at the patterns and with high confidence when I get an image later know if someone said yes or no.”

Testing their system on 10 different smartphones, Fu’s team found that it could recognize spoken digits with 80.66 percent accuracy, identify which of 20 speakers said the words with 91.28 percent accuracy, and guess the gender of speakers with 99.67 percent accuracy. 

This could, of course, be a cybersecurity nightmare, if people with nefarious intentions are able to hear what is being said from still images and videos where no audio was (intentionally) captured. The team attempted to address solutions, including stronger springs, locking lenses, and randomizing how the rolling shutter captures pixels. 

Advertisement

Ultimately though, the team is more interested in how extracted audio could be used in legal cases.

“Maybe there’s an alibi and it’s being admitted to court and somebody wants to prove somebody was or wasn’t there,” Fu said. “You might be able to use this technique if you have an authenticated video with a known timestamp to confirm one way or the other. If you hear the person’s voice, they’re more than likely there.”

The study is posted on pre-print server arXiv, and was presented at the 2023 IEEE Symposium on Security and Privacy.

Deborah Bloomfield
Deborah Bloomfield

Related posts:

  1. Chinese court rules against #MeToo plaintiff
  2. France says Mali must stick to election timetable
  3. Blinken meets Lopez Obrador to soothe thorny U.S.-Mexico relations
  4. What Would Happen To Humanity If All Microbes Suddenly Disappeared?

Source Link: It's Possible To Extract Audio From A Still, Soundless Image

Filed Under: News

Primary Sidebar

  • If Birds Are Dinosaurs, Why Are None As Big As T. Rexes?
  • Psychologists Demonstrate Illusion That Could Be Screwing Up Our Perception Of Time
  • Why Are So Many Enormous Roman Shoes Being Discovered At Hadrian’s Wall?
  • Scientists Think They’ve Pinpointed Structural Differences In Psychopaths’ Brains
  • We’ve Found Our Third-Ever Interstellar Visitor, Orcas Filmed Kissing (With Tongues) In The Wild, And Much More This Week
  • The “Eyes Of Clavius” Will Be Visible On The Moon Today, Thanks To Clair-Obscur Effect
  • Shockingly High Microplastic Levels Found On Remote Mediterranean Coral Reef Island
  • Interstellar Object, Cheesy Nightmares, And Smooching Orcas
  • World’s Largest Martian Meteorite Up For Auction Could Reach Whopping $2-4 Million
  • Kimalu The Beluga Whale Undergoes Pioneering Surgery And Becomes First Beluga To Survive General Aesthetic
  • The 1986 Soviet Space Mission That’s Never Been Repeated: Mir To Salyut And Back Again
  • Grisly Incident In Yellowstone National Park Shows Just How Dangerous This Vibrant Wilderness Can Be
  • Out Of All Greenhouse Gas Emitters On Earth, One US Organization Takes The Biscuit
  • Overly Ambitious Adder Attempts To Eat Hare 10 Times Its Mass In Gnarly Video
  • How Fast Does A Spacecraft Need To Go To Escape The Solar System?
  • President Trump’s Cuts To USAID Could Result In A “Staggering” 14 Million Avoidable Deaths By 2030
  • Dzo: Hybrids Beasts That Are Perfectly Crafted For Life On Earth’s Highest Mountains
  • “Rarest Event Ever” Had A Half-Life 1 Trillion Times Longer Than The Age Of The Universe – How Did We See It?
  • Meet The Bille, A Self-Righting Tetrahedron That Nobody Was Sure Could Exist
  • Neurogenesis Confirmed: Adult Brains Really Do Make New Hippocampal Neurons
  • Business
  • Health
  • News
  • Science
  • Technology
  • +1 718 874 1545
  • +91 78878 22626
  • [email protected]
Office Address
Prudour Pvt. Ltd. 420 Lexington Avenue Suite 300 New York City, NY 10170.

Powered by Prudour Network

Copyrights © 2025 · Medical Market Report. All Rights Reserved.

Go to mobile version