Deepfakes have become increasingly worrying in recent years, and it’s easy enough to imagine a future where their appearance is so good they defy even our best methods to unmask them. Of course, in such a situation, the technology would be valuable for criminals and any hostile state-sponsored persons to create deepfakes that slander or frame an inconvenient individual. However, although such a disturbing possibility remains a feature of a dystopian future, it may not be all that distant.
According to a new study, the early signs of this nightmare scenario becoming reality are already starting to appear as the methods for creating deepfakes can now bypass a tactic for identifying them: the lack of a pulse.
“Here we show for the first time that recent high-quality deepfake videos can feature a realistic heartbeat and minute changes in the color of the face, which makes them much harder to detect,” Peter Eisert, a professor at the Humboldt University of Berlin, explained in a statement.
Deepfakes are created by manipulating videos and audio files using deep learning. They change facial expressions and gestures through subtle methods, such as swapping them between different people. Sometimes, this technique can be used for a laugh, such as when an app can make you look like an animal or show you what you might look like when you are older.
But no matter whether a deepfake is intended for fun or for more malicious purposes, there has been one thing that gives them away.
Everyone has a pulse; it’s a vital sign that has been used in medicine for centuries to check people’s health (or indeed that they are even alive at all). The analysis of the transmission of light through the skin and underlying blood vessels has become a modern adaptation of this principle and is a critical medical tool – for example, a pulse oximeter is an easy clip-on device that measures someone’s blood oxygen levels and pulse using this method.
Going even further, so-called remote photoplethysmography (rPPP) is an emerging telehealthcare method that uses webcams to estimate vital signs remotely. It has been hypothesized that rPPP, in addition to having health care benefits, could also be used to identify deepfakes.
Previous research using rPPP-based deepfake detectors suggested that they were good at telling the difference between real and deepfake videos, and so confidence in this approach led some to believe that deepfakes were therefore not yet capable of mimicking heart rates. However, it seems this belief might already be out of date.
Playing with the fakes
During their work, Eisert and colleagues created a state-of-the-art deepfake detector that automatically extracts and assesses the pulse rate from videos, requiring only a 10-second-long video of a single person’s face in order to work. The coded system uses novel methods to compensate for movement and remove noise from the video.
The team also created a dataset of driving videos, which were used to create deepfakes of different target identities with the facial motion of the captured videos. During filming, the subjects being recorded had their heartbeats tracked using an electrocardiogram (ECG). This provided the team with a measure that could be used to test the rPPP’s accuracy. They found the system only had a difference of two to three beats per minute.
They then tested the rPPP on two older, widely used collections of videos of real people, which also demonstrated that it could extract heartbeat signals from all genuine videos.
But what about known deepfakes?
Well, the team used recent deepfake methods to swap faces between the genuine videos in their collection. And as you may have now anticipated, the rPPP detected pulses in these deepfakes too, even though the researchers hadn’t consciously included them. Concerningly, this fake pulse also often appeared to be highly realistic.
“Our results show that a realistic heartbeat may be added by an attacker on purpose, but can also be ‘inherited’ inadvertently from the driving genuine video. Small variations in skin tone of the real person get transferred to the deepfake together with facial motion, so that the original pulse is replicated in the fake video,” Eisert added.
So, have we already lost to the deepfakes? Well, perhaps not.
“Our experiments have shown that current deepfakes may show a realistic heartbeat, but do not show physiologically realistic variations in blood flow across space and time within the face,” Eisert explained.
“We suggest that this weakness of state-of-the-art deepfakes should be exploited by the next generation of deep fake detectors.”
The study is published in Frontiers in Imaging.
Source Link: Deepfakes Now So Realistic They Appear To Have Heartbeats, Making Their Detection Far Harder
