Nick Bilton reports on the next round of fake news tools that allow users to manipulate audio and video to change what’s being said, a sort of real-time Photoshop for moving images and audio. Want to make it look like a celebrity used a taboo word, or misquote a politician? No problem!
From the Face2Face research team:
We present a novel approach for real-time facial reenactment of a monocular target video sequence (e.g., Youtube video). The source sequence is also a monocular video stream, captured live with a commodity webcam. Our goal is to animate the facial expressions of the target video by a source actor and re-render the manipulated output video in a photo-realistic fashion. To this end, we first address the under-constrained problem of facial identity recovery from monocular video by non-rigid model-based bundling. At run time, we track facial expressions of both source and target video using a dense photometric consistency measure. Reenactment is then achieved by fast and efficient deformation transfer between source and target. The mouth interior that best matches the re-targeted expression is retrieved from the target sequence and warped to produce an accurate fit. Finally, we convincingly re-render the synthesized target face on top of the corresponding video stream such that it seamlessly blends with the real-world illumination. We demonstrate our method in a live setup, where Youtube videos are reenacted in real time.
Bilton also reports on Adobe VoCo, a similar audio technology.
Audio advancements may be just as harrowing. At its annual developer’s conference, in November, Adobe showed off a new product that has been nicknamed “Photoshop for audio.” The product allows users to feed about ten to 20 minutes of someone’s voice into the application and then allows them to type words that are expressed in that exact voice. The resultant voice, which is comprised of the person’s phonemes, or the distinct units of sound that distinguish one word from another in each language, doesn’t sound even remotely computer-generated or made up. It sounds real.
Get ready for the world to be even more post-truthy!
• Fake news is about to get scarier than you ever dreamed (Vanity Fair)