By Andrew Myers
With Machine Learning, The Era of Synthetic Video Is Here
A new algorithm allows video editors to modify talking head videos as if they were editing text – copying, pasting, or adding and deleting words.
A new algorithm makes it possible to perform text-based editing of videos of “talking heads”; that is, speakers from the shoulders up.
To make the video appear more natural, the algorithm applies intelligent smoothing to the motion parameters and renders a 3D animated version of the desired result. However, that rendered face is still far from realistic. As a final step, a machine learning technique called Neural Rendering converts the low-fidelity digital model into a photorealistic video in perfect lip-synch.