Facebook, Israeli Researchers Develop Technology To Simulate Facial Expressions Using Single Photograph

Creating videos from photos is something that Apple and Google have been working on with their respective offerings such as Live Photos and Motion Photos. A new research has now suggested a way to use photos more diversely — by creating several images from just one selfie.

Researchers from Tel Aviv University, Israel, have developed a software that can automatically animate a still portrait and needs only a single frame to do that. They use what they call a “driving video” of a subject to map out facial expressions and add them to any photo. All the user needs is one input photo to create animated pictures with myriad expressions such as happiness, sadness, anger or surprise.

The technique has been mapped out in a paper titled “Bringing portraits to life.” The researchers use two-dimensional warps, which add facial transformations to a basic image, creating an imitation of various emotions. They don’t only add expressions by changing how the facial skin, lips, and teeth look in the images but also add dynamic details such as creases and wrinkles, which make the images more believable. Researchers have paid special attention to details and have animated even the "inner mouth of a subject."

The result is a GIF like function, which shows the subject displaying different images. However, one can still differentiate between the images that are created — which are still slightly unnatural — and the actual image. The model is likely to improve further.

This is not the first time researchers have tried to do this. But the previous models were based on video editing and they would add expressions by editing multiple frames. However, this is the first time that the researchers have been able to generate images with different emotions from a single basic image.

The most important issue with such research is getting the finer details of human expressions correct — such as the effect of a smile on various portions of the face. Getting complicated expressions such as blushing correct is something that not many photo or video software have gotten correct.

Using the software, a subject can automatically interact with viewers using a still image, which will respond to stimuli with changing facial expressions. Two of the researchers, working on the project, belong to the Tel Aviv University, while two others belong to Facebook. The association of Facebook with the project implies there is a possibility the software could be used in real-world applications.

Whether this feature would end up on our Facebook profiles soon is not yet known. But chances are likely the company may include it in its reactions. The company recently offered reactions in the form of emojis that can let the user respond to a comment and might use this technology to enhance that feature further.