Virtual 'Friends': Video Chatbot May Bring Joey Tribbiani To Your Phone

If talking to Siri, Cortana and Alexa aren’t cutting it for you, know that researchers are looking for a way to spice things up with characters from popular TV shows. Like Joey Tribbiani from NBC’s cult-favorite sitcom Friends .

A team of researchers at the University of Leeds—James Charles, Derek Magee and David Hogg—have turned to algorithms and machine learning tools for digitizing avatars of well known characters. The algorithms will have the ability to identify specific characters and catch individual facial expressions, speech patterns, body language and voice. Then, a machine learning tool will analyze scripts to learn how the character pieces words together. The final result will be a “virtually immortalized” version of the character that can create new sentences in the style and appearance of the character.

“The objective of this work is to build virtual talking avatars of characters fully automatically from TV shows,” said the team in a paper, according to Prosthetic Knowledge. “From this unconstrained data, we show how to capture a character’s style of speech, intonation, visual appearance and language in an effort to construct an interactive avatar of the person and effectively immortalize them in a computational model.”

The team, which worked on Joey Tribbiani first, has successfully created Virtual Joey using all 236 episodes from the series. A digital avatar of the fan-favorite character, played on the show by Matt LeBlanc, can speak in a similar fashion of Joey Tribbiani in that it can “generate novel sentences as well as character specific speech and video.” For example, two sentences Virtual Joey came up with are “Hey Ross do you want me to talk to some lady” and “I like pizza with cheese.”

There are; however, some existing limitations and room for improvement. For one, the digital mouth is superimposed over LeBlanc’s original footage. And the dialogue is not as natural as it could be.

“We plan to improve the rendering of the avatar and extend our model to include interaction with real people and also between avatars,” said the authors in a paper.