4chan Members Allegedly Used Eleven Labs To Create Disturbing Audio Content In Celeb Voices

Artificial intelligence speech software Eleven Labs, which claims to create the "most realistic and versatile" audio content, addressed one of its shortcomings Monday.

The software acknowledged its inability to stop the misuse of its technology for the creation of disturbing and racist content in the voices of celebrities. In one example, a generated voice that sounded like actor Emma Watson reading a section of Mein Kampf surfaced online.

"Crazy weekend - thank you to everyone for trying out our Beta platform. While we see our tech being overwhelmingly applied to positive use, we also see an increasing number of voice cloning misuse cases. We want to reach out to Twitter community for thoughts and feedback!" a tweet by the software company read.

The beta version of the software's text-to-speech platform for English and Polish was recently launched. But, who's responsible for the hack?

Members of 4chan, an anonymous English-language imageboard website, apparently used the platform to generate clips that sound like celebs saying objectionable things, according to VICE. The clips were posted to 4chan Sunday, days after Eleven Labs raised $2 million in a pre-seed funding round. The company has around 500 users and around 5,000 others are reportedly on the waitlist.

The high quality of the audio clips and the ease with which they were created signal a boom in risqué and dangerous deepfake audios flooding the internet in the near future. Recently, Twitch streamer Atrioc claimed he landed on the deepfake images of internet personalities Maya Higa and Pokimane by clicking on an ad while researching AI, an explanation that fans refuse to buy.

Coming back to the misuse of Eleven Labs, another example that came to light was the AI-generated voice that sounded like controversial commentator Ben Shapiro making racist comments against Alexandria Ocasio-Cortez. The two butted heads in 2018 after AOC refused Shapiro's request to appear on his podcast for a $10K donation to her campaign.

"Just like catcalling, I don't owe a response to unsolicited requests from men with bad intentions. And also like catcalling, for some reason they feel entitled to one," AOC had tweeted at the time, in reference to Shapiro's request.

Another audio also surfaced that sounded like the character Rick Sanchez from "Rick and Morty" saying, "I'm going to beat my wife Morty. I'm going to beat my f---ing wife Morty. I'm going to beat her to death Morty." For context, the creator of the show, Justin Roiland, who voices both Rick and Morty's characters was charged with corporal injury and false imprisonment by menace, fraud, violence, or deceit against his former girlfriend, according to the criminal complaint obtained by AP earlier this month.

A 4chan post that shared a wide range of similar audio clips also linked Eleven Labs' beta version, suggesting the platform was used to create the same.

Eleven Labs is now working on ways to stop the misuse. "Our current ideas: (1) Additional account verifications to enable Voice Cloning: such as payment info or even full ID verification (2) Verifying copyright to the voice by submitting sample with prompted text (3) Drop Voice Lab altogether and manually verify each Cloning Request," they tweeted. The platform has also invited thoughts/suggestions from users.