Newsletter
Following a successful beta testing phase, ElevenLabs has officially launched its new AI platform, which can convincingly mimic voices in an impressive array of 30 languages.
According to ElevenLabs, the big breakthrough centers around its new Multilingual v2 AI model, which can not only detect languages automatically, including Korean, Dutch, and Vietnamese, but can also generate these languages with their particular emotional nuances. This AI model, combined with a user-friendly interface where users only need to upload a few speech samples to clone their voices, simplifies the process for everyday users.
In terms of options, ElevenLabs offers two types of voice cloning services. The first is a text-to-speech tool, which transforms written content into spoken words. The second is the “VoiceLab,” designed to clone specific voices.
“With this release, we’re one step closer to making this dream a reality and making human-quality AI voices available in every dialect. Our text-to-speech generation tools help level the playing field and bring top-quality spoken audio capabilities to all the creators out there,” said ElevenLabs CEO and co-founder Mati Staniszewski.
Ethical and moral complications
While this new development is certainly a big step forward in AI, several major concerns persist. These include the potential misuse of voice cloning software by threat actors to replicate the voices of prominent figures for promoting hate speech and misinformation. However, the company has acknowledged those concerns and has implemented safeguards, including limiting voice cloning to paid accounts and introducing AI-based detection tools.
Another concern involves AI potentially replacing the jobs of voice-over artists, as companies may find using AI more cost-effective. Additionally, the fact that publishers like Lukeman Literary are already using AIs to translate their audiobooks into other languages suggests that this trend will grow further with time.
“It used to take Lukeman’s team weeks to produce a single audiobook because it required them to find the right voiceover artist, book a recording studio, record and manage the post-production. Now, the entire process takes a few hours,” said ElevenLabs.