Meta’s looking to expand access to more content in the app, by enabling audio translations of content, using its advancing AI translation tools, which are even able to simulate lip movements to line up with audio translations.
As you can see in this example, Facebook users will now be able to view certain content in another spoken language, using Meta AI’s evolving audio translation elements, which can make it look like the people in the video are speaking in a different dialect.
As explained by Meta:
“Meta AI Translations uses the sound and tone of your own voice so it feels authentically you. And the lip syncing feature matches your mouth’s movements to the translated audio, so that it looks like you’re really speaking the language in the reel.”
Meta’s been developing this for some time, with its latest advances in AI re-creation now enabling a broad rollout of the option, beginning with auto translations from English to Spanish (and the reverse).
Creators will be able to choose if they want to activate audio translations within the Reels composer, with a new toggle to “Translate your voice with Meta AI”.
As you can see in this example, you can also choose to switch on auto lip synching, or just leave the presentation as is, and only translate the audio of the clip.
“If you want to review the translation before it’s published, you can select the toggle to enable this option, you will then receive a notification (or you can go to Professional Dashboard) to review and approve the translation. Accepting or rejecting the translation will not impact your original, non-translated reel.”
Translated reels will be presented to viewers in their preferred language, and will be notified that your reel has been translated with Meta AI. Viewers will also be able to control whether they see translations in specific languages.
It could be a handy tool, which could expand your audience, with realistic-looking translations of your content into other languages. And while only Spanish is available at launch, Meta says that it will be adding many more languages to the option in the near future.
Though as with all auto-translation options, mistakes will happen, and Meta has a few notes of advice to help ensure that its system is able to read your lips.
The system supports translation for up to two speakers, though creators should try not to speak over each other to improve accuracy
Creators should look to minimize background noise and/or music in translated reels
Meta says that building a new audience in a new language is, in some ways, like starting again, so it will take time to gain traction with this segment.
So it’s not perfect, and there will be limitations in translating your audio. But it could help you reach many, many more people, by enabling new audiences to tune into your work.
In addition to this, Meta’s also giving Facebook Page managers the option to add up to 20 of your own dubbed audio tracks to a single Reel.
“Like Meta AI translations, viewers will hear your reel in their own language so you can continue to build deeper connections with diverse audiences without language barriers.”
These manual tracks won’t be lip-synched in the same way, but they will provide another means to connect in other languages.
Facebook creators who’ve switched on Professional Mode, and have more than 1,000 followers, will be able to access the feature in the latest version of the app, while Instagram creators with public accounts in regions where Meta AI is available will also be able to activate the option.
You can learn more about Meta’s latest AI translation features here.