Google’s language model’s ability to translate audio text into the speaker’s voice is somewhere cool, somewhere scary. It is worth listening to the results with which he works.

Google PaLM-2 gets an important update. In the form of AudioPaLM, they’re adding a new capability to your system that allows AI to synchronize the translation of specific text to the speaker’s voice.

the decoder Writes: The system can process and generate both text and audio, and can also be used to recognize speech and generate subtitles with the original audio. The latter is a really exciting innovation, as it allows one person to speak several languages ​​at the same time.

See also  Sandor Csanyi, Zsolt Hernadi, Marton Nagy and Mihaly Varga in one place, at the Budapest Economic Forum!

Only 3 seconds of audio material is enough to mimic the original voice. If the audio file is shorter than that, it will loop through the system until it is three seconds long.

By incorporating AudioLM, the AI ​​can produce speech of convincing quality while preserving the unique style of the original speaker.





The language model is also capable of taking notes from the speech heard, translating it in real time, and transcribing it instantly in another language.

The potential field of application of technology is very diverse. It could be a multilingual virtual assistant, or it could be paired with a video chat service to help an international team communicate. In addition, YouTube videos can be synchronized with it, for example, or subtitles can be automatically generated in several languages.

About the project on github You can read more.

If you want to know similar things other times, like it HVG Tech department’s Facebook page.




HVG


In addition to diverse, independent and factual information, our readers who join the Pártoló membership can also enjoy a number of benefits for their financial support.
Depending on your membership level, we offer, among others:

  • We send you an exclusive weekly digest of the interesting things in the world;
  • You can gain insight into the work of HVG, you can meet our authors;
  • You can take part in pre-premier screenings of the latest films, in various events;
  • You can buy HVG books and publications at a discount;
  • You can read hvg360 digital news magazine.