GOOGLE LLC (20240404506). MULTILINGUAL SPEECH SYNTHESIS AND CROSS-LANGUAGE VOICE CLONING

MULTILINGUAL SPEECH SYNTHESIS AND CROSS-LANGUAGE VOICE CLONING

Organization Name

GOOGLE LLC

Inventor(s)

Yu Zhang of Mountain View CA (US)

Ron J. Weiss of New York NY (US)

Byungha Chun of Tokyo (JP)

Yonghui Wu of Fremont CA (US)

Zhifeng Chen of Sunnyvale CA (US)

Russell John Wyatt Skerry-ryan of Mountain View CA (US)

Ye Jia of Mountain View CA (US)

Andrew M. Rosenberg of Brooklyn NY (US)

Bhuvana Ramabhadran of Mt. Kisco NY (US)

MULTILINGUAL SPEECH SYNTHESIS AND CROSS-LANGUAGE VOICE CLONING

This abstract first appeared for US patent application 20240404506 titled 'MULTILINGUAL SPEECH SYNTHESIS AND CROSS-LANGUAGE VOICE CLONING

Original Abstract Submitted

a method includes receiving an input text sequence to be synthesized into speech in a first language and obtaining a speaker embedding, the speaker embedding specifying specific voice characteristics of a target speaker for synthesizing the input text sequence into speech that clones a voice of the target speaker. the target speaker includes a native speaker of a second language different than the first language. the method also includes generating, using a text-to-speech (tts) model, an output audio feature representation of the input text by processing the input text sequence and the speaker embedding. the output audio feature representation includes the voice characteristics of the target speaker specified by the speaker embedding.