Google llc (20250095634). Language Agnostic Multilingual End-To-End Streaming On-Device ASR System

Language Agnostic Multilingual End-To-End Streaming On-Device ASR System

Organization Name

google llc

Inventor(s)

Bo Li of Fremont CA US

Tara N. Sainath of Jersey City NJ US

Ruoming Pang of New York NY US

Shuo-yiin Chang of Sunnyvale CA US

Qiumin Xu of Mountain View CA US

Trevor Strohman of Mountain View CA US

Vince Chen of Mountain View CA US

Qiao Liang of Mountain View CA US

Heguang Liu of Mountain View CA US

Yanzhang He of Mountain View CA US

Parisa Haghani of Atlanta GA US

Sameer Bidichandani of Mountain View CA US

Language Agnostic Multilingual End-To-End Streaming On-Device ASR System

This abstract first appeared for US patent application 20250095634 titled 'Language Agnostic Multilingual End-To-End Streaming On-Device ASR System

Original Abstract Submitted

a method includes receiving a sequence of acoustic frames characterizing one or more utterances as input to a multilingual automated speech recognition (asr) model. the method also includes generating a higher order feature representation for a corresponding acoustic frame. the method also includes generating a hidden representation based on a sequence of non-blank symbols output by a final softmax layer. the method also includes generating a probability distribution over possible speech recognition hypotheses based on the hidden representation generated by the prediction network at each of the plurality of output steps and the higher order feature representation generated by the encoder at each of the plurality of output steps. the method also includes predicting an end of utterance (eou) token at an end of each utterance. the method also includes classifying each acoustic frame as either speech, initial silence, intermediate silence, or final silence.