18835613. MERGING ELEMENTS OF SEQUENCES DURING NEURAL NETWORK PROCESSING (Google LLC)

MERGING ELEMENTS OF SEQUENCES DURING NEURAL NETWORK PROCESSING

Organization Name

Google LLC

Inventor(s)

Cédric Benjamin Renggli of Uster CH

Carlos Riquelme Ruiz of Zurich CH

André Susano Pinto of Zürich CH

Basil Mustafa of Zurich CH

Joan Puigcerver I Perez of Zurich CH

Neil Matthew Tinmouth Houlsby of Zurich CH

MERGING ELEMENTS OF SEQUENCES DURING NEURAL NETWORK PROCESSING

This abstract first appeared for US patent application 18835613 titled 'MERGING ELEMENTS OF SEQUENCES DURING NEURAL NETWORK PROCESSING

Original Abstract Submitted

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network including one or more merger neural network blocks that each generate block output sequence that has fewer elements than the block input sequence that is processed by the merger neural network block.