Google llc (20240428071). GRANULAR NEURAL NETWORK ARCHITECTURE SEARCH OVER LOW-LEVEL PRIMITIVES
Contents
GRANULAR NEURAL NETWORK ARCHITECTURE SEARCH OVER LOW-LEVEL PRIMITIVES
Organization Name
Inventor(s)
David Richard So of Brooklyn NY (US)
Quoc V. Le of Sunnyvale CA (US)
Hanxiao Liu of Santa Clara CA (US)
Wojciech Andrzej Manke of Zürich (CH)
Zihang Dai of Pittsburgh PA (US)
Noam M. Shazeer of Palo Alto CA (US)
GRANULAR NEURAL NETWORK ARCHITECTURE SEARCH OVER LOW-LEVEL PRIMITIVES
This abstract first appeared for US patent application 20240428071 titled 'GRANULAR NEURAL NETWORK ARCHITECTURE SEARCH OVER LOW-LEVEL PRIMITIVES
Original Abstract Submitted
methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output. one of the systems includes an attention neural network configured to perform the machine learning task. the attention neural network includes one or more attentions layers that each include a squared relu activation layer, a depth-wise convolution layer, or both.
