18834070. ROUTING TO EXPERT SUBNETWORKS IN MIXTURE-OF-EXPERTS NEURAL NETWORKS (Google LLC)
ROUTING TO EXPERT SUBNETWORKS IN MIXTURE-OF-EXPERTS NEURAL NETWORKS
Organization Name
Inventor(s)
Hanxiao Liu of Santa Clara CA US
Yuzhe Zhao of San Francisco CA US
Yanping Huang of Mountain View CA US
Zhifeng Chen of Sunnyvale CA US
Andrew M. Dai of San Francisco CA US
ROUTING TO EXPERT SUBNETWORKS IN MIXTURE-OF-EXPERTS NEURAL NETWORKS
This abstract first appeared for US patent application 18834070 titled 'ROUTING TO EXPERT SUBNETWORKS IN MIXTURE-OF-EXPERTS NEURAL NETWORKS
Original Abstract Submitted
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a neural network configured to perform the machine learning task, the neural network including one or more expert neural network blocks that each include router that performs expert-choice routing between multiple expert neural networks.
- Google LLC
- Hanxiao Liu of Santa Clara CA US
- Quoc V. Le of Sunnyvale CA US
- Yanqi Zhou of Sunnyvale CA US
- Tao Lei of Sunnyvale CA US
- Yuzhe Zhao of San Francisco CA US
- Yanping Huang of Mountain View CA US
- Nan Du of San Jose CA US
- Zhifeng Chen of Sunnyvale CA US
- Andrew M. Dai of San Francisco CA US
- James Laudon of Madison WI US
- G06N3/048
- CPC G06N3/048
