Microsoft technology licensing, llc (20250005365). NEURAL NETWORK INFERENCE BASED ON TABLE LOOKUP
NEURAL NETWORK INFERENCE BASED ON TABLE LOOKUP
Organization Name
microsoft technology licensing, llc
Inventor(s)
NEURAL NETWORK INFERENCE BASED ON TABLE LOOKUP
This abstract first appeared for US patent application 20250005365 titled 'NEURAL NETWORK INFERENCE BASED ON TABLE LOOKUP
Original Abstract Submitted
according to implementations of the subject matter described herein, a solution for neural network inference based on table lookup is provided. according to this solution, respective centroids in a first plurality of codebooks for a first layer of a neural network are determined along with a first weight matrix through a training procedure of the neural network. a first input for the first layer is divided into a first plurality of input sub-vectors, and target centroids are determined for the input sub-vectors based on respective distances between the input sub-vectors and the centroids. target computation results of the target centroids with the first weight matrix are selected from a lookup table. a first output for the first layer is determined based on aggregation of the target computation results. in this way, better model accuracy can be achieved while leveraging the computation acceleration in table lookup-based model inference.