US Patent Application 17732372. Instruction Set Architecture for Implementing Linear Activation Functions in Neural Networks simplified abstract
Contents
Instruction Set Architecture for Implementing Linear Activation Functions in Neural Networks
Organization Name
Inventor(s)
Srijesh Sudarsanan of Waltham MA (US)
Deepak Mathew of Acton MA (US)
Marc Hoffman of Mansfield MA (US)
Sundar Rajan Balasubramanian of Groton MA (US)
Mansi Jain of Littleton MA (US)
James Lee of Northborough MA (US)
Gerald Sweeney of Chelmsford MA (US)
Instruction Set Architecture for Implementing Linear Activation Functions in Neural Networks - A simplified explanation of the abstract
- This abstract for appeared for US patent application number 17732372 Titled 'Instruction Set Architecture for Implementing Linear Activation Functions in Neural Networks'
Simplified Explanation
This application describes a method for performing computational operations related to a neural network activation function using a single instruction. The method involves applying a linear activation operation to a set of input elements stored in registers. The linear activation operation is implemented by detecting the sign value of each input element, selecting a scalar value based on the sign value, and applying the linear activation operation using the selected scalar and a bias value. The resulting output elements are then quantized.
Original Abstract Submitted
This application is directed to using a single instruction to initiate a sequence of computational operations related to a neural network activation function. An electronic device receives a single instruction to apply a linear activation operation to a set of first elements stored in one or more input vector registers. In response to the single instruction, the linear activation operation is implemented on the set of first elements to generate a set of output elements. For each first element, the electronic device detects a sign value of the respective first element, selects a respective scalar from one or more scalars based on the sign value, and applies the linear activation operation on the respective first element based on the selected respective scalar and a bias value to generate a respective element of the set of output elements. The electronic device quantizes the set of output elements.
