18392618. APPROXIMATING ACTIVATION FUNCTION IN NEURAL NETWORK WITH LOOK-UP TABLE HAVING HYBRID ARCHITECTURE simplified abstract (Intel Corporation)

APPROXIMATING ACTIVATION FUNCTION IN NEURAL NETWORK WITH LOOK-UP TABLE HAVING HYBRID ARCHITECTURE

Organization Name

Intel Corporation

Inventor(s)

Dinakar Kondru of Frisco TX (US)

Deepak Abraham Mathaikutty of Chandler AZ (US)

Arnab Raha of San Jose CA (US)

Umer Iftikhar Cheema of Hillsboro OR (US)

APPROXIMATING ACTIVATION FUNCTION IN NEURAL NETWORK WITH LOOK-UP TABLE HAVING HYBRID ARCHITECTURE - A simplified explanation of the abstract

This abstract first appeared for US patent application 18392618 titled 'APPROXIMATING ACTIVATION FUNCTION IN NEURAL NETWORK WITH LOOK-UP TABLE HAVING HYBRID ARCHITECTURE

Simplified Explanation

The patent application describes a method for approximating a non-linear activation function using linear functions and storing the parameters in lookup tables for efficient processing in post-processing engines.

• The input range of the activation function is divided into segments based on statistical analysis of input data elements.
• Parameters of linear functions approximating the activation function for selected and unselected input segments are stored in dedicated and shared portions of lookup tables, respectively.

Potential Applications

This technology could be applied in various fields such as artificial intelligence, machine learning, signal processing, and data analytics.

Problems Solved

1. Efficient approximation of non-linear activation functions using linear functions. 2. Optimization of processing in post-processing engines by storing parameters in lookup tables.

Benefits

1. Improved performance and accuracy in processing non-linear activation functions. 2. Reduced computational complexity and memory usage. 3. Scalability for handling large datasets and complex models.

Potential Commercial Applications

"Optimizing Non-linear Activation Functions with Linear Approximations" could find applications in neural networks, deep learning algorithms, image recognition systems, and natural language processing technologies.

Possible Prior Art

One possible prior art could be the use of lookup tables for storing parameters in signal processing applications. Another could be the approximation of non-linear functions using piecewise linear functions in mathematical modeling.

How does this technology compare to existing methods for approximating non-linear activation functions?

This article does not provide a direct comparison with other methods or technologies for approximating non-linear activation functions. Further research or experimentation may be needed to evaluate the performance and efficiency of this approach in comparison to existing methods.

What are the potential limitations or drawbacks of using linear approximations for non-linear activation functions?

The article does not address any potential limitations or drawbacks of using linear approximations for non-linear activation functions. It would be important to investigate factors such as accuracy, precision, and generalizability of the approximations in real-world applications to understand any limitations of this approach.

Original Abstract Submitted

A non-linear activation function may be approximated by linear functions. The input range of the activation function may be divided into input segments. One or more input segments may be selected based on statistical analysis of input data elements in the input range. A parameter of a first linear function that approximates the activation function for at least part of a selected input segment may be stored in a first portion of a first look-up table (LUT). The first portion of the first LUT is dedicated to a first group of post processing engines (PPEs). A parameter of a second linear function that approximates the activation function for at least part of an unselected input segment may be stored in a shared pool of LUT entries, which includes a second portion of the first LUT and a portion of a second LUT and is shared by multiple groups of PPEs.