17525237. METHODS OF OPERATING AN ARTIFICIAL NEURAL NETWORK USING A COMPUTE-IN-MEMORY ACCELERATOR AND A BITWISE ACTIVATION FUNCTION simplified abstract (Samsung Electronics Co., Ltd.)
METHODS OF OPERATING AN ARTIFICIAL NEURAL NETWORK USING A COMPUTE-IN-MEMORY ACCELERATOR AND A BITWISE ACTIVATION FUNCTION
Organization Name
Inventor(s)
YEN-KAI Lin of San Jose CA (US)
YUANCHEN Chu of Sunnyvale CA (US)
WOOSUNG Choi of Milpitas CA (US)
METHODS OF OPERATING AN ARTIFICIAL NEURAL NETWORK USING A COMPUTE-IN-MEMORY ACCELERATOR AND A BITWISE ACTIVATION FUNCTION - A simplified explanation of the abstract
This abstract first appeared for US patent application 17525237 titled 'METHODS OF OPERATING AN ARTIFICIAL NEURAL NETWORK USING A COMPUTE-IN-MEMORY ACCELERATOR AND A BITWISE ACTIVATION FUNCTION
Simplified Explanation
The abstract describes a method for training an artificial neural network using a modified activation function called the bitwise modified rectified linear unit (ReLU). Here are the key points:
- The method involves using an artificial neural network with a compute-in-memory accelerator and a hidden layer of artificial neurons.
- The training process involves applying the bitwise modified ReLU activation function to some of the artificial neurons in the hidden layer.
- The bitwise modified ReLU activation function has two modes: proportional output when the input is below a critical threshold, and independent output when the input is above the critical threshold.
- The input to the activation function is the sum of the products of the outputs from the neurons in the preceding layer and the corresponding weights, with each weight represented by a single bit.
Potential applications of this technology:
- Artificial intelligence and machine learning: The method can be used to train artificial neural networks for various AI and ML applications, such as image recognition, natural language processing, and data analysis.
- Edge computing: The compute-in-memory accelerator can enable efficient processing of neural networks on edge devices, reducing the need for cloud-based processing and improving real-time decision-making capabilities.
- Internet of Things (IoT): The method can be applied to train neural networks for IoT devices, allowing them to perform complex tasks locally without relying on cloud services.
Problems solved by this technology:
- Improved efficiency: The compute-in-memory accelerator and the bitwise modified ReLU activation function help reduce the computational complexity and memory requirements of training artificial neural networks.
- Faster training: The simplified activation function allows for faster training of neural networks, enabling quicker deployment of AI models in various applications.
- Lower power consumption: The compute-in-memory accelerator and efficient training process contribute to lower power consumption, making it suitable for resource-constrained devices.
Benefits of this technology:
- Enhanced performance: The method enables improved performance of artificial neural networks by optimizing the training process and reducing computational overhead.
- Scalability: The compute-in-memory accelerator can be scaled up to handle larger and more complex neural networks, accommodating the growing demands of AI and ML applications.
- Cost-effective: The reduced computational requirements and power consumption make this technology cost-effective, especially for edge devices and IoT applications.
Original Abstract Submitted
A method includes providing an artificial neural network comprising a compute-in-memory accelerator, the artificial neural network further comprising a hidden layer including a first plurality of artificial neurons; and training the artificial neural network using a bitwise modified rectified linear unit activation function for ones of the first plurality of artificial neurons, the bitwise modified rectified linear unit activation function comprising a bit activation function, which is configured to generate an output that is proportional to an input when the input is less than a critical threshold and configured to generate an output that is independent of the input when the input is greater than the critical threshold, wherein the input comprises a sum, across a second plurality of artificial neurons of a preceding layer of the artificial neural network having a plurality of weights associated therewith, respectively, of a product of an output from a respective one of the second plurality of artificial neurons and one bit of a respective one of the plurality of weights.
