18048436. DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE simplified abstract (INTERNATIONAL BUSINESS MACHINES CORPORATION)

DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE

Organization Name

INTERNATIONAL BUSINESS MACHINES CORPORATION

Inventor(s)

Malte Johannes Rasch of Chappaqua NY (US)

DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE - A simplified explanation of the abstract

This abstract first appeared for US patent application 18048436 titled 'DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE

Simplified Explanation

The abstract describes a method for updating weights in a deep neural network using a Resistive Processing Unit (RPU) crossbar array and a digital medium.

• The method involves performing a gradient update for stochastic gradient descent using hidden weights stored in an RPU crossbar array.
• A second set of hidden weights is stored in a digital medium.
• Reference values are computed and stored in the digital medium after transferring the first set of weights from the RPU crossbar array.
• When a threshold is reached for the second set of weights, a third set of weights is updated for the DNN from the digital medium.

Potential Applications

This technology could be applied in various fields such as artificial intelligence, machine learning, and data analysis.

Problems Solved

This technology helps in efficiently updating weights in deep neural networks, improving the performance and accuracy of the models.

Benefits

The use of RPU crossbar arrays and digital mediums allows for faster and more energy-efficient weight updates in deep neural networks.

Potential Commercial Applications

Potential commercial applications of this technology include in industries such as healthcare, finance, and autonomous vehicles for advanced data processing and decision-making.

Possible Prior Art

Prior art may include research on hardware acceleration for deep learning algorithms and techniques for optimizing neural network training processes.

What are the specific technical details of the RPU crossbar array used in this method?

The specific technical details of the RPU crossbar array, such as the size, structure, and material composition, are not provided in the abstract. Further information on the design and implementation of the RPU crossbar array would be necessary to understand its full capabilities and limitations.

How does the method ensure the accuracy and reliability of the weight updates in the deep neural network?

The abstract does not mention specific mechanisms or algorithms used to ensure the accuracy and reliability of the weight updates in the deep neural network. Additional details on error correction, validation processes, or quality control measures would be needed to address this aspect of the technology.

Original Abstract Submitted

A computer implemented method includes performing a gradient update for a stochastic gradient descent (SGD) of a deep neural network (DNN) using a first set of hidden weights stored in a first matrix comprising a Resistive Processing Unit (RPU) crossbar array. A second matrix comprising a second set of hidden weights is stored in a digital medium. A third matrix comprising a set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a sign-change (chopper). The third matrix is stored in the digital medium. A third set of weights is updated for the DNN from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a RPU crossbar array.