International business machines corporation (20240232610). DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE simplified abstract

From WikiPatents
Jump to navigation Jump to search

DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE

Organization Name

international business machines corporation

Inventor(s)

Malte Johannes Rasch of Chappaqua NY (US)

DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240232610 titled 'DNN TRAINING ALGORITHM WITH DYNAMICALLY COMPUTED ZERO-REFERENCE

The abstract describes a computer-implemented method for updating the weights of a deep neural network using resistive processing units (RPU) and digital storage.

  • Gradient updates are performed for stochastic gradient descent of a deep neural network.
  • The first set of hidden weights is stored in a resistive processing unit (RPU) crossbar array.
  • A second set of hidden weights is stored in a digital medium.
  • A third matrix of reference values is computed during a transfer cycle of weights from the RPU to the digital medium.
  • The third matrix is stored in the digital medium.
  • A third set of weights is updated from the second matrix when a threshold is reached for the second set of weights.

Potential Applications: - This technology can be applied in various fields such as artificial intelligence, machine learning, and data analysis. - It can be used in developing more efficient and accurate deep neural networks for tasks like image recognition, natural language processing, and autonomous driving.

Problems Solved: - Addresses the need for efficient weight updates in deep neural networks. - Improves the performance and accuracy of neural network models.

Benefits: - Faster and more efficient training of deep neural networks. - Enhanced accuracy and performance of machine learning models. - Reduced energy consumption in training neural networks.

Commercial Applications: - This technology has commercial applications in industries such as healthcare, finance, e-commerce, and cybersecurity. - It can be used to develop advanced AI systems for personalized medicine, fraud detection, recommendation systems, and more.

Questions about the technology: 1. How does this technology improve the efficiency of deep neural network training? 2. What are the potential implications of using resistive processing units (RPU) in updating neural network weights?


Original Abstract Submitted

a computer implemented method includes performing a gradient update for a stochastic gradient descent (sgd) of a deep neural network (dnn) using a first set of hidden weights stored in a first matrix comprising a resistive processing unit (rpu) crossbar array. a second matrix comprising a second set of hidden weights is stored in a digital medium. a third matrix comprising a set of reference values is computed upon a transfer cycle of the first set of weights from the first matrix to the second matrix, accounting for a sign-change (chopper). the third matrix is stored in the digital medium. a third set of weights is updated for the dnn from the second matrix when a threshold is reached for the second set of weights, in a fourth matrix comprising a rpu crossbar array.

Retrieved from "http://wikipatents.org/index.php?title=International_business_machines_corporation_(20240232610)._DNN_TRAINING_ALGORITHM_WITH_DYNAMICALLY_COMPUTED_ZERO-REFERENCE_simplified_abstract&oldid=92131"