20240028905. ARTIFICIAL NEURAL NETWORK TRAINING USING FLEXIBLE FLOATING POINT TENSORS simplified abstract (Intel Corporation)

From WikiPatents
Jump to navigation Jump to search

ARTIFICIAL NEURAL NETWORK TRAINING USING FLEXIBLE FLOATING POINT TENSORS

Organization Name

Intel Corporation

Inventor(s)

Krishnakumar Nair of Santa Clara CA (US)

Andrew Yang of Cupertino CA (US)

Brian Morris of Santa Clara CA (US)

ARTIFICIAL NEURAL NETWORK TRAINING USING FLEXIBLE FLOATING POINT TENSORS - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240028905 titled 'ARTIFICIAL NEURAL NETWORK TRAINING USING FLEXIBLE FLOATING POINT TENSORS

Simplified Explanation

The present disclosure is about systems and methods for training neural networks using a tensor that contains multiple fp16 values and a set of bits defining a shared exponent for some or all of the fp16 values in the tensor. The fp16 values are in the IEEE 754 format, which is a 16-bit floating point format. The tensor also includes bits that define the shared exponent.

  • The patent/application is about training neural networks using a tensor with fp16 values and a shared exponent.
  • The fp16 values are in the IEEE 754 format, a 16-bit floating point format.
  • The tensor includes bits that define the shared exponent.
  • The tensor can have a shared exponent and fp16 values with variable bit-length mantissa and exponent.
  • The processor circuitry can dynamically set the bit-length of the mantissa and exponent.
  • The processor circuitry can set a shared exponent switch to selectively combine the fp16 value exponent with the shared exponent.

Potential applications of this technology:

  • Training neural networks: The systems and methods described in the patent/application can be used to train neural networks more efficiently by using fp16 values and a shared exponent in the tensor. This can potentially reduce memory requirements and computational complexity.

Problems solved by this technology:

  • Memory efficiency: By using fp16 values and a shared exponent, the patent/application addresses the problem of memory consumption in training neural networks. This can be particularly beneficial when dealing with large-scale neural networks that require significant memory resources.

Benefits of this technology:

  • Reduced memory requirements: The use of fp16 values and a shared exponent in the tensor can reduce the memory footprint of neural network training, allowing for more efficient use of available resources.
  • Improved computational efficiency: By optimizing the representation of fp16 values and utilizing a shared exponent, the patent/application can potentially improve the computational efficiency of training neural networks, leading to faster training times.


Original Abstract Submitted

thus, the present disclosure is directed to systems and methods for training neural networks using a tensor that includes a plurality of fp16 values and a plurality of bits that define an exponent shared by some or all of the fp16 values included in the tensor. the fp16 values may include ieee 754 format 16-bit floating point values and the tensor may include a plurality of bits defining the shared exponent. the tensor may include a shared exponent and fp16 values that include a variable bit-length mantissa and a variable bit-length exponent that may be dynamically set by processor circuitry. the tensor may include a shared exponent and fp16 values that include a variable bit-length mantissa; a variable bit-length exponent that may be dynamically set by processor circuitry; and a shared exponent switch set by the processor circuitry to selectively combine the fp16 value exponent with the shared exponent.

Retrieved from "http://wikipatents.org/index.php?title=20240028905._ARTIFICIAL_NEURAL_NETWORK_TRAINING_USING_FLEXIBLE_FLOATING_POINT_TENSORS_simplified_abstract_(Intel_Corporation)&oldid=29832"