18371934. METHODS AND SYSTEMS FOR BUDGETED AND SIMPLIFIED TRAINING OF DEEP NEURAL NETWORKS simplified abstract (Intel Corporation)

METHODS AND SYSTEMS FOR BUDGETED AND SIMPLIFIED TRAINING OF DEEP NEURAL NETWORKS

Organization Name

Intel Corporation

Inventor(s)

Yiwen Guo of Beijing (CN)

Yuqing Hou of Beijing (CN)

Anbang Yao of Beijing (CN)

Dongqi Cai of Beijing (CN)

Lin Xu of Beijing (CN)

Ping Hu of Beijing (CN)

Shandong Wang of Shanghai (CN)

Wenhua Cheng of Shanghai (CN)

Yurong Chen of Beijing (CN)

Libin Wang of Beijing (CN)

METHODS AND SYSTEMS FOR BUDGETED AND SIMPLIFIED TRAINING OF DEEP NEURAL NETWORKS - A simplified explanation of the abstract

This abstract first appeared for US patent application 18371934 titled 'METHODS AND SYSTEMS FOR BUDGETED AND SIMPLIFIED TRAINING OF DEEP NEURAL NETWORKS

Simplified Explanation

The patent application describes methods and systems for budgeted and simplified training of deep neural networks (DNNs). One example involves training a DNN using sub-images derived from down-sampled training images, and testing the trained DNN using sub-images derived from down-sampled testing images. Another example involves a recurrent deep Q-network (RDQN) with a local attention mechanism between a convolutional neural network (CNN) and a long-short time memory (LSTM), where feature maps are generated by the CNN and then subjected to hard and soft attention by the local attention mechanism before being stored in the LSTM for Q value calculation.

• Training DNNs using sub-images from down-sampled training images
• Testing trained DNNs using sub-images from down-sampled testing images
• RDQN with local attention mechanism for feature map selection and Q value calculation

Potential Applications

The technology described in the patent application could be applied in various fields such as image recognition, natural language processing, and autonomous systems.

Problems Solved

1. Simplified and budgeted training of deep neural networks 2. Efficient feature map selection and Q value calculation in recurrent deep Q-networks

Benefits

1. Improved training efficiency for deep neural networks 2. Enhanced performance in tasks requiring attention mechanisms 3. Cost-effective training methods for neural networks

Potential Commercial Applications

Optimizing Training of Deep Neural Networks for Efficient AI Systems

Unanswered Questions

How does the local attention mechanism improve the performance of the recurrent deep Q-network?

The local attention mechanism helps the RDQN focus on relevant features in the input image, leading to more accurate Q value calculations for different actions.

What are the potential limitations of using down-sampled images in training and testing deep neural networks?

Down-sampling images may result in loss of detail and information, which could affect the overall performance of the trained DNNs in certain applications.

Original Abstract Submitted

Methods and systems for budgeted and simplified training of deep neural networks (DNNs) are disclosed. In one example, a trainer is to train a DNN using a plurality of training sub-images derived from a down-sampled training image. A tester is to test the trained DNN using a plurality of testing sub-images derived from a down-sampled testing image. In another example, in a recurrent deep Q-network (RDQN) having a local attention mechanism located between a convolutional neural network (CNN) and a long-short time memory (LSTM), a plurality of feature maps are generated by the CNN from an input image. Hard-attention is applied by the local attention mechanism to the generated plurality of feature maps by selecting a subset of the generated feature maps. Soft attention is applied by the local attention mechanism to the selected subset of generated feature maps by providing weights to the selected subset of generated feature maps in obtaining weighted feature maps. The weighted feature maps are stored in the LSTM. A Q value is calculated for different actions based on the weighted feature maps stored in the LSTM.