17469853. SHRINK DARTS simplified abstract (Samsung Electronics Co., Ltd.)
Contents
SHRINK DARTS
Organization Name
Inventor(s)
Jun Fang of Santa Clara CA (US)
David Philip Lloyd Thorsley of Morgan Hill CA (US)
Chengyao Shen of San Jose CA (US)
Joseph H. Hassoun of Los Gatos CA (US)
SHRINK DARTS - A simplified explanation of the abstract
This abstract first appeared for US patent application 17469853 titled 'SHRINK DARTS
Simplified Explanation
The patent application describes a method for reducing computation in a differentiable architecture search for neural networks. The method involves forming an output node with a smaller channel dimension by averaging the channel outputs of intermediate nodes in a normal cell. The output node is then preprocessed using a 1×1 convolution to create input nodes for the next layer of cells in the neural network architecture. The formation of the output node includes dividing the channel outputs of the intermediate nodes into groups and forming an average channel output for each group. The output node is formed by concatenating the average channel outputs with the channel outputs of the intermediate nodes.
- The method reduces computation in a differentiable architecture search for neural networks.
- It forms an output node with a smaller channel dimension by averaging the channel outputs of intermediate nodes.
- The output node is preprocessed using a 1×1 convolution to create input nodes for the next layer of cells.
- The channel outputs of the intermediate nodes are divided into groups and an average channel output is formed for each group.
- The average channel outputs are concatenated with the channel outputs of the intermediate nodes to form the output node.
Potential Applications
- Neural network architecture search
- Deep learning models
- Computer vision tasks
- Natural language processing tasks
Problems Solved
- Reduces computation in differentiable architecture search
- Addresses the challenge of high computational complexity in neural network architecture search
Benefits
- Faster and more efficient architecture search
- Reduced computational resources required
- Improved performance of deep learning models
- Enables faster development of computer vision and natural language processing applications
Original Abstract Submitted
A method is disclosed for reducing computation of a differentiable architecture search. An output node is formed having a channel dimension that is one-fourth of a channel dimension of a normal cell of a neural network architecture by averaging channel outputs of intermediate nodes of the normal cell. The output node is preprocessed using a 1×1 convolution to form channels of input nodes for a next layer of the cells in the neural network architecture. Forming the output node includes forming s groups of channel outputs of the intermediate nodes by dividing the channel outputs of the intermediate nodes by a splitting parameter s. An average channel output for each group of channel outputs is formed, and the output node is formed by concatenating the average channel output for each group of channels with channel outputs of the intermediate nodes of the normal cell.
