18150748. NEURAL NETWORK BUILDING METHOD AND APPARATUS simplified abstract (Huawei Technologies Co., Ltd.)
Contents
NEURAL NETWORK BUILDING METHOD AND APPARATUS
Organization Name
Inventor(s)
Weinan Zhang of Shanhghai (CN)
NEURAL NETWORK BUILDING METHOD AND APPARATUS - A simplified explanation of the abstract
This abstract first appeared for US patent application 18150748 titled 'NEURAL NETWORK BUILDING METHOD AND APPARATUS
Simplified Explanation
The abstract describes a method and apparatus for building neural networks in the field of artificial intelligence. The method involves initializing a search space and a set of building blocks, where the search space consists of different operators, and the building blocks are network structures formed by connecting nodes using these operators. During training, some operators are randomly discarded in each training round, and the building blocks are updated using the remaining operators. Finally, a target neural network is built based on the updated building blocks. This approach breaks the association between operators and addresses the problem of co-adaptation during training, resulting in a neural network with improved performance.
- Neural network building method and apparatus in the field of artificial intelligence
- Initialization of a search space and a set of building blocks
- Search space consists of different operators
- Building blocks are network structures formed by connecting nodes using operators
- Randomly discarding some operators during training rounds
- Updating building blocks using the remaining operators
- Building a target neural network based on the updated building blocks
- Overcoming co-adaptation problem during training
- Obtaining a neural network with better performance
Potential Applications
- Artificial intelligence research and development
- Machine learning applications
- Pattern recognition systems
- Natural language processing
- Computer vision systems
Problems Solved
- Co-adaptation problem during training of neural networks
- Difficulty in finding optimal network structures
- Over-reliance on specific operators leading to suboptimal performance
Benefits
- Improved performance of neural networks
- Increased flexibility in network structure design
- Enhanced adaptability to different tasks and datasets
- Faster convergence during training process
- Reduction in overfitting and generalization errors
Original Abstract Submitted
A neural network building method and apparatus are disclosed, and relate to the field of artificial intelligence. The method includes: initializing a search space and a plurality of building blocks, where the search space includes a plurality of operators, and the building block is a network structure obtained by connecting a plurality of nodes by using the operator; during training, in at least one training round, randomly discarding some operators, and updating the plurality of building blocks by using operators that are not discarded; and building a target neural network based on the plurality of updated building blocks. In the method, some operators are randomly discarded. This breaks association between operators, and overcomes a co-adaptation problem during training, to obtain a target neural network with better performance.
