17875594. OneShot Neural Architecture and Hardware Architecture Search simplified abstract (GOOGLE LLC)

OneShot Neural Architecture and Hardware Architecture Search

Organization Name

GOOGLE LLC

Inventor(s)

Sheng Li of Cupertino CA (US)

Norman Paul Jouppi of Palo Alto CA (US)

Garrett Axel Andersen of Austin TX (US)

Quoc V. Le of Sunnyvale CA (US)

Liqun Cheng of Palo Alto CA (US)

Parthasarathy Ranganathan of San Jose CA (US)

OneShot Neural Architecture and Hardware Architecture Search - A simplified explanation of the abstract

This abstract first appeared for US patent application 17875594 titled 'OneShot Neural Architecture and Hardware Architecture Search

Simplified Explanation

The disclosed patent application focuses on jointly searching machine learning model architectures and hardware architectures in a combined space of models, hardware, and mapping strategies. This is achieved through a search strategy that evaluates all models, hardware, and mappings together using weight sharing and a supernetwork. A multi-objective reward function is utilized with objectives for quality, performance, power, and area.

• The patent application proposes a method for simultaneously searching machine learning model architectures and hardware architectures.
• The method involves evaluating all possible combinations of models, hardware, and mappings together.
• Weight sharing and a supernetwork are used to efficiently evaluate the combinations.
• A multi-objective reward function is employed to optimize for quality, performance, power, and area.
• The disclosed approach aims to find the best combination of model architecture, hardware architecture, and mapping strategy.

Potential Applications:

• This technology can be applied in various fields that utilize machine learning, such as computer vision, natural language processing, and speech recognition.
• It can be used to optimize the performance and efficiency of machine learning systems in applications like autonomous vehicles, robotics, and smart devices.
• The approach can be employed in cloud computing environments to enhance the efficiency and scalability of machine learning models.

Problems Solved:

• The disclosed technology addresses the challenge of finding the optimal combination of machine learning model architectures and hardware architectures.
• It solves the problem of separately optimizing models and hardware, by jointly searching them in a combined space.
• The approach tackles the trade-off between model quality, performance, power consumption, and area requirements.

Benefits:

• The technology enables the discovery of highly optimized machine learning systems with improved performance, efficiency, and accuracy.
• It allows for the efficient exploration of a large search space, leading to better model and hardware architectures.
• The approach can reduce the time and resources required for manually designing and optimizing machine learning systems.
• By considering multiple objectives, it provides a balanced optimization solution that considers various trade-offs.

Original Abstract Submitted

Aspects of the disclosure are directed to jointly searching machine learning model architectures and hardware architectures in a combined space of models, hardware, and mapping strategies. A search strategy is utilized where all models, hardware, and mappings are evaluated together at once via weight sharing and a supernetwork. A multi-objective reward function is utilized with objectives for quality, performance, power, and area.