17986803. TRUST-REGION AWARE NEURAL NETWORK ARCHITECTURE SEARCH FOR KNOWLEDGE DISTILLATION simplified abstract (QUALCOMM Incorporated)

From WikiPatents
Jump to navigation Jump to search

TRUST-REGION AWARE NEURAL NETWORK ARCHITECTURE SEARCH FOR KNOWLEDGE DISTILLATION

Organization Name

QUALCOMM Incorporated

Inventor(s)

Taehyeon Kim of Seoul (KR)

Heesoo Myeong of Seoul (KR)

TRUST-REGION AWARE NEURAL NETWORK ARCHITECTURE SEARCH FOR KNOWLEDGE DISTILLATION - A simplified explanation of the abstract

This abstract first appeared for US patent application 17986803 titled 'TRUST-REGION AWARE NEURAL NETWORK ARCHITECTURE SEARCH FOR KNOWLEDGE DISTILLATION

Simplified Explanation

The patent application describes a method for searching and selecting a neural network architecture for knowledge distillation using a trust-region Bayesian optimization technique.

  • The method involves defining a search space that includes various convolutional and transformer operators for student neural network architectures.
  • The trust-region Bayesian optimization is used to select the most suitable student neural network architecture based on a pre-defined teacher model.
  • The innovation allows for efficient and effective knowledge distillation in neural networks.

Potential Applications

This technology has potential applications in various fields that utilize neural networks, including:

  • Computer vision: Enhancing image recognition and object detection algorithms.
  • Natural language processing: Improving language translation and sentiment analysis models.
  • Speech recognition: Enhancing voice recognition and speech-to-text systems.

Problems Solved

The technology addresses the following problems:

  • Finding the optimal neural network architecture for knowledge distillation can be time-consuming and resource-intensive.
  • Existing methods may not effectively explore the search space of possible architectures.
  • The selection of a suitable student neural network architecture is crucial for successful knowledge distillation.

Benefits

The technology offers the following benefits:

  • Efficient search and selection of neural network architectures for knowledge distillation.
  • Improved performance and accuracy of student neural networks through optimized architecture selection.
  • Reduction in the time and resources required for knowledge distillation.


Original Abstract Submitted

A processor-implemented method of searching for a neural network architecture includes defining a search space of student neural network architectures for knowledge distillation. The search space includes multiple convolutional operators and multiple transformer operators. A trust-region Bayesian optimization is performed to select a student neural network architecture from the search space based on a pre-defined teacher model.

Retrieved from "http://wikipatents.org/index.php?title=17986803._TRUST-REGION_AWARE_NEURAL_NETWORK_ARCHITECTURE_SEARCH_FOR_KNOWLEDGE_DISTILLATION_simplified_abstract_(QUALCOMM_Incorporated)&oldid=23294"