Google llc (20240347043). Robustness Aware Norm Decay for Quantization Aware Training and Generalization simplified abstract
Contents
Robustness Aware Norm Decay for Quantization Aware Training and Generalization
Organization Name
Inventor(s)
David Rim of Mountain View CA (US)
Shaojin Ding of Mountain View CA (US)
Yanzhang He of Mountain View CA (US)
Robustness Aware Norm Decay for Quantization Aware Training and Generalization - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240347043 titled 'Robustness Aware Norm Decay for Quantization Aware Training and Generalization
The patent application describes a method for training an automatic speech recognition (ASR) model using random noise and quantization techniques.
- Obtaining a plurality of training samples for the ASR model.
- Determining a minimum integer fixed-bit width to represent the maximum quantization of the ASR model.
- Training the ASR model on the training samples with random noise.
- Quantizing the weights of the ASR model to a target integer fixed-bit width after training.
- Providing the quantized trained ASR model to a user device.
Potential Applications: - Improving the efficiency and accuracy of automatic speech recognition systems. - Enhancing the performance of ASR models in various applications such as virtual assistants, transcription services, and voice-controlled devices.
Problems Solved: - Addressing the challenge of optimizing the quantization of weights in ASR models. - Improving the robustness and computational efficiency of ASR systems.
Benefits: - Increased accuracy and reliability of speech recognition. - Reduced computational resources required for ASR models. - Enhanced user experience with voice-enabled technologies.
Commercial Applications: Title: "Enhanced Automatic Speech Recognition Technology for Improved Performance" This technology can be utilized in industries such as telecommunications, customer service, healthcare, and automotive for developing advanced speech recognition solutions that offer superior performance and user experience.
Questions about the technology: 1. How does the use of random noise during training improve the performance of the ASR model? 2. What are the implications of quantizing weights to a target integer fixed-bit width on the efficiency of the ASR system?
Original Abstract Submitted
a method includes obtaining a plurality of training samples, determining a minimum integer fixed-bit width representing a maximum quantization of an automatic speech recognition (asr) model, and training the asr model on the plurality of training samples using a quantity of random noise. the asr model includes a plurality of weights that each include a respective float value. the quantity of random noise is based on the minimum integer fixed-bit value. after training the asr model, the method also includes selecting a target integer fixed-bit width greater than or equal to the minimum integer fixed-bit width, and for each respective weight of the plurality of weights, quantizing the respective weight from the respective float value to a respective integer associated with a value of the selected target integer fixed-bit width. the operations also include providing the quantized trained asr model to a user device.
