18571616. APPARATUS, METHOD, DEVICE AND MEDIUM FOR LOSS BALANCING IN MULTI-TASK LEARNING simplified abstract (Intel Corporation)
Contents
APPARATUS, METHOD, DEVICE AND MEDIUM FOR LOSS BALANCING IN MULTI-TASK LEARNING
Organization Name
Inventor(s)
APPARATUS, METHOD, DEVICE AND MEDIUM FOR LOSS BALANCING IN MULTI-TASK LEARNING - A simplified explanation of the abstract
This abstract first appeared for US patent application 18571616 titled 'APPARATUS, METHOD, DEVICE AND MEDIUM FOR LOSS BALANCING IN MULTI-TASK LEARNING
The disclosure presents a method for loss balancing in Multi-Task Learning (MTL) using a deep neural network.
- The apparatus includes interface circuitry and processor circuitry.
- The processor circuitry initializes parameters of shared layers of a deep neural network for MTL using a pre-trained neural network.
- It determines a custom interval with a designated number of mini-batch training steps and a window of N custom intervals (N>2).
- For each task, it calculates a loss change rate and gradient magnitude with respect to selected shared weights within the designated window.
- The weight of the task is adjusted based on the calculated loss change rate and gradient magnitude with respect to selected shared weights.
Potential Applications: - This technology can be applied in various fields such as natural language processing, computer vision, and speech recognition. - It can be used in autonomous vehicles for real-time decision-making based on multiple tasks simultaneously.
Problems Solved: - Addresses the challenge of balancing loss in Multi-Task Learning to improve the performance of deep neural networks. - Helps in optimizing the shared weights of tasks in a deep neural network for efficient training.
Benefits: - Enhanced performance and accuracy in multi-task learning scenarios. - Improved efficiency in training deep neural networks with shared layers.
Commercial Applications: Title: "Enhancing Multi-Task Learning Efficiency with Loss Balancing Technology" This technology can be utilized in industries such as healthcare for medical image analysis, finance for fraud detection, and e-commerce for personalized recommendation systems.
Prior Art: Researchers can explore prior studies on loss balancing techniques in deep neural networks and multi-task learning to understand the evolution of this technology.
Frequently Updated Research: Stay updated on the latest advancements in loss balancing methods for deep neural networks and multi-task learning to incorporate cutting-edge techniques into applications.
Questions about Loss Balancing in MTL: 1. How does loss balancing in Multi-Task Learning contribute to the overall performance of deep neural networks? 2. What are the key challenges in implementing loss balancing techniques in complex multi-task learning scenarios?
By addressing these questions, researchers can gain a deeper understanding of the significance and implications of loss balancing technology in Multi-Task Learning.
Original Abstract Submitted
The disclosure provides an apparatus, method, device, and medium for loss balancing in MTL. The apparatus includes interface circuitry and processor circuitry. The processor circuitry is configured to initialize parameters of shared layers of a deep neural network for MTL using a pre-trained neural network; determine a custom interval consisting of a designated number of mini-batch training steps and a designated window of N custom intervals (N>2); for each task, calculate a loss change rate between each pair of N−1 pairs of neighboring custom intervals within a designated window prior to a present custom interval and a gradient magnitude with respect to selected shared weights within the designated window prior to the present custom interval, and adjust, a weight of the task, based on the calculated loss change rate and gradient magnitude with respect to selected shared weights.
