20240013094. PROMPT AUGMENTED GENERATIVE REPLAY VIA SUPERVISED CONTRASTIVE TRAINING FOR LIFELONG INTENT DETECTION simplified abstract (Tata Consultancy Services Limited)
PROMPT AUGMENTED GENERATIVE REPLAY VIA SUPERVISED CONTRASTIVE TRAINING FOR LIFELONG INTENT DETECTION
Organization Name
Tata Consultancy Services Limited
Inventor(s)
Vaibhav Varshney of Noida (IN)
PROMPT AUGMENTED GENERATIVE REPLAY VIA SUPERVISED CONTRASTIVE TRAINING FOR LIFELONG INTENT DETECTION - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240013094 titled 'PROMPT AUGMENTED GENERATIVE REPLAY VIA SUPERVISED CONTRASTIVE TRAINING FOR LIFELONG INTENT DETECTION
Simplified Explanation
The disclosed embodiments relate to a method for lifelong intent detection, where new intents are added incrementally. To address the issue of catastrophic forgetting, an incremental learner is provided with prompt augmented generative replay. This approach stores concept words obtained from old intents instead of real samples, reducing memory consumption and speeding up training while still preserving knowledge of old intents. The joint training of the incremental learner includes lid and pseudo-labeled utterance generation, aiming to classify user utterances into pre-defined intents by minimizing a total loss function.
- The method models lifelong intent detection as class-incremental learning.
- Prompt augmented generative replay is used to mitigate catastrophic forgetting.
- Only concept words from old intents are stored, reducing memory consumption.
- Joint training of an incremental learner is carried out for lid and utterance generation.
- The total loss function includes lid loss, labeled utterance generation loss, supervised contrastive training loss, and knowledge distillation loss.
Potential applications of this technology:
- Natural language processing systems
- Virtual assistants and chatbots
- Voice-controlled devices
- Customer service automation
- Language understanding in various domains
Problems solved by this technology:
- Catastrophic forgetting during lifelong intent detection
- Memory consumption and training speed in incremental learning
- Classification of user utterances into pre-defined intents
Benefits of this technology:
- Improved performance in lifelong intent detection
- Reduced memory consumption and faster training
- Preserves knowledge of old intents while adding new ones
- Enables accurate classification of user utterances
Original Abstract Submitted
embodiments disclosed herein model lifelong intent detection as a class-incremental learning where a new set of intents/classes are added at each incremental step. to address the issue of catastrophic forgetting during lifelong intent detection (lid), an incremental learner is provided with prompt augmented generative replay, wherein unlike existing approaches that store real samples in replay memory, only concept words obtained from old intents are stored, which reduces memory consumption and speeds up incremental training still enabling not forgetting the old intents. joint training of an incremental learner is carried out for lid and a pseudo-labeled utterance generation with objective is to classify a user utterance into one of multiple pre-defined intents by minimizing a total loss function comprising a lid loss function, a labeled utterance generation loss function, a supervised contrastive training loss function, and a knowledge distillation loss function.
