Google llc (20240135187). Method for Training Large Language Models to Perform Query Intent Classification simplified abstract

Method for Training Large Language Models to Perform Query Intent Classification

Organization Name

google llc

Inventor(s)

Krishna Pragash Srinivasan of Union City CA (US)

Michael Bendersky of Cupertino CA (US)

Anupam Samanta of Mountain View CA (US)

Lingrui Liao of Foster City CA (US)

Luca Bertelli of Redwood City CA (US)

Ming-Wei Chang of Redmond WA (US)

Iftekhar Naim of San Jose CA (US)

Siddhartha Brahma of San Jose CA (US)

Siamak Shakeri of New York NY (US)

Hongkun Yu of Redwood City CA (US)

John Nham of Fremont CA (US)

Karthik Raman of Sunnyvale CA (US)

Raphael Dominik Hoffmann of Los Altos CA (US)

Method for Training Large Language Models to Perform Query Intent Classification - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240135187 titled 'Method for Training Large Language Models to Perform Query Intent Classification

Simplified Explanation

The patent application describes a method for training query processing models to perform query intent classification tasks using retrieval augmentation and multi-stage distillation.

• Query processing models are trained to classify query intent tasks by using retrieval augmentation and multi-stage distillation.
• Unlabeled training examples of queries are obtained and augmented with additional feature annotations to generate augmented training examples.
• A first query processing model annotates the retrieval augmented queries to generate inferred labels for the augmented training examples.
• A second query processing model is trained on the inferred labels, distilling the model trained with retrieval augmentation into a non-retrieval augmented model.
• The second model annotates the entire set of unlabeled training examples.
• Another stage of distillation trains a third query processing model using the entire set of unlabeled training examples without retrieval augmentation.

Potential Applications

The technology described in the patent application could be applied in various fields such as search engines, e-commerce platforms, and customer service chatbots to improve query processing and intent classification tasks.

Problems Solved

This technology helps in improving the accuracy and efficiency of query processing models in classifying query intent tasks, leading to better search results and user experience.

Benefits

The benefits of this technology include enhanced performance of query processing models, increased accuracy in query intent classification, and improved user satisfaction with search results.

Potential Commercial Applications

The technology could be commercially applied in search engine optimization tools, online advertising platforms, and customer support systems to enhance query processing capabilities and improve user engagement.

Possible Prior Art

One possible prior art for this technology could be the use of machine learning algorithms in query processing and intent classification tasks in search engines and natural language processing systems.

How does this technology compare to existing query processing models in terms of accuracy and efficiency?

This article does not provide a direct comparison with existing query processing models to evaluate the performance improvements achieved by the proposed method.

What are the potential limitations or challenges in implementing this technology in real-world applications?

The article does not address any potential obstacles or difficulties that may arise when implementing this technology in practical settings, such as data privacy concerns or computational resource requirements.

Original Abstract Submitted

provided are computing systems, methods, and platforms that train query processing models, such as large language models, to perform query intent classification tasks by using retrieval augmentation and multi-stage distillation. unlabeled training examples of queries may be obtained, and a set of the training examples may be augmented with additional feature annotations to generate augmented training examples. a first query processing model may annotate the retrieval augmented queries to generate inferred labels for the augmented training examples. a second query processing model may be trained on the inferred labels, distilling the query processing model that was trained with retrieval augmentation into a non-retrieval augmented query processing model. the second query processing model may annotate the entire set of unlabeled training examples. another stage of distillation may train a third query processing model using the entire set of unlabeled training examples without retrieval augmentation.