Microsoft technology licensing, llc (20240126993). TRANSFORMER-BASED TEXT ENCODER FOR PASSAGE RETRIEVAL simplified abstract
Contents
- 1 TRANSFORMER-BASED TEXT ENCODER FOR PASSAGE RETRIEVAL
- 1.1 Organization Name
- 1.2 Inventor(s)
- 1.3 TRANSFORMER-BASED TEXT ENCODER FOR PASSAGE RETRIEVAL - A simplified explanation of the abstract
- 1.4 Simplified Explanation
- 1.5 Potential Applications
- 1.6 Problems Solved
- 1.7 Benefits
- 1.8 Potential Commercial Applications
- 1.9 Possible Prior Art
- 1.10 Original Abstract Submitted
TRANSFORMER-BASED TEXT ENCODER FOR PASSAGE RETRIEVAL
Organization Name
microsoft technology licensing, llc
Inventor(s)
Xiaodong Liu of Redmond WA (US)
Jianfeng Gao of Woodinville WA (US)
TRANSFORMER-BASED TEXT ENCODER FOR PASSAGE RETRIEVAL - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240126993 titled 'TRANSFORMER-BASED TEXT ENCODER FOR PASSAGE RETRIEVAL
Simplified Explanation
The computing system described in the abstract includes a transformer-based text encoder with multiple transformer blocks trained to encode computer-readable text representations of input text questions and passages. The encoder includes a shared transformer block for both question and passage representations, as well as a specialized transformer block with input-specific subnetworks and a routing function to select the appropriate subnetwork for each text representation.
- Transformer-based text encoder with multiple transformer blocks
- Shared transformer block for question and passage representations
- Specialized transformer block with input-specific subnetworks and routing function
- Selection of input-specific subnetwork for each text representation
Potential Applications
The technology described in this patent application could be applied in various fields such as natural language processing, question-answering systems, information retrieval, and text summarization.
Problems Solved
This technology addresses the challenge of efficiently encoding and processing text data, especially in tasks involving understanding and generating responses to questions based on given passages.
Benefits
The transformer-based text encoder offers improved performance in processing and analyzing text data, leading to more accurate and efficient results in tasks such as question-answering and information retrieval.
Potential Commercial Applications
Potential commercial applications of this technology include developing advanced chatbots, search engines, content recommendation systems, and automated customer support services.
Possible Prior Art
One potential prior art for this technology could be the original transformer model introduced by Vaswani et al. in the paper "Attention is All You Need" published in 2017.
What are the specific components of the specialized transformer block in the encoder described in the abstract?
The specialized transformer block includes two or more input-specific subnetworks and a routing function to select the appropriate subnetwork for each computer-readable text representation.
The shared transformer block is trained to handle both question and passage representations, while the specialized transformer block contains input-specific subnetworks and a routing function for selecting the subnetwork based on the input text representation.
Original Abstract Submitted
a computing system includes a logic subsystem and a storage subsystem holding instructions executable by the logic subsystem to implement a transformer-based text encoder. the transformer-based text encoder includes a plurality of transformer blocks previously-trained to apply encoding operations to computer-readable text representations of input text strings, the computer-readable text representations including computer-readable question representations of input text questions, and computer-readable passage representations of input text passages. the plurality of transformer blocks include a shared transformer block trained for both the computer-readable question representations and the computer-readable passage representations and a specialized transformer block including two or more input-specific subnetworks, and a routing function to select an input-specific subnetwork of the two or more input-specific subnetworks for each of the computer-readable text representations.
