Google llc (20240185065). TRAINING TEXT SUMMARIZATION NEURAL NETWORKS WITH AN EXTRACTED SEGMENTS PREDICTION OBJECTIVE simplified abstract
Contents
- 1 TRAINING TEXT SUMMARIZATION NEURAL NETWORKS WITH AN EXTRACTED SEGMENTS PREDICTION OBJECTIVE
- 1.1 Organization Name
- 1.2 Inventor(s)
- 1.3 TRAINING TEXT SUMMARIZATION NEURAL NETWORKS WITH AN EXTRACTED SEGMENTS PREDICTION OBJECTIVE - A simplified explanation of the abstract
- 1.4 Simplified Explanation
- 1.5 Potential Applications
- 1.6 Problems Solved
- 1.7 Benefits
- 1.8 Potential Commercial Applications
- 1.9 Possible Prior Art
- 1.10 Original Abstract Submitted
TRAINING TEXT SUMMARIZATION NEURAL NETWORKS WITH AN EXTRACTED SEGMENTS PREDICTION OBJECTIVE
Organization Name
Inventor(s)
Mohammad Saleh of Santa Clara CA (US)
Yao Zhao of San Carlos CA (US)
Peter J. Liu of Santa Clara CA (US)
TRAINING TEXT SUMMARIZATION NEURAL NETWORKS WITH AN EXTRACTED SEGMENTS PREDICTION OBJECTIVE - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240185065 titled 'TRAINING TEXT SUMMARIZATION NEURAL NETWORKS WITH AN EXTRACTED SEGMENTS PREDICTION OBJECTIVE
Simplified Explanation
The patent application describes methods, systems, and apparatus for training a text summarization neural network using self-supervised learning and labeled data.
- Pre-training the text summarization neural network involves learning values of network parameters through self-supervised learning using unlabeled data.
- The pre-training process includes obtaining an unlabeled text, selecting segments, processing a masked text to generate predictions, and updating network parameters based on the prediction difference.
- Adapting the pre-trained network for a specific text summarization task is done using labeled data containing texts and summaries.
Potential Applications
The technology can be applied in various fields such as natural language processing, content generation, and information retrieval.
Problems Solved
This technology helps in improving the efficiency and accuracy of text summarization tasks by pre-training the neural network with self-supervised learning.
Benefits
The benefits of this technology include enhanced text summarization capabilities, improved performance, and adaptability to different summarization tasks.
Potential Commercial Applications
Potential commercial applications of this technology include automated content summarization tools, news aggregation platforms, and document summarization software.
Possible Prior Art
One possible prior art in this field is the use of supervised learning for training text summarization neural networks. However, the use of self-supervised learning in pre-training is a novel approach.
What are the advantages of using self-supervised learning for pre-training a text summarization neural network?
Using self-supervised learning allows the network to learn from unlabeled data, which can be more abundant and cost-effective compared to labeled data. This approach can also help in capturing more nuanced patterns and structures in the text.
How does adapting the pre-trained network with labeled data improve its performance for specific tasks?
Adapting the pre-trained network with labeled data fine-tunes the network parameters to better suit the requirements of a specific text summarization task. This process helps in optimizing the network for the task at hand, leading to improved summarization accuracy and efficiency.
Original Abstract Submitted
methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a text summarization neural network. one of the methods includes pre-training the text summarization neural network including learning values of a plurality of network parameters through self-supervised learning using unlabeled data comprising unlabeled first texts, the pre-training including: obtaining an unlabeled first text comprising a plurality of segments; selecting one or more of the plurality of segments; processing a masked first text that excludes the one or more selected segments to generate a prediction of the one or more selected segments; and determining, based on a difference between the prediction and the one or more selected segments, an update to the current values of the plurality of network parameters; adapting the pre-trained text summarization neural network for a specific text summarization task using labeled data comprising second texts and respective summaries of the second texts.
