17896524. REINFORCED GENERATION: REINFORCEMENT LEARNING FOR TEXT AND KNOWLEDGE GRAPH BI-DIRECTIONAL GENERATION USING PRETRAINED LANGUAGE MODELS simplified abstract (International Business Machines Corporation)

REINFORCED GENERATION: REINFORCEMENT LEARNING FOR TEXT AND KNOWLEDGE GRAPH BI-DIRECTIONAL GENERATION USING PRETRAINED LANGUAGE MODELS

Organization Name

International Business Machines Corporation

Inventor(s)

Pierre L. Dognin of White Plains NY (US)

Inkit Padhi of White Plains NY (US)

Igor Melnyk of White Plains NY (US)

Payel Das of Yorktown Heights NY (US)

REINFORCED GENERATION: REINFORCEMENT LEARNING FOR TEXT AND KNOWLEDGE GRAPH BI-DIRECTIONAL GENERATION USING PRETRAINED LANGUAGE MODELS - A simplified explanation of the abstract

This abstract first appeared for US patent application 17896524 titled 'REINFORCED GENERATION: REINFORCEMENT LEARNING FOR TEXT AND KNOWLEDGE GRAPH BI-DIRECTIONAL GENERATION USING PRETRAINED LANGUAGE MODELS

Simplified Explanation

The abstract describes a method for fine-tuning a pretrained encoder-decoder language model using a dataset of text-graph pairs, resulting in a final model available for deployment.

• Fine-tuning a pretrained language model by minimizing cross-entropy loss with a dataset of text-graph pairs.
• Text portion includes text tokens, while graph portion includes graph tokens.
• First fine-tuning training produces an intermediate model.
• Second fine-tuning training on the intermediate model using reinforcement learning to obtain a final model.

1. 1. 1. Potential Applications

• Natural language processing tasks such as text summarization, question answering, and dialogue generation.
• Data analysis and visualization tasks where text and graph data need to be processed together.

1. 1. 1. Problems Solved

• Enhances the performance of language models on tasks involving both text and graph data.
• Enables more efficient and accurate processing of text-graph pairs.

1. 1. 1. Benefits

• Improved accuracy and efficiency in handling text-graph data.
• Enhanced capabilities for various natural language processing tasks.
• Potential for deployment in a wide range of applications requiring text and graph data processing.

Original Abstract Submitted

Obtain access to a pretrained encoder-decoder language model. Using a dataset including a plurality of text-graph pairs, carry out first fine-tuning training on the pre-trained language model by minimizing cross-entropy loss. A text portion of each text-graph pair includes a list of text tokens and a graph portion of each text-graph pair includes a list of graph tokens. The first fine-tuning training results in an intermediate model. Carry out second fine-tuning training on the intermediate model, by reinforcement learning, to obtain a final model. Make the final model available for deployment.