18200778. METHOD AND APPARATUS FOR GENERATING LANGUAGE MODEL USING CROSSMODAL INFORMATION simplified abstract (ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE)
METHOD AND APPARATUS FOR GENERATING LANGUAGE MODEL USING CROSSMODAL INFORMATION
Organization Name
ELECTRONICS AND TELECOMMUNICATIONS RESEARCH INSTITUTE
Inventor(s)
METHOD AND APPARATUS FOR GENERATING LANGUAGE MODEL USING CROSSMODAL INFORMATION - A simplified explanation of the abstract
This abstract first appeared for US patent application 18200778 titled 'METHOD AND APPARATUS FOR GENERATING LANGUAGE MODEL USING CROSSMODAL INFORMATION
Simplified Explanation
The method described in the abstract involves generating a language model using crossmodal information. Here is a simplified explanation of the abstract:
- Receiving language-based and non-language-based information
- Converting the information into byte sequences
- Generating embedding vectors for each modality
- Creating semantic association information using a crossmodal transformer
- Learning the language model using the generated semantic association information as training data
- Potential Applications of this Technology
- Improving natural language processing systems - Enhancing crossmodal information processing in AI systems
- Problems Solved by this Technology
- Integrating different types of information for more comprehensive language models - Improving the understanding of relationships between different modalities
- Benefits of this Technology
- Enhanced language model accuracy - Better utilization of crossmodal information for AI applications
- Potential Commercial Applications of this Technology
- Optimizing Language Models with Crossmodal Information
- Potential Commercial Applications of this Technology
- Possible Prior Art
There may be prior art related to crossmodal information processing in AI systems, but specific examples are not provided in the abstract.
- Unanswered Questions
- How does the crossmodal transformer handle different types of modality information?
The abstract does not delve into the specifics of how the crossmodal transformer processes and integrates information from different modalities.
- What embedding techniques are used for converting the byte sequences into embedding vectors?
The abstract mentions applying an embedding technique for each modality, but the specific techniques are not detailed.
Original Abstract Submitted
Provided is a method of generating a language model using crossmodal information. The method includes: receiving language-based first modality information and non-language-based second modality information; converting the first modality information into a first byte sequence; converting the second modality information into a second byte sequence; converting the first and second byte sequences into a first embedding vector and a second embedding vector by applying an embedding technique for each modality; generating semantic association information between first and second modality information by inputting the first and second embedding vectors to a crossmodal transformer; and learning the language model by setting the generated semantic association information as training data.
