View source for 18626959. SYSTEMS AND METHODS FOR GESTURE GENERATION FROM TEXT AND NON-SPEECH simplified abstract (Datum Point Labs Inc.)

=SYSTEMS AND METHODS FOR GESTURE GENERATION FROM TEXT AND NON-SPEECH=


==Organization Name==

[[:Category:Datum Point Labs Inc.|Datum Point Labs Inc.]]


[[Category:Datum Point Labs Inc.]]

==Inventor(s)==

[[:Category:Gwantae Kim of Seoul (KR)|Gwantae Kim of Seoul (KR)]][[Category:Gwantae Kim of Seoul (KR)]]

[[:Category:Hanseok Ko of Seoul (KR)|Hanseok Ko of Seoul (KR)]][[Category:Hanseok Ko of Seoul (KR)]]

==SYSTEMS AND METHODS FOR GESTURE GENERATION FROM TEXT AND NON-SPEECH - A simplified explanation of the abstract==


This abstract first appeared for US patent application 18626959 titled 'SYSTEMS AND METHODS FOR GESTURE GENERATION FROM TEXT AND NON-SPEECH

The abstract describes a system and method for generating gestures from multimodal input, involving masking input, generating embeddings, extracting features, generating output, computing loss, and updating parameters.

* System and method for gesture generation from multimodal input
* Involves masking input, generating embeddings, extracting features, generating output, computing loss, and updating parameters
* Utilizes an embedder, encoder with attention layers, and generator for processing multimodal data
* Loss is computed based on input and output, guiding parameter updates for the encoder

Potential Applications:
- Human-computer interaction
- Virtual reality and augmented reality systems
- Sign language recognition and translation
- Gesture-based control interfaces
- Multimodal communication platforms

Problems Solved:
- Enhances the interpretation of complex multimodal input
- Improves the generation of gestures from diverse input sources
- Facilitates seamless communication between humans and machines

Benefits:
- Enhanced user experience in interactive systems
- Improved accuracy in gesture recognition and generation
- Increased efficiency in multimodal data processing
- Enables natural and intuitive communication interfaces

Commercial Applications:
Title: Multimodal Gesture Generation System
This technology can be applied in various industries such as:
- Gaming and entertainment
- Education and training
- Healthcare and telemedicine
- Automotive and robotics
- Communication and collaboration tools

Questions about Multimodal Gesture Generation:
1. How does this technology improve human-computer interaction?
2. What are the potential challenges in implementing this system in real-world applications?

Frequently Updated Research:
Stay updated on advancements in multimodal data processing, gesture recognition, and human-machine interaction for insights into the latest developments in the field.


==Original Abstract Submitted==

Embodiments described herein provide systems and methods for gesture generation from multimodal input. A method includes receiving a multimodal input. The method may further include masking a subset of the multimodal input; generating, via an embedder, a multimodal embedding based on the masked multimodal input; generating, via an encoder, multimodal features based on the multimodal embedding, wherein the encoder includes one or more attention layers connecting different modalities; generating, via a generator, multimodal output based on the multimodal features; computing a loss based on the multimodal input and the multimodal output. The method may further include updating parameters of the encoder based on the loss.



[[Category:G06N3/08]]
[[Category:G06N3/0455]]
[[Category:CPC_G06N3/08]]