18517825. Modality Learning on Mobile Devices simplified abstract (Google LLC)

Modality Learning on Mobile Devices

Organization Name

Google LLC

Inventor(s)

Yu Ouyang of San Jose CA (US)

Diego Melendo Casado of San Francisco CA (US)

Mohammadinamul Hasan Sheik of San Jose CA (US)

Francoise Beaufays of Mountain View CA (US)

Dragan Zivkovic of Sunnyvale CA (US)

Meltem Oktem of Toronto (CA)

Modality Learning on Mobile Devices - A simplified explanation of the abstract

This abstract first appeared for US patent application 18517825 titled 'Modality Learning on Mobile Devices

Simplified Explanation

The abstract describes a method for cross input modality learning in a mobile device, where user inputs are recognized using different modalities and shared between modalities for improved recognition models.

• Explanation:

 * Method for recognizing user inputs using different modalities
 * Generating input context data structure based on recognized terms
 * Sharing input context data between modalities for updating recognition models

Potential Applications

This technology could be applied in:

• Mobile devices
• Virtual assistants
• Smart home devices

Problems Solved

This technology solves the following problems:

• Improving user input recognition accuracy
• Enhancing cross-modality learning
• Updating recognition models efficiently

Benefits

The benefits of this technology include:

• Enhanced user experience
• Improved accuracy in recognizing user inputs
• Seamless integration of different input modalities

Potential Commercial Applications

The potential commercial applications of this technology include:

• Mobile device manufacturers
• Virtual assistant developers
• Smart home automation companies

Possible Prior Art

One possible prior art for this technology could be:

• Cross-modality learning in machine learning models
• Multi-modal input recognition systems

Unanswered Questions

How does this technology handle privacy and security concerns related to sharing user input data between modalities?

This article does not address the specific methods or protocols used to ensure the privacy and security of user input data shared between modalities.

What are the potential limitations or challenges in implementing this technology in real-world applications?

The article does not discuss the potential obstacles or limitations that may arise when implementing this technology in practical settings, such as compatibility issues with existing systems or user acceptance.

Original Abstract Submitted

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for cross input modality learning in a mobile device are disclosed. In one aspect, a method includes activating a first modality user input mode in which user inputs by way of a first modality are recognized using a first modality recognizer; and receiving a user input by way of the first modality. The method includes, obtaining, as a result of the first modality recognizer recognizing the user input, a transcription that includes a particular term; and generating an input context data structure that references at least the particular term. The method further includes, transmitting, by the first modality recognizer, the input context data structure to a second modality recognizer for use in updating a second modality recognition model associated with the second modality recognizer.