18384322. ML USING N-GRAM INDUCED INPUT REPRESENTATION simplified abstract (MICROSOFT TECHNOLOGY LICENSING, LLC)

ML USING N-GRAM INDUCED INPUT REPRESENTATION

Organization Name

MICROSOFT TECHNOLOGY LICENSING, LLC

Inventor(s)

Pengcheng He of Beijing (CN)

Xiaodong Liu of Woodinville WA (US)

Jianfeng Gao of Woodinville WA (US)

Weizhu Chen of Kirkland WA (US)

ML USING N-GRAM INDUCED INPUT REPRESENTATION - A simplified explanation of the abstract

This abstract first appeared for US patent application 18384322 titled 'ML USING N-GRAM INDUCED INPUT REPRESENTATION

Simplified Explanation

The patent application discusses a method for generating an embedding that is both local string dependent and global string dependent to improve machine learning model performance. The method involves converting a string of words into tokens, generating local and global string-dependent embeddings for each token, combining these embeddings to create an n-gram induced embedding, and using a masked language model to generate masked word predictions based on the n-gram induced embeddings.

• Converting string of words into tokens
• Generating local and global string-dependent embeddings for each token
• Combining embeddings to create n-gram induced embeddings
• Using a masked language model for generating masked word predictions

Potential Applications

The technology described in the patent application could be applied in various fields such as natural language processing, sentiment analysis, and text classification.

Problems Solved

This technology addresses the challenge of improving machine learning model performance by generating embeddings that capture both local and global string dependencies in text data.

Benefits

The benefits of this technology include enhanced accuracy and efficiency in machine learning tasks that involve processing textual data.

Potential Commercial Applications

The technology could be utilized in industries such as e-commerce, customer service, and social media analytics to improve the performance of machine learning models in processing and analyzing text data.

Possible Prior Art

One possible prior art could be the use of traditional word embeddings in machine learning models for text processing tasks. Another could be the use of n-gram models in natural language processing applications.

How does this technology compare to existing methods for generating embeddings in machine learning models?

The technology described in the patent application combines local and global string-dependent embeddings to create n-gram induced embeddings, which could potentially offer a more comprehensive representation of text data compared to traditional methods.

What are the potential limitations or challenges in implementing this technology in real-world applications?

One potential limitation could be the computational resources required to generate and process the combined embeddings for large datasets. Additionally, the effectiveness of the method may vary depending on the specific characteristics of the text data being analyzed.

Original Abstract Submitted

Generally discussed herein are devices, systems, and methods for generating an embedding that is both local string dependent and global string dependent. The generated embedding can improve machine learning (ML) model performance. A method can include converting a string of words to a series of tokens, generating a local string-dependent embedding of each token of the series of tokens, generating a global string-dependent embedding of each token of the series of tokens, combining the local string dependent embedding the global string dependent embedding to generate an n-gram induced embedding of each token of the series of tokens, obtaining a masked language model (MLM) previously trained to generate a masked word prediction, and executing the MLM based on the n-gram induced embedding of each token to generate the masked word prediction.