18654691. MULTI-TASK MACHINE LEARNING ARCHITECTURES AND TRAINING PROCEDURES simplified abstract (Microsoft Technology Licensing, LLC)

MULTI-TASK MACHINE LEARNING ARCHITECTURES AND TRAINING PROCEDURES

Organization Name

Microsoft Technology Licensing, LLC

Inventor(s)

Weizhu Chen of Kirkland WA (US)

Pengcheng He of Sammamish WA (US)

Xiaodong Liu of Bellevue WA (US)

Jianfeng Gao of Woodinville WA (US)

MULTI-TASK MACHINE LEARNING ARCHITECTURES AND TRAINING PROCEDURES - A simplified explanation of the abstract

This abstract first appeared for US patent application 18654691 titled 'MULTI-TASK MACHINE LEARNING ARCHITECTURES AND TRAINING PROCEDURES

• • Simplified Explanation:**

This document discusses architectures and training procedures for multi-task machine learning models, such as neural networks. One example method involves providing a multi-task machine learning model with shared layers and task-specific layers, performing pretraining on the shared layers, and tuning on both shared and task-specific layers.

• • Key Features and Innovation:**
• Multi-task machine learning model with shared and task-specific layers
• Pretraining on shared layers using unsupervised prediction tasks
• Tuning on shared and task-specific layers using respective task-specific objectives

• • Potential Applications:**

This technology can be applied in various fields such as natural language processing, computer vision, and speech recognition where multiple tasks need to be performed simultaneously.

• • Problems Solved:**

This technology addresses the challenge of efficiently training multi-task machine learning models by leveraging shared layers and task-specific layers.

• • Benefits:**
• Improved efficiency in training multi-task machine learning models
• Enhanced performance in handling multiple tasks simultaneously
• Versatility in application across different domains

• • Commercial Applications:**

The technology can be utilized in industries such as healthcare, finance, and e-commerce for tasks like medical image analysis, fraud detection, and personalized recommendations.

• • Questions about Multi-Task Machine Learning Models:**

1. How does pretraining on shared layers benefit the overall performance of multi-task machine learning models?

  - Pretraining on shared layers helps in learning general features that can be useful for multiple tasks, improving the model's ability to handle diverse tasks effectively.

2. What are the advantages of using task-specific layers in multi-task machine learning models?

  - Task-specific layers allow the model to specialize in individual tasks, leading to better performance and efficiency in handling specific objectives.

Original Abstract Submitted

This document relates to architectures and training procedures for multi-task machine learning models, such as neural networks. One example method involves providing a multi-task machine learning model having one or more shared layers and two or more task-specific layers. The method can also involve performing a pretraining stage on the one or more shared layers using one or more unsupervised prediction tasks. The method can also involve performing a tuning stage on the one or more shared layers and the two or more task-specific layers using respective task-specific objectives