18597750. METHOD AND APPARATUS FOR TRAINING VIDEO GENERATION MODEL, STORAGE MEDIUM, AND COMPUTER DEVICE simplified abstract (Tencent Technology (Shenzhen) Company Limited)

METHOD AND APPARATUS FOR TRAINING VIDEO GENERATION MODEL, STORAGE MEDIUM, AND COMPUTER DEVICE

Organization Name

Tencent Technology (Shenzhen) Company Limited

Inventor(s)

Yang Wu of Shenzhen (CN)

Pengfei Hu of Shenzhen (CN)

Xiaojuan Qi of Shenzhen (CN)

Xiuzhe Wu of Shenzhen (CN)

Ying Shan of Shenzhen (CN)

Jing Xu of Shenzhen (CN)

METHOD AND APPARATUS FOR TRAINING VIDEO GENERATION MODEL, STORAGE MEDIUM, AND COMPUTER DEVICE - A simplified explanation of the abstract

This abstract first appeared for US patent application 18597750 titled 'METHOD AND APPARATUS FOR TRAINING VIDEO GENERATION MODEL, STORAGE MEDIUM, AND COMPUTER DEVICE

Simplified Explanation: This patent application describes a method for training a computer device to generate videos based on input features extracted from a target user's training video. By incorporating head pose and position information during training, the resulting video generation model can produce more coordinated and stable motions between the head and shoulders.

Key Features and Innovation:

• Method for training a video generation model using extracted phonetic, expression, and head parameters from a training video.
• Utilizes neural radiance field and image reconstruction loss for network training to obtain the video generation model.
• Incorporates head pose and position information to improve coordination and stability of motions between the head and shoulders.

Potential Applications: This technology can be applied in various fields such as:

• Virtual reality content creation
• Animation production
• Video editing software development

Problems Solved:

• Lack of coordination between head and shoulder motions in video generation models
• Difficulty in capturing natural and stable movements in generated videos

Benefits:

• Enhanced realism and naturalness in generated videos
• Improved quality and accuracy in motion capture and animation
• Increased efficiency in video production processes

Commercial Applications: Potential commercial uses include:

• Development of advanced video editing tools
• Integration into virtual reality and augmented reality applications
• Implementation in animation studios for more realistic character animations

Prior Art: To explore prior art related to this technology, researchers can look into patents and publications in the fields of computer graphics, machine learning, and video processing.

Frequently Updated Research: Researchers in the fields of computer vision and artificial intelligence are continuously exploring advancements in video generation models and motion capture techniques that could further enhance the capabilities of this technology.

Questions about Video Generation Models: 1. How does the incorporation of head pose information improve the coordination of motions in generated videos? 2. What are the potential limitations of using neural radiance fields for network training in video generation models?

Original Abstract Submitted

This application discloses a method for training a video generation model performed by a computer device. A phonetic feature, an expression parameter, and a head parameter are extracted from a training video of a target user. Network training is performed on a neural radiance field based on the condition input, three-dimensional coordinates, and a viewing direction to obtain a video generation model. The video generation model is obtained through training based on an image reconstruction loss. By introducing the head pose information and the head position information in the training process, a consideration of a shoulder motion status can be introduced into the video generation model so that a motion between the head and the shoulder is more coordinated and stable.