18057117. REFINEMENT OF FACIAL KEYPOINT METADATA GENERATION FOR VIDEO CONFERENCING OR OTHER APPLICATIONS simplified abstract (Samsung Electronics Co., Ltd.)
Contents
REFINEMENT OF FACIAL KEYPOINT METADATA GENERATION FOR VIDEO CONFERENCING OR OTHER APPLICATIONS
Organization Name
Inventor(s)
Wenbo Li of Santa Clara CA (US)
Jing Zhu of Jackson Heights NY (US)
Hongxia Jin of San Jose CA (US)
REFINEMENT OF FACIAL KEYPOINT METADATA GENERATION FOR VIDEO CONFERENCING OR OTHER APPLICATIONS - A simplified explanation of the abstract
This abstract first appeared for US patent application 18057117 titled 'REFINEMENT OF FACIAL KEYPOINT METADATA GENERATION FOR VIDEO CONFERENCING OR OTHER APPLICATIONS
Simplified Explanation
The method described in the patent application involves generating metadata to animate a source image of a user's face or head to mimic movements seen in a driving video. This metadata includes identifying animations to be applied to the source image, while also suppressing artifacts associated with objects that may temporarily occlude or appear in the driving video.
- Obtaining a source image of a user
- Obtaining a driving video showing the user's face or head in motion
- Generating metadata to animate the source image to mimic movements in the driving video
- Transmitting the source image and metadata to an end user device for animation
Potential Applications
- Virtual reality applications
- Augmented reality applications
- Video conferencing
- Gaming
Problems Solved
- Enhancing user experience in virtual environments
- Improving realism in animations
- Minimizing distractions from occluding objects in videos
Benefits
- Enhanced user engagement
- Improved visual effects
- Seamless integration of animations with real-world videos
Original Abstract Submitted
A method includes obtaining a source image of a user and obtaining a driving video in which a face or a head of the user is moving. The method also includes generating metadata identifying animations to be applied to the source image so that the source image mimics at least some movements of the user's face or head in the driving video. The method further includes transmitting the source image and the metadata to an end user device configured to animate the source image based on the metadata. Generating the metadata includes suppressing one or more artifacts associated with one or more objects that temporarily occlude at least a portion of the user's head or body in the driving video or that temporarily appear in the driving video.
