INTERNATIONAL BUSINESS MACHINES CORPORATION (20240320926). MIXED REALITY AVATAR EYE INPAINTING BASED ON USER SPEECH simplified abstract
Contents
MIXED REALITY AVATAR EYE INPAINTING BASED ON USER SPEECH
Organization Name
INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor(s)
Jing Chen of Chengzhong Garden (CN)
MIXED REALITY AVATAR EYE INPAINTING BASED ON USER SPEECH - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240320926 titled 'MIXED REALITY AVATAR EYE INPAINTING BASED ON USER SPEECH
The abstract describes a method, computer system, and computer program product for mixed reality. It involves receiving 3D non-eye landmarks and voice audio of a user, generating 3D eye landmarks using a trained model, refining the generated landmarks, and rendering the user's face model using a 3D face mesh.
- Receiving 3D non-eye landmarks and voice audio of a user
- Generating 3D eye landmarks using a trained model
- Refining the generated landmarks iteratively
- Rendering the user's face model using a 3D face mesh
Potential Applications: - Virtual reality applications - Augmented reality experiences - Facial recognition technology - Gaming and entertainment industries - Medical simulations and training
Problems Solved: - Enhancing realism in mixed reality environments - Improving facial tracking accuracy - Providing a more immersive user experience - Streamlining the process of creating realistic avatars - Advancing the field of computer vision
Benefits: - Enhanced user engagement - Realistic virtual interactions - Improved user customization options - Increased accuracy in facial animations - Potential for new and innovative applications in various industries
Commercial Applications: Title: "Advanced Mixed Reality Technology for Enhanced User Experiences" This technology can be utilized in various commercial applications such as virtual reality gaming, augmented reality marketing campaigns, virtual try-on experiences for e-commerce, virtual meetings and conferences, and medical simulations for training purposes. The market implications include increased user engagement, improved brand perception, and potential revenue growth for businesses implementing this technology.
Questions about Mixed Reality: 1. How does this technology improve the accuracy of facial tracking in mixed reality environments? 2. What are the potential limitations of using a trained eye landmark generative model in mixed reality applications?
Frequently Updated Research: Researchers are constantly exploring new ways to improve the accuracy and efficiency of facial tracking in mixed reality environments. Stay updated on the latest advancements in computer vision and virtual reality technology to understand the evolving landscape of mixed reality applications.
Original Abstract Submitted
according to one embodiment, a method, computer system, and computer program product for mixed reality is provided. the present invention may include receiving one or more 3d non-eye landmarks of a user; receiving at least one voice audio of the user; using random noise sampled with a unit normal distribution as one or more noised 3d eye landmarks for the user; inputting the received one or more 3d non-eye landmarks of the user, the at least one voice audio of the user, and the one or more noised 3d eye landmarks for the user, into a trained eye landmark generative model; generating one or more 3d eye landmarks for the user using the trained eye landmark generative model; performing iterative refinement of the one or more generated 3d eye landmarks using the trained eye landmark generative model; and rendering the user's generated face model using a formed 3d face mesh.
