18180290. GENERATING 3D HAND KEYPOINTS FOR A MIXED REALITY AVATAR simplified abstract (INTERNATIONAL BUSINESS MACHINES CORPORATION)
Contents
GENERATING 3D HAND KEYPOINTS FOR A MIXED REALITY AVATAR
Organization Name
INTERNATIONAL BUSINESS MACHINES CORPORATION
Inventor(s)
Wei Jun Zheng of Shanghai (CN)
GENERATING 3D HAND KEYPOINTS FOR A MIXED REALITY AVATAR - A simplified explanation of the abstract
This abstract first appeared for US patent application 18180290 titled 'GENERATING 3D HAND KEYPOINTS FOR A MIXED REALITY AVATAR
Simplified Explanation: This patent application describes a method for mixed reality that involves generating 3D keypoints of a user's visible and uncapturable hand joints, refining the uncapturable hand joints using a generative model, and rendering the generated 3D keypoints using a virtual hand modeler.
Key Features and Innovation:
- Receiving 3D hand keypoints of a user's visible hand joints and visible hand joints of the uncapturable hand
- Using random noise as initial keypoints for the uncapturable hand joints
- Inputting received and initial keypoints into a trained 3D hand joint generative model
- Iteratively refining the uncapturable hand joints using the generative model
- Identifying synchronization between generated keypoints of the uncapturable hand and the visible hand
- Rendering the generated 3D keypoints using a virtual hand modeler
Potential Applications: This technology can be used in virtual reality applications, gaming, training simulations, and interactive experiences.
Problems Solved: This technology addresses the challenge of accurately representing and rendering a user's uncapturable hand joints in mixed reality environments.
Benefits: The benefits of this technology include improved realism and immersion in mixed reality experiences, enhanced user interaction, and more accurate hand tracking.
Commercial Applications: Potential commercial applications of this technology include virtual reality gaming, virtual training simulations, interactive storytelling experiences, and virtual collaboration platforms.
Prior Art: Prior art related to this technology may include research on hand tracking in virtual reality, generative models for 3D hand reconstruction, and mixed reality applications for interactive experiences.
Frequently Updated Research: Researchers are continually exploring advancements in hand tracking technology, generative models for 3D reconstruction, and mixed reality applications for various industries.
Questions about Mixed Reality: 1. How does this technology improve user interaction in mixed reality environments? 2. What are the potential limitations of using generative models for refining uncapturable hand joints in mixed reality applications?
Original Abstract Submitted
According to one embodiment, a method, computer system, and computer program product for mixed reality is provided. The present invention may include receiving 3D hand keypoints (keypoints) of a user's visible hand joints from the user's capturable hand, and visible hand joints, if any, from the user's uncapturable hand; using random noise sampled with a unit normal distribution as initial keypoints for the uncapturable hand joints from the user's uncapturable hand; inputting the received and the initial keypoints, in a preset order, into a trained 3D hand joint generative model; performing an iterative refinement of the uncapturable hand joints from the user's uncapturable hand using the trained 3D hand joint generative model; identifying whether generated keypoints of the user's uncapturable hand are synchronized with the keypoints of the user's capturable hand; and rendering the generated 3D keypoints for the user's uncapturable hand joints using a 3D virtual hand modeler.
