Meta platforms technologies, llc (20240346729). SYNCHRONIZING VIDEO OF AN AVATAR WITH LOCALLY CAPTURED AUDIO FROM A USER CORRESPONDING TO THE AVATAR simplified abstract
SYNCHRONIZING VIDEO OF AN AVATAR WITH LOCALLY CAPTURED AUDIO FROM A USER CORRESPONDING TO THE AVATAR
Organization Name
meta platforms technologies, llc
Inventor(s)
Plamen Alexandrov Ivanov of Goleta CA (US)
Antti Kelloniemi of Redwood City CA (US)
SYNCHRONIZING VIDEO OF AN AVATAR WITH LOCALLY CAPTURED AUDIO FROM A USER CORRESPONDING TO THE AVATAR - A simplified explanation of the abstract
This abstract first appeared for US patent application 20240346729 titled 'SYNCHRONIZING VIDEO OF AN AVATAR WITH LOCALLY CAPTURED AUDIO FROM A USER CORRESPONDING TO THE AVATAR
The patent application describes a system where multiple users in a local area use headsets to communicate with each other and with users in a remote area.
- The headsets have acoustic sensors, transducer arrays, and external capturing sensors to capture information about the local area.
- When a user's headset detects audio from another user in the local area, the avatar representing that user is modified to appear synchronized with the audio heard by the user.
- Each headset also provides face tracking through internal imaging devices, sending this information along with captured audio to a server for users in the remote area.
Potential Applications:
- Virtual reality communication systems
- Multi-user gaming environments
- Remote collaboration tools
Problems Solved:
- Enhancing the immersive experience of communication in virtual environments
- Improving synchronization of audio and visual representations in virtual reality
Benefits:
- Enhanced user experience in virtual communication
- Improved realism and immersion in virtual environments
Commercial Applications:
- Virtual reality gaming platforms
- Remote team collaboration tools
- Virtual events and conferences
Questions about the Technology: 1. How does the system ensure accurate synchronization between audio and visual representations? 2. What are the potential privacy concerns related to capturing and transmitting facial tracking data?
Frequently Updated Research: There may be ongoing research in the fields of virtual reality communication systems, audio-visual synchronization, and remote collaboration tools that could impact this technology.
Original Abstract Submitted
a local area includes multiple users each using a headset to communicate with other users in the local area, as well as with additional users in a remote area. the headset includes one or more acoustic sensors, a transducer array, one or more external capturing sensors (e.g., cameras) capturing information describing the local area. a headset of a user in the local area local detects audio from an additional user in the local area. in response to detecting the audio, an avatar representing the additional user that is displayed by the user's headset is modified to appear to be synchronized with the audio from the additional user that is heard by the user. additionally, each headset provides respective face tracking through one or more internal imaging devices that is provided with and captured audio to a server which provides the information users in the remote area.
