RENSSELAER POLYTECHNIC INSTITUTE (20240422495). MULTI-SENSOR SYSTEMS AND METHODS FOR PROVIDING IMMERSIVE VIRTUAL ENVIRONMENTS
MULTI-SENSOR SYSTEMS AND METHODS FOR PROVIDING IMMERSIVE VIRTUAL ENVIRONMENTS
Organization Name
RENSSELAER POLYTECHNIC INSTITUTE
Inventor(s)
Jonathan Mathews of Troy NY (US)
Jonas Braasch of Latham NY (US)
Richard J. Radke of SLINGERLANDS NY (US)
Gyanendra Sharma of FOSTER CITY CA (US)
Devavrat Jivani of Woburn MA (US)
MULTI-SENSOR SYSTEMS AND METHODS FOR PROVIDING IMMERSIVE VIRTUAL ENVIRONMENTS
This abstract first appeared for US patent application 20240422495 titled 'MULTI-SENSOR SYSTEMS AND METHODS FOR PROVIDING IMMERSIVE VIRTUAL ENVIRONMENTS
Original Abstract Submitted
a time-of-flight (tof) array including a plurality of cameras is positioned with a coplanar higher order ambisonics (hoa) array including a plurality of microphones above a target area such as a room or space for analyzing speaker activity in that area. the hoa microphone array iteratively samples sound energy level frames captured by the microphones to identify a sound vector associated with the global maximum energy level in the sound energy level frame. this sound vector is fused with the data from the tof camera array that identified the positions of sound sources in the target area to associate produced sound corresponding with the sound vector with a physical active sound source in the target area. a beamformer can then be used to save audio corresponding to the sound vector and discard audio not associated with the sound vector to produce a sound recording associated with the active sound source.
