Google LLC (20250117185). USING AUDIO SEPARATION AND CLASSIFICATION TO ENHANCE AUDIO IN VIDEOS
USING AUDIO SEPARATION AND CLASSIFICATION TO ENHANCE AUDIO IN VIDEOS
Organization Name
Inventor(s)
Moonseok Kim of Mountain View CA US
Elliot Patros of Mountain View CA US
Sneh Singaraju of Mountain View CA US
Michelle Ansai of Mountain View CA US
Efthymios Tzinis of Mountain View CA US
USING AUDIO SEPARATION AND CLASSIFICATION TO ENHANCE AUDIO IN VIDEOS
This abstract first appeared for US patent application 20250117185 titled 'USING AUDIO SEPARATION AND CLASSIFICATION TO ENHANCE AUDIO IN VIDEOS
Original Abstract Submitted
a media application obtains a video that includes an audio portion. the media application separates the audio portion into a plurality of channels, where each channel corresponds to a particular audio source. an on-screen classifier model obtains an indication of whether the particular audio source for each channel is depicted in the video. an audio-type classifier model determines, an auditory object classification for each channel. the media application determines a respective gain for each channel based on the indication of whether the particular audio source for the channel is depicted in the video and the auditory object classification for the channel. the media application modifies each channel by applying the respective gain. the media application mixes the modified channels with the audio portion to generate a combined audio.
