View source for 18472528. IMPROVEMENT OF AUDIO-VISUAL QUESTION ANSWERING (Lemon Inc.)

=IMPROVEMENT OF AUDIO-VISUAL QUESTION ANSWERING=


==Organization Name==

[[:Category:Lemon Inc.|Lemon Inc.]]


[[Category:Lemon Inc.]]

==Inventor(s)==

[[:Category:Peng Zhang of Los Angeles CA US|Peng Zhang of Los Angeles CA US]][[Category:Peng Zhang of Los Angeles CA US]]

[[:Category:Xiulong Liu of Culver City CA US|Xiulong Liu of Culver City CA US]][[Category:Xiulong Liu of Culver City CA US]]

[[:Category:Zhikang Dong of Culver City CA US|Zhikang Dong of Culver City CA US]][[Category:Zhikang Dong of Culver City CA US]]

==IMPROVEMENT OF AUDIO-VISUAL QUESTION ANSWERING==


This abstract first appeared for US patent application 18472528 titled 'IMPROVEMENT OF AUDIO-VISUAL QUESTION ANSWERING

==Original Abstract Submitted==

The present disclosure describes techniques for improving audio-visual question answering. A machine learning model is configured for audio-visual question answering (AVQA). The machine learning model comprises a first sub-model configured to capture semantic audio information and output an audio spatial feature map x. The machine learning model comprises a second sub-model configured to extract visual features xand audio features xand further configured to obtain a question vector x. The machine learning model comprises a third sub-model configured to capture audio-visual correspondence at a granular level. A balanced AVQA dataset is created. The balanced AVQA dataset comprises balanced answer distribution in each question category. The machine learning model is trained to answer questions about visual objects, sounds, and their associations in videos using at least a subset of the balanced AVAQ dataset.



[[Category:G10L15/22]]
[[Category:G06V10/77]]
[[Category:G06V10/774]]
[[Category:G10L15/06]]
[[Category:CPC_G10L15/22]]