US Patent Application 18331643. Contextual Assistant Using Mouse Pointing or Touch Cues simplified abstract
Contents
Contextual Assistant Using Mouse Pointing or Touch Cues
Organization Name
Inventor(s)
Dongeek Shin of San Jose CA (US)
Contextual Assistant Using Mouse Pointing or Touch Cues - A simplified explanation of the abstract
- This abstract for appeared for US patent application number 18331643 Titled 'Contextual Assistant Using Mouse Pointing or Touch Cues'
Simplified Explanation
This abstract describes a method for a contextual assistant to understand and respond to user queries based on audio input and spatial input on a screen. The assistant receives audio data of the user's query and a user input indication showing where the user interacted with the screen. It processes the audio to transcribe the query and interprets it to understand that it refers to an object on the screen without knowing which specific object. The assistant then requests information about the object and uses the user's spatial input to identify the object uniquely. It obtains the requested information and provides a response to the user's query.
Original Abstract Submitted
A method for a contextual assistant to use mouse pointing or touch cues includes receiving audio data corresponding to a query spoken by a user, receiving, in a graphical user interface displayed on a screen, a user input indication indicating a spatial input applied at a first location on the screen, and processing the audio data to determine a transcription of the query. The method also includes performing query interpretation on the transcription to determine that the query is referring to an object displayed on the screen without uniquely identifying the object, and requesting information about the object. The method further includes disambiguating, using the user input indication indicating the spatial input applied at the first location on the screen, the query to uniquely identify the object that the query is referring to, obtaining the information about the object requested by the query, and providing a response to the query.
