Deepmind technologies limited (20250093970). LEARNED COMPUTER CONTROL USING POINTING DEVICE AND KEYBOARD ACTIONS
LEARNED COMPUTER CONTROL USING POINTING DEVICE AND KEYBOARD ACTIONS
Organization Name
Inventor(s)
Peter Conway Humphreys of London GB
Timothy Paul Lillicrap of London GB
Tobias Markus Pohlen of London GB
Adam Anthony Santoro of London GB
LEARNED COMPUTER CONTROL USING POINTING DEVICE AND KEYBOARD ACTIONS
This abstract first appeared for US patent application 20250093970 titled 'LEARNED COMPUTER CONTROL USING POINTING DEVICE AND KEYBOARD ACTIONS
Original Abstract Submitted
a computer-implemented method for controlling a particular computer to execute a task is described. the method includes receiving a control input comprising a visual input, the visual input including one or more screen frames of a computer display that represent at least a current state of the particular computer; processing the control input using a neural network to generate one or more control outputs that are used to control the particular computer to execute the task, in which the one or more control outputs include an action type output that specifies at least one of a pointing device action or a keyboard action to be performed to control the particular computer; determining one or more actions from the one or more control outputs; and executing the one or more actions to control the particular computer.
