Google llc (20240346824). ACTION LOCALIZATION IN VIDEOS USING LEARNED QUERIES simplified abstract

ACTION LOCALIZATION IN VIDEOS USING LEARNED QUERIES

Organization Name

google llc

Inventor(s)

Alexey Alexeevich Gritsenko of Amsterdam (NL)

Xuehan Xiong of Mountain View CA (US)

Josip Djolonga of Zurich (CH)

Mostafa Dehghani of Amsterdam (NL)

Chen Sun of San Francisco CA (US)

Mario Lucic of Adliswil (CH)

Cordelia Luise Schmid of Saint Ismier (FR)

Anurag Arnab of Grenoble (FR)

ACTION LOCALIZATION IN VIDEOS USING LEARNED QUERIES - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240346824 titled 'ACTION LOCALIZATION IN VIDEOS USING LEARNED QUERIES

Simplified Explanation:

The patent application describes methods, systems, and apparatus for action localization on an input video. This involves maintaining a set of query vectors and using them with the input video to generate an action localization output, specifying bounding boxes and actions for agents in the video frames.

• Uses query vectors to localize actions in a video
• Specifies bounding boxes and actions for agents in video frames

Key Features and Innovation:

• Utilizes query vectors to identify actions in a video
• Generates action localization output for each agent in the video
• Specifies bounding boxes and actions for agents in video frames

Potential Applications:

• Video surveillance systems
• Sports analysis software
• Video editing tools

Problems Solved:

• Efficiently localizing actions in videos
• Enhancing video analysis accuracy
• Streamlining video editing processes

Benefits:

• Improved video analysis capabilities
• Enhanced surveillance monitoring
• Simplified video editing workflows

Commercial Applications:

Potential commercial applications include:

• Security and surveillance systems
• Sports analytics software
• Video editing platforms

Questions about Action Localization Technology:

1. How does the system use query vectors to localize actions in a video?

  - The system uses query vectors to compare patterns in the input video frames and identify specific actions performed by agents.

2. What are the potential applications of action localization technology beyond video analysis?

  - Action localization technology can also be applied in robotics for object recognition and tracking.

Original Abstract Submitted

methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing action localization on an input video. in particular, a system maintains a set of query vectors and uses the input video and the set of query vectors to generate an action localization output for the input video. the action localization output includes, for each of one or more agents depicted in the video, data specifying, for each of one or more video frames in the video, a respective bounding box in the video frame that depicts the agent and a respective action from a set of actions that is being performed by the agent in the video frame.