TRAJECTORY PREDICTION BY SAMPLING SEQUENCES OF DISCRETE MOTION TOKENS

Organization Name

waymo llc

Inventor(s)

Ari Seff of Mountain View CA (US)

Rami Al-rfou of Menlo Park CA (US)

Angelo Brian Cera of Mountain View CA (US)

Nigamaa Nayakanti of San Jose CA (US)

Aurick Qikun Zhou of San Francisco CA (US)

Mason Ng of Palo Alto CA (US)

Benjamin Sapp of Marina del Rey CA (US)

Dian Chen of Auxtin TX (US)

Khaled Refaat of Mountain View CA (US)

TRAJECTORY PREDICTION BY SAMPLING SEQUENCES OF DISCRETE MOTION TOKENS - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240300542 titled 'TRAJECTORY PREDICTION BY SAMPLING SEQUENCES OF DISCRETE MOTION TOKENS

The patent application describes methods, systems, and apparatus for generating trajectory predictions for agents in an environment using a trajectory prediction neural network conditioned on scene context data.

Obtaining scene context data characterizing a scene in an environment at a current time point.
Generating predicted future trajectories for multiple agents in the scene by sampling a sequence of discrete motion tokens.
Defining a joint future trajectory for the agents using the trajectory prediction neural network.

1. 1. Potential Applications:

This technology can be applied in autonomous vehicles, robotics, surveillance systems, and virtual reality environments.

1. 1. Problems Solved:

This technology addresses the challenge of predicting future trajectories of multiple agents in complex environments accurately.

1. 1. Benefits:

- Improved safety in autonomous vehicles and robotics. - Enhanced decision-making capabilities in surveillance systems. - Realistic movement predictions in virtual reality environments.

1. 1. Commercial Applications:

Predictive analytics software for autonomous vehicles, motion planning systems for robots, and immersive virtual reality experiences.

1. 1. Prior Art:

Researchers can explore existing trajectory prediction models, neural network architectures, and scene understanding techniques in related fields.

1. 1. Frequently Updated Research:

Stay updated on advancements in neural network training methods, scene representation techniques, and trajectory prediction algorithms.

1. 1. 1. Questions about trajectory prediction technology:

1. How does this technology improve safety in autonomous vehicles?

  - This technology enhances safety by accurately predicting the future trajectories of multiple agents in the environment, allowing vehicles to make informed decisions.

2. What are the key challenges in developing trajectory prediction neural networks?

  - Challenges include handling complex scene contexts, ensuring real-time predictions, and optimizing network performance for efficiency.

Original Abstract Submitted

methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating trajectory predictions for one or more agents in an environment. in one aspect, a method comprises: obtaining scene context data characterizing a scene in an environment at a current time point and generating a respective predicted future trajectory for each of a plurality of agents in the scene at the current time point by sampling a sequence of discrete motion tokens that defines a joint future trajectory for the plurality of agents using a trajectory prediction neural network that is conditioned on the scene context data.