GM Cruise Holdings LLC (20240400088). OPTIMIZATIONS FOR REAL-TIME SENSOR FUSION IN VEHICLE UNDERSTANDING MODELS

OPTIMIZATIONS FOR REAL-TIME SENSOR FUSION IN VEHICLE UNDERSTANDING MODELS

Organization Name

GM Cruise Holdings LLC

Inventor(s)

Ajaya Hanasoge Shankaranarayana Rao of Sunnyvale CA (US)

Navaneeth Bodla of Redwood City CA (US)

Gia Tri Nguyen of Kirkland WA (US)

Xi Yi of Mountain View CA (US)

OPTIMIZATIONS FOR REAL-TIME SENSOR FUSION IN VEHICLE UNDERSTANDING MODELS

This abstract first appeared for US patent application 20240400088 titled 'OPTIMIZATIONS FOR REAL-TIME SENSOR FUSION IN VEHICLE UNDERSTANDING MODELS

Original Abstract Submitted

autonomous vehicles utilize perception and understanding of vehicles to predict behaviors of the vehicles, and to plan a trajectory. understanding of attributes of vehicles may be improved through sensor fusion. sensor fusion can be computationally expensive and may be difficult to implement in a real-time vehicle understanding system. to limit computational complexity while benefiting from machine learning across modalities, sensor fusion may be selectively implemented for a subset of task groups of a multi-task machine learning model. in some cases, part-based understanding may be implemented before fusion to limit the features being fused together to part features that are most salient for the task group. in addition, sensor data and features that may be fused together can be limited to sensor data and features within a desired field of view. a model that implements sensor fusion may be disabled for objects that are beyond a threshold distance.