18466779. VOXEL SEARCH USING MULTI-MODAL EMBEDDINGS (GM Cruise Holdings LLC)
VOXEL SEARCH USING MULTI-MODAL EMBEDDINGS
Organization Name
Inventor(s)
Carden Bagwell of San Francisco CA (US)
VOXEL SEARCH USING MULTI-MODAL EMBEDDINGS
This abstract first appeared for US patent application 18466779 titled 'VOXEL SEARCH USING MULTI-MODAL EMBEDDINGS
Original Abstract Submitted
Aspects of the disclosed technology provide solutions for searching voxel data, such as voxelized Light Detection and Ranging (LiDAR) point cloud data and in particular, for using multi-modal embeddings for searching objects within a voxel data set. A process of the disclosed technology can include steps for receiving sensor data, wherein the sensor data represents a real-world environment encountered by an autonomous vehicle (AV) and wherein the sensor data comprises point cloud data representing a plurality of objects; generating a voxel representation for the plurality of objects; generating, based on the voxel representation for the plurality of objects, a corresponding set of first embeddings; receiving a text string; generating a second embedding corresponding to the text string; and identifying a matching object among the plurality of objects based on a comparison of the set of first embeddings and the second embedding. System and machine-readable media are also provided.