Nvidia corporation (20240111025). OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS simplified abstract

OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS

Organization Name

nvidia corporation

Inventor(s)

Tilman Wekel of Sunnyvale CA (US)

Sangmin Oh of san Jose CA (US)

David Nister of Bellevue WA (US)

Joachim Pehserl of Lynnwood WA (US)

Neda Cvijetic of East palo Alto CA (US)

Ibrahim Eden of Redmond WA (US)

OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240111025 titled 'OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS

Simplified Explanation

The abstract describes a patent application for using a deep neural network (DNN) to detect and classify animate objects and/or parts of an environment, trained using camera-to-lidar cross injection to generate ground truth data for lidar range images.

• The DNN is trained using camera-to-lidar cross injection to generate reliable ground truth data for lidar range images.
• Annotations generated in the image domain are propagated to the lidar domain to increase the accuracy of the ground truth data in the lidar domain.
• The DNN outputs instance segmentation masks, class segmentation masks, and/or bounding shape proposals corresponding to 2D lidar range images, which are fused together to project the outputs into 3D lidar point clouds.
• The 2D and/or 3D information output by the DNN is provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.

Potential Applications

This technology can be applied in autonomous vehicles, robotics, surveillance systems, and industrial automation.

Problems Solved

This technology solves the problem of accurately detecting and classifying objects in complex environments, providing crucial information for safe navigation and decision-making in autonomous systems.

Benefits

The benefits of this technology include improved accuracy in object detection and classification, enhanced safety in autonomous systems, and increased efficiency in various applications.

Potential Commercial Applications

Potential commercial applications of this technology include autonomous vehicles, smart cities, security systems, and industrial automation.

Possible Prior Art

Prior art may include similar methods of using deep neural networks for object detection and classification in various domains, such as computer vision and robotics.

How does the accuracy of the ground truth data impact the performance of the DNN in detecting and classifying objects in lidar range images?

The abstract mentions that the accuracy of the ground truth data is increased by propagating annotations from the image domain to the lidar domain. However, it does not specify how this increased accuracy affects the overall performance of the DNN in object detection and classification.

What are the potential limitations or challenges of using a DNN for detecting and classifying objects in lidar range images?

While the abstract highlights the benefits and applications of using a DNN for this purpose, it does not address any potential limitations or challenges that may arise, such as computational complexity, data annotation requirements, or real-world performance issues.

Original Abstract Submitted

in various examples, a deep neural network (dnn) may be used to detect and classify animate objects and/or parts of an environment. the dnn may be trained using camera-to-lidar cross injection to generate reliable ground truth data for lidar range images. for example, annotations generated in the image domain may be propagated to the lidar domain to increase the accuracy of the ground truth data in the lidar domain—e.g., without requiring manual annotation in the lidar domain. once trained, the dnn may output instance segmentation masks, class segmentation masks, and/or bounding shape proposals corresponding to two-dimensional (2d) lidar range images, and the outputs may be fused together to project the outputs into three-dimensional (3d) lidar point clouds. this 2d and/or 3d information output by the dnn may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.