18531103. OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS simplified abstract (NVIDIA Corporation)

OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS

Organization Name

NVIDIA Corporation

Inventor(s)

Tilman Wekel of Sunnyvale CA (US)

Sangmin Oh of san Jose CA (US)

David Nister of Bellevue WA (US)

Joachim Pehserl of Lynnwood WA (US)

Neda Cvijetic of East palo Alto CA (US)

Ibrahim Eden of Redmond WA (US)

OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS - A simplified explanation of the abstract

This abstract first appeared for US patent application 18531103 titled 'OBJECT DETECTION AND CLASSIFICATION USING LIDAR RANGE IMAGES FOR AUTONOMOUS MACHINE APPLICATIONS

Simplified Explanation

The abstract describes a patent application for using a deep neural network (DNN) to detect and classify animate objects and/or parts of an environment, trained using camera-to-LiDAR cross injection to generate reliable ground truth data for LiDAR range images.

• The DNN is trained using camera-to-LiDAR cross injection to generate ground truth data for LiDAR range images.
• Annotations from the image domain are propagated to the LiDAR domain to increase the accuracy of ground truth data without manual annotation in the LiDAR domain.
• The DNN outputs instance segmentation masks, class segmentation masks, and/or bounding shape proposals for 2D LiDAR range images, which are fused to project the outputs into 3D LiDAR point clouds.
• The 2D and/or 3D information output by the DNN is provided to an autonomous vehicle drive stack for safe planning and control of the vehicle.

---

1. 1. 1. Potential Applications

• Autonomous driving systems
• Robotics
• Surveillance systems

1. 1. 1. Problems Solved

• Accurate detection and classification of objects in complex environments
• Efficient data annotation for LiDAR range images
• Integration of 2D and 3D information for autonomous vehicle control

1. 1. 1. Benefits

• Improved safety in autonomous driving
• Enhanced object detection capabilities
• Streamlined data annotation processes

1. 1. 1. Potential Commercial Applications
         1. Enhancing Autonomous Vehicle Technology for Safer Driving

1. 1. 1. Possible Prior Art

There may be prior art related to using deep neural networks for object detection and classification in LiDAR range images, but specific examples are not provided in the abstract.

---

1. 1. 1. 1. Unanswered Questions

1. 1. 1. How does the DNN handle occlusions and complex environmental conditions during object detection?

The abstract does not mention how the DNN addresses occlusions and complex environmental conditions that may affect object detection accuracy.

1. 1. 1. What is the computational efficiency of the proposed method compared to existing techniques?

The abstract does not provide information on the computational efficiency of the proposed method in comparison to other techniques available in the field.

Original Abstract Submitted

In various examples, a deep neural network (DNN) may be used to detect and classify animate objects and/or parts of an environment. The DNN may be trained using camera-to-LiDAR cross injection to generate reliable ground truth data for LiDAR range images. For example, annotations generated in the image domain may be propagated to the LiDAR domain to increase the accuracy of the ground truth data in the LiDAR domain—e.g., without requiring manual annotation in the LiDAR domain. Once trained, the DNN may output instance segmentation masks, class segmentation masks, and/or bounding shape proposals corresponding to two-dimensional (2D) LiDAR range images, and the outputs may be fused together to project the outputs into three-dimensional (3D) LiDAR point clouds. This 2D and/or 3D information output by the DNN may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.