Nvidia corporation (20240273919). MULTI-VIEW DEEP NEURAL NETWORK FOR LIDAR PERCEPTION simplified abstract

MULTI-VIEW DEEP NEURAL NETWORK FOR LIDAR PERCEPTION

Organization Name

nvidia corporation

Inventor(s)

Nikolai Smolyanskiy of Seattle WA (US)

Ryan Oldja of Redmond WA (US)

Ke Chen of Sunnyvale CA (US)

Alexander Popov of Kirkland WA (US)

Joachim Pehserl of Lynnnwood WA (US)

Ibrahim Eden of Redmond WA (US)

Tilman Wekel of Sunnyvale CA (US)

David Wehr of Redmond WA (US)

Ruchi Bhargava of Redmond WA (US)

David Nister of Bellevue WA (US)

MULTI-VIEW DEEP NEURAL NETWORK FOR LIDAR PERCEPTION - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240273919 titled 'MULTI-VIEW DEEP NEURAL NETWORK FOR LIDAR PERCEPTION

The abstract describes the use of deep neural networks (DNNs) to detect objects in a three-dimensional (3D) environment, such as in autonomous vehicles.

• Multi-view perception DNNs may consist of multiple stages that process different views of the 3D environment sequentially.
• The DNN may include stages for class segmentation and instance geometry regression in different views.
• Outputs of the DNN can generate 2D and/or 3D bounding boxes and class labels for detected objects.
• The technology enables the detection and classification of animate objects in the environment for safe planning and control of autonomous vehicles.

1. 1. 1. Potential Applications:

This technology can be applied in autonomous vehicles, robotics, surveillance systems, and augmented reality applications.

1. 1. 1. Problems Solved:

The technology addresses the need for accurate object detection and classification in complex 3D environments to enhance the safety and efficiency of autonomous systems.

1. 1. 1. Benefits:

The benefits include improved object detection accuracy, enhanced situational awareness, and safer navigation for autonomous vehicles.

1. 1. 1. Commercial Applications:

This technology has commercial applications in the automotive industry, robotics, smart cities, and security systems, where accurate object detection is crucial for operational success.

1. 1. 1. Prior Art:

Researchers can explore prior art related to object detection in 3D environments, deep learning applications in autonomous systems, and multi-view perception techniques.

1. 1. 1. Frequently Updated Research:

Stay updated on advancements in deep learning algorithms, object detection technologies, and applications of DNNs in autonomous systems for continuous improvement and innovation.

1. 1. 1. 1. Questions about Object Detection with DNNs:

1. How does the use of multi-view perception DNNs improve object detection accuracy in complex environments? 2. What are the key challenges in implementing deep neural networks for object detection in autonomous vehicles?

Original Abstract Submitted

a deep neural network(s) (dnn) may be used to detect objects from sensor data of a three dimensional (3d) environment. for example, a multi-view perception dnn may include multiple constituent dnns or stages chained together that sequentially process different views of the 3d environment. an example dnn may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and/or regresses instance geometry in a second view (e.g., top-down). the dnn outputs may be processed to generate 2d and/or 3d bounding boxes and class labels for detected objects in the 3d environment. as such, the techniques described herein may be used to detect and classify animate objects and/or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.