View source for 20230154104. UNCERTAINTY-AWARE FUSION TOWARDS LARGE-SCALE NeRF simplified abstract (NEC Laboratories America, Inc.)

=UNCERTAINTY-AWARE FUSION TOWARDS LARGE-SCALE NeRF=


==Organization Name==

[[:Category:NEC Laboratories America, Inc.|NEC Laboratories America, Inc.]]


[[Category:NEC Laboratories America, Inc.]]

==Inventor(s)==

[[:Category:Bingbing Zhuang of San Jose CA (US)|Bingbing Zhuang of San Jose CA (US)]][[Category:Bingbing Zhuang of San Jose CA (US)]]

[[:Category:Samuel Schulter of New York NY (US)|Samuel Schulter of New York NY (US)]][[Category:Samuel Schulter of New York NY (US)]]

[[:Category:Yi-Hsuan Tsai of Santa Clara CA (US)|Yi-Hsuan Tsai of Santa Clara CA (US)]][[Category:Yi-Hsuan Tsai of Santa Clara CA (US)]]

[[:Category:Buyu Liu of Cupertino CA (US)|Buyu Liu of Cupertino CA (US)]][[Category:Buyu Liu of Cupertino CA (US)]]

[[:Category:Nanbo Li of Edinburgh (GB)|Nanbo Li of Edinburgh (GB)]][[Category:Nanbo Li of Edinburgh (GB)]]

==UNCERTAINTY-AWARE FUSION TOWARDS LARGE-SCALE NeRF - A simplified explanation of the abstract==


This abstract first appeared for US patent application 20230154104 titled 'UNCERTAINTY-AWARE FUSION TOWARDS LARGE-SCALE NeRF

==Simplified Explanation==

The patent application describes a method for achieving high-quality 3D reconstruction and novel view synthesis for large-scale scenes using multiple video image capturing devices. 

* The method involves obtaining images from a video stream captured by multiple devices.
* The images are grouped into different clusters representing a large-scale 3D scene.
* Neural Radiance Field (NeRF) and Uncertainty Multilayer Perceptron (MLP) models are trained for each image cluster to generate NeRFs and uncertainty MLPs.
* Rendering loss and entropy loss are applied to the NeRFs to improve their quality.
* Uncertainty-based fusion is performed to combine the NeRFs into a fused NeRF.
* The NeRFs and uncertainty MLPs are jointly fine-tuned to enhance their performance.
* During inference, the fused NeRF is used for generating novel views of the large-scale 3D scene.

Potential applications of this technology:

* Virtual reality (VR) and augmented reality (AR) applications
* Gaming and entertainment industry
* Architectural visualization and design
* Virtual tours and simulations
* Training and education in various fields

Problems solved by this technology:

* Achieving high-fidelity 3D reconstruction and novel view synthesis for large-scale scenes
* Overcoming limitations of single-camera systems in capturing detailed and realistic 3D scenes
* Improving the quality and accuracy of virtual views generated from captured images

Benefits of this technology:

* High-quality and realistic virtual views of large-scale scenes
* Enhanced immersion and user experience in VR and AR applications
* Improved accuracy and detail in 3D reconstruction
* Efficient utilization of multiple video image capturing devices for better scene representation


==Original Abstract Submitted==

a method for achieving high-fidelity novel view synthesis and 3d reconstruction for large-scale scenes is presented. the method includes obtaining images from a video stream received from a plurality of video image capturing devices, grouping the images into different image clusters representing a large-scale 3d scene, training a neural radiance field (nerf) and an uncertainty multilayer perceptron (mlp) for each of the image clusters to generate a plurality of nerfs and a plurality of uncertainty mlps for the large-scale 3d scene, applying a rendering loss and an entropy loss to the plurality of nerfs, performing uncertainty-based fusion to the plurality of nerfs to define a fused nerf, and jointly fine-tuning the plurality of nerfs and the plurality of uncertainty mlps, and during inference, applying the fused nerf for novel view synthesis of the large-scale 3d scene.



[[Category:G06T15/20]]
[[Category:G06T17/00]]
[[Category:G06V10/774]]
[[Category:G06V20/40]]
[[Category:G06V10/82]]