18262662. COLOR AND INFRA-RED THREE-DIMENSIONAL RECONSTRUCTION USING IMPLICIT RADIANCE FUNCTIONS simplified abstract (Google LLC)

COLOR AND INFRA-RED THREE-DIMENSIONAL RECONSTRUCTION USING IMPLICIT RADIANCE FUNCTIONS

Organization Name

Google LLC

Inventor(s)

Ricardo Martin Brualla of Seattle WA (US)

COLOR AND INFRA-RED THREE-DIMENSIONAL RECONSTRUCTION USING IMPLICIT RADIANCE FUNCTIONS - A simplified explanation of the abstract

This abstract first appeared for US patent application 18262662 titled 'COLOR AND INFRA-RED THREE-DIMENSIONAL RECONSTRUCTION USING IMPLICIT RADIANCE FUNCTIONS

Simplified Explanation

An image is rendered based on a neural radiance field (NeRF) volumetric representation of a scene, where the NeRF representation is based on captured frames of video data, each frame including a color image, a widefield IR image, and a plurality of depth IR images of the scene. Each depth IR image is captured when the scene is illuminated by a different pattern of points of IR light, and the illumination by the patterns occurs at different times. The NeRF representation provides a mapping between positions and viewing directions to a color and optical density at each position in the scene, where the color and optical density at each position enables a viewing of the scene from a new perspective, and the NeRF representation provides a mapping between positions and viewing directions to IR values for each of the different patterns of points of IR light from the new perspective.

• NeRF representation based on captured frames of video data
• Includes color image, widefield IR image, and depth IR images of the scene
• Depth IR images captured with different patterns of IR light
• Provides mapping between positions and viewing directions to color and optical density
• Enables viewing of the scene from a new perspective
• Provides mapping to IR values for different patterns of IR light

Potential Applications

This technology could be applied in various fields such as virtual reality, augmented reality, 3D modeling, and computer graphics.

Problems Solved

This technology solves the problem of accurately rendering images from different perspectives using captured video data and depth information.

Benefits

The benefits of this technology include realistic rendering of scenes, enhanced visualization from different viewpoints, and improved accuracy in image reconstruction.

Potential Commercial Applications

Potential commercial applications of this technology include video game development, virtual tours, architectural visualization, and medical imaging.

Possible Prior Art

One possible prior art could be volumetric rendering techniques used in computer graphics and medical imaging to visualize 3D structures from 2D data.

How does this technology compare to traditional rendering methods?

This article does not provide a direct comparison between this technology and traditional rendering methods.

What are the limitations of using NeRF representations for rendering images?

This article does not address the potential limitations or challenges of using NeRF representations for rendering images.

Original Abstract Submitted

An image is rendered based a neural radiance field (NeRF) volumetric representation of a scene, where the NeRF representation is based on captured frames of video data, each frame including a color image, a widefield IR image, and a plurality of depth IR images of the scene. Each depth IR image is captured when the scene is illuminated by a different pattern of points of IR light, and the illumination by the patterns occurs at different times. The NeRF representation provides a mapping between positions and viewing directions to a color and optical density at each position in the scene, where the color and optical density at each position enables a viewing of the scene from a new perspective, and the NeRF representation provides a mapping between positions and viewing directions to IR values for each of the different patterns of points of IR light from the new perspective.