18083354. METHOD OF GENERATING MULTI-LAYER REPRESENTATION OF SCENE AND COMPUTING DEVICE IMPLEMENTING THE SAME simplified abstract (Samsung Electronics Co., Ltd.)

METHOD OF GENERATING MULTI-LAYER REPRESENTATION OF SCENE AND COMPUTING DEVICE IMPLEMENTING THE SAME

Organization Name

Samsung Electronics Co., Ltd.

Inventor(s)

Gleb Sterkin of Moscow (RU)

Pavel Ilyich Solovev of Moscow (RU)

Denis Mikhaylovich Korzhenkov of Moscow (RU)

Victor Sergeevich Lempitsky of Yerevan (RU)

Taras Andreevich Khakhulin of Moscow (RU)

METHOD OF GENERATING MULTI-LAYER REPRESENTATION OF SCENE AND COMPUTING DEVICE IMPLEMENTING THE SAME - A simplified explanation of the abstract

This abstract first appeared for US patent application 18083354 titled 'METHOD OF GENERATING MULTI-LAYER REPRESENTATION OF SCENE AND COMPUTING DEVICE IMPLEMENTING THE SAME

Simplified Explanation

The present disclosure is about a method of generating a multi-layer representation of a scene using neural networks trained in an end-to-end fashion. The method involves obtaining a pair of images of the scene, performing a reprojection operation on the images to generate a plane-sweep volume, predicting a layered structure of the scene using a geometry network, and estimating color values and opacity values for the predicted layered structure using a coloring network.

• The method uses neural networks to generate a multi-layer representation of a scene.
• It involves obtaining a pair of images, a reference image, and a source image.
• A reprojection operation is performed on the images to create a plane-sweep volume.
• A geometry network predicts the layered structure of the scene based on the plane-sweep volume.
• A coloring network estimates color values and opacity values for the predicted layered structure.
• The geometry network and the coloring network are trained in an end-to-end manner.

Potential Applications

• This technology can be used in computer graphics and virtual reality applications to generate realistic multi-layer representations of scenes.
• It can be applied in video games to create more immersive and visually appealing environments.
• It can be used in film and animation production to generate complex and detailed scenes.

Problems Solved

• Traditional methods of generating multi-layer representations of scenes may be time-consuming and require manual intervention.
• This technology solves the problem of accurately representing the layered structure of a scene using neural networks.
• It automates the process of generating multi-layer representations, saving time and effort.

Benefits

• The method provides a more efficient and automated way of generating multi-layer representations of scenes.
• It produces realistic and detailed representations, enhancing the visual quality of computer graphics and virtual reality applications.
• The end-to-end training of the neural networks ensures a seamless integration of the geometry and coloring processes.

Original Abstract Submitted

The present disclosure relates to the field of artificial intelligence (AI) and neural rendering, and particularly to a method of generating a multi-layer representation of a scene using neural networks trained in an end-to-end fashion and to a computing device implementing the method. The method of generating a multi-layer representation of a scene includes: obtaining a pair of images of the scene, the pair of the images comprising a reference image and a source image; performing a reprojection operation on the pair of images to generate a plane-sweep volume; predicting, using a geometry network, a layered structure of the scene based on the plane-sweep volume; and estimating, using a coloring network, color values and opacity values for the predicted layered structure of the scene to obtain the multi-layer representation of the scene; wherein the geometry network and the coloring network are trained in end-to-end manner.