18539157. NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS simplified abstract (THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY)

NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS

Organization Name

THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY

Inventor(s)

Stephen Tian of Stanford CA (US)

Yancheng Cai of Cambridge (GB)

Hong-Xing Yu of Stanford CA (US)

Sergey Zakharov of Stanford CA (US)

Katherine Liu of Mountain View CA (US)

Adrien David Gaidon of Mountain View CA (US)

Yunzhu Li of Stanford CA (US)

Jiajun Wu of Stanford CA (US)

NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS - A simplified explanation of the abstract

This abstract first appeared for US patent application 18539157 titled 'NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS

The abstract describes a method for dynamic modeling and manipulation of multi-object scenes using object-centric neural implicit scattering functions (OSFs) within a model-predictive control (MPC) framework.

• Object-centric neural implicit scattering functions (OSFs) are used as object representations in the model-predictive control (MPC) framework.
• Per-object light transport is modeled to enable scene re-rendering under object rearrangement and varying lighting conditions.
• Inverse parameter estimation and graph neural network (GNN) dynamics models are applied to estimate initial object poses and light positions in the scene.
• Objects in the multi-object scene are manipulated based on the results of the inverse parameter estimation and GNN dynamics models.

• • Potential Applications:**

- Computer graphics - Virtual reality - Robotics

• • Problems Solved:**

- Dynamic modeling of multi-object scenes - Object manipulation in complex scenes - Real-time scene re-rendering

• • Benefits:**

- Enhanced realism in virtual environments - Improved object manipulation capabilities - Adaptive lighting conditions in scenes

• • Commercial Applications:**

This technology could be utilized in industries such as gaming, animation, and architectural visualization to create more realistic and interactive virtual environments.

• • Questions about the Technology:**

1. How does this method improve upon existing techniques for modeling and manipulating multi-object scenes? 2. What are the potential limitations or challenges of implementing this technology in real-world applications?

Original Abstract Submitted

A method for dynamic modeling and manipulation of multi-object scenes is described. The method includes using object-centric neural implicit scattering functions (OSFs) as object representations in a model-predictive control (MPC) framework for the multi-object scenes. The method also includes modeling a per-object light transport to enable compositional scene re-rendering under object rearrangement and varying lighting conditions. The method further includes applying inverse parameter estimation and graph neural network (GNN) dynamics models to estimate initial object poses and a light position in the multi-object scene. The method also includes manipulating an object perceived in the multi-object scene according to the applying of the inverse parameter estimation and the GNN dynamics models.