18539157. NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS simplified abstract (TOYOTA JIDOSHA KABUSHIKI KAISHA)

NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS

Organization Name

TOYOTA JIDOSHA KABUSHIKI KAISHA

Inventor(s)

Stephen Tian of Stanford CA (US)

Yancheng Cai of Cambridge (GB)

Hong-Xing Yu of Stanford CA (US)

Sergey Zakharov of Stanford CA (US)

Katherine Liu of Mountain View CA (US)

Adrien David Gaidon of Mountain View CA (US)

Yunzhu Li of Stanford CA (US)

Jiajun Wu of Stanford CA (US)

NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS - A simplified explanation of the abstract

This abstract first appeared for US patent application 18539157 titled 'NEURAL IMPLICIT SCATTERING FUNCTIONS FOR INVERSE PARAMETER ESTIMATION AND DYNAMICS MODELING OF MULTI-OBJECT INTERACTIONS

• • Simplified Explanation:**

The patent application describes a method for dynamically modeling and manipulating multi-object scenes using object-centric neural implicit scattering functions (OSFs) within a model-predictive control (MPC) framework. This method enables re-rendering of scenes under object rearrangement and changing lighting conditions.

• • Key Features and Innovation:**

- Utilizes object-centric neural implicit scattering functions (OSFs) as object representations in a model-predictive control (MPC) framework. - Models per-object light transport for compositional scene re-rendering under varying lighting conditions. - Applies inverse parameter estimation and graph neural network (GNN) dynamics models to estimate object poses and light positions. - Enables manipulation of objects in multi-object scenes based on the estimated parameters.

• • Potential Applications:**

- Computer graphics and animation - Virtual reality and augmented reality environments - Robotics and autonomous systems - Industrial automation and manufacturing processes

• • Problems Solved:**

- Dynamic modeling and manipulation of multi-object scenes - Efficient re-rendering of scenes under changing conditions - Accurate estimation of object poses and light positions - Real-time control and manipulation of objects in complex scenes

• • Benefits:**

- Enhanced realism and flexibility in scene rendering - Improved efficiency in object manipulation and control - Seamless integration with existing modeling and control frameworks - Potential for automation and optimization in various industries

• • Commercial Applications:**

Dynamic Scene Modeling and Manipulation Technology for Enhanced Visualization and Control

• • Questions about Dynamic Scene Modeling and Manipulation:**

1. How does the use of object-centric neural implicit scattering functions improve scene modeling and manipulation? 2. What are the potential industrial applications of this technology in automation and control systems?

Original Abstract Submitted

A method for dynamic modeling and manipulation of multi-object scenes is described. The method includes using object-centric neural implicit scattering functions (OSFs) as object representations in a model-predictive control (MPC) framework for the multi-object scenes. The method also includes modeling a per-object light transport to enable compositional scene re-rendering under object rearrangement and varying lighting conditions. The method further includes applying inverse parameter estimation and graph neural network (GNN) dynamics models to estimate initial object poses and a light position in the multi-object scene. The method also includes manipulating an object perceived in the multi-object scene according to the applying of the inverse parameter estimation and the GNN dynamics models.