18653917. HUMAN SUBJECT GAUSSIAN SPLATTING USING MACHINE LEARNING (Apple Inc.)
HUMAN SUBJECT GAUSSIAN SPLATTING USING MACHINE LEARNING
Organization Name
Inventor(s)
Anurag Ranjan of Sunnyvale CA US
Muhammed Kocabas of Seattle WA US
James C. Gabriel of Seattle WA US
Jen-Hao Chang of Santa Clara CA US
Cuneyt O. Tuzel of Cupertino CA US
HUMAN SUBJECT GAUSSIAN SPLATTING USING MACHINE LEARNING
This abstract first appeared for US patent application 18653917 titled 'HUMAN SUBJECT GAUSSIAN SPLATTING USING MACHINE LEARNING
Original Abstract Submitted
Aspects of the subject technology provide for human subject Gaussian splatting using machine learning. A method includes receiving a video input having a scene and a subject. The method also includes obtaining a three-dimensional (3D) reconstruction of the subject and the scene from the video input. The method includes generating a 3D Gaussian representation of each of the scene and the subject. The method also includes generating a deformed 3D Gaussian representation of the subject by adapting the 3D Gaussian representation of the subject to the 3D reconstruction of the subject. The method includes rendering a visual output comprising an animatable avatar of the subject and the scene using differentiable Gaussian rasterization based at least in part on the deformed 3D Gaussian representation of the subject and the 3D Gaussian representation of the scene.