US Patent Application 17814391. CROSS-MODAL SHAPE AND COLOR MANIPULATION simplified abstract
Contents
CROSS-MODAL SHAPE AND COLOR MANIPULATION
Organization Name
Inventor(s)
Menglei Chai of Los Angeles CA (US)
Sergey Tulyakov of Marina del Rey CA (US)
Jian Ren of Marina Del Ray CA (US)
Hsin-Ying Lee of San Jose CA (US)
Kyle Olszewski of Los Angeles CA (US)
Zeng Huang of Los Angeles CA (US)
Zezhou Cheng of Hadley MA (US)
CROSS-MODAL SHAPE AND COLOR MANIPULATION - A simplified explanation of the abstract
This abstract first appeared for US patent application 17814391 titled 'CROSS-MODAL SHAPE AND COLOR MANIPULATION
Simplified Explanation
- The patent application describes an editing system for 3D objects using 2D sketches or RGB views. - The system utilizes multi-modal variational auto-decoders (MM-VADs) trained with a shared latent space. - Editing 2D sketches of a 3D object allows for editing the corresponding 3D object. - A latent code is determined based on the edited or sketched 2D sketch. - The latent code is used to generate a 3D object using the MM-VADs. - The latent space is divided into separate spaces for shapes and colors. - The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.
Original Abstract Submitted
Systems, computer readable media, and methods herein describe an editing system where a three-dimensional (3D) object can be edited by editing a 2D sketch or 2D RGB views of the 3D object. The editing system uses multi-modal (MM) variational auto-decoders (VADs)(MM-VADs) that are trained with a shared latent space that enables editing 3D objects by editing 2D sketches of the 3D objects. The system determines a latent code that corresponds to an edited or sketched 2D sketch. The latent code is then used to generate a 3D object using the MM-VADs with the latent code as input. The latent space is divided into a latent space for shapes and a latent space for colors. The MM-VADs are trained with variational auto-encoders (VAE) and a ground truth.
