18522763. MODELING PHOTOREALISTIC FACES WITH EYEGLASSES simplified abstract (Meta Platforms Technologies, LLC)

MODELING PHOTOREALISTIC FACES WITH EYEGLASSES

Organization Name

Meta Platforms Technologies, LLC

Inventor(s)

Shunsuke Saito of Pittsburgh PA (US)

Junxuan Li of Pittsburgh PA (US)

Tomas Simon Kreuz of Pittsburgh PA (US)

Jason Saragih of Pittsburgh PA (US)

Shun Iwase of Pittsburgh PA (US)

Timur Bagautdinov of Pittsburgh PA (US)

Rohan Joshi of Pittsburgh PA (US)

Fabian Andres Prada Nino of Pittsburgh PA (US)

Takaaki Shiratori of Pittsburgh PA (US)

Yaser Sheikh of Pittsburgh PA (US)

Stephen Anthony Lombardi of Pittsburgh PA (US)

MODELING PHOTOREALISTIC FACES WITH EYEGLASSES - A simplified explanation of the abstract

This abstract first appeared for US patent application 18522763 titled 'MODELING PHOTOREALISTIC FACES WITH EYEGLASSES

Simplified Explanation

The patent application describes methods, systems, and storage media for modeling subjects in a virtual environment. Here is a simplified explanation of the abstract:

• Receiving image data from a client device with at least one subject.
• Extracting a face of the subject and an object interacting with the face, such as glasses.
• Generating face primitives with geometry and appearance information.
• Generating object primitives based on latent codes for the object.
• Creating an appearance model of photometric interactions between the face and the object.
• Rendering an avatar in the virtual environment based on the appearance model, face primitives, and object primitives.

Potential Applications

This technology could be used in virtual reality applications, video games, virtual try-on experiences for glasses or other accessories, and virtual meetings or events.

Problems Solved

This technology solves the problem of accurately modeling and rendering subjects in a virtual environment, including their faces and interactions with objects, to create realistic avatars.

Benefits

The benefits of this technology include more realistic and detailed virtual representations of subjects, improved user experiences in virtual environments, and enhanced customization options for avatars.

Potential Commercial Applications

The technology could be commercially applied in industries such as gaming, fashion, virtual events, virtual reality training simulations, and online retail for virtual try-on experiences.

Possible Prior Art

One possible prior art could be existing methods for creating avatars in virtual environments, such as using motion capture technology or 3D scanning techniques.

How does this technology handle different lighting conditions in the virtual environment?

The patent application does not specifically address how the technology adapts to varying lighting conditions to ensure realistic rendering of avatars.

What level of detail can be achieved in modeling the appearance of objects interacting with the subject's face?

The abstract mentions generating object primitives based on latent codes, but it does not detail the level of detail or complexity that can be achieved in modeling these objects.

Original Abstract Submitted

Methods, systems, and storage media for modeling subjects in a virtual environment are disclosed. Exemplary implementations may: receiving, from a client device, image data including at least one subject; extracting, from the image data, a face of the at least one subject and an object interacting with the face, wherein the object may be glasses worn by the subject; generating a set of face primitives based on the face, the set of face primitives comprising geometry and appearance information; generating a set of object primitives based on a set of latent codes for the object; generating an appearance model of photometric interactions between the face and the object; and rendering an avatar in the virtual environment based on the appearance model, the set of face primitives, and the set of object primitives.