Meta Platforms, Inc. (20240219720). Reading Out Scene Analysis for Users Wearing Head-mounted Devices simplified abstract

Reading Out Scene Analysis for Users Wearing Head-mounted Devices

Organization Name

Meta Platforms, Inc.

Inventor(s)

Johana Gabriela Coyoc Escudero of Milpitas CA (US)

Pascale El Kallassi of Menlo Park CA (US)

Omar John Pualuan of San Jose CA (US)

James Schultz of Redmond WA (US)

Agustya Mehta of San Carlos CA (US)

Gergana Angelova of San Jose CA (US)

Leonid Vladimirov of New York NY (US)

Fan Zhang of Kirkland WA (US)

Tiara Griffith of Mount Vernon NY (US)

Reading Out Scene Analysis for Users Wearing Head-mounted Devices - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240219720 titled 'Reading Out Scene Analysis for Users Wearing Head-mounted Devices

The abstract describes a method involving a head-mounted device that accesses sensor signals to analyze a scene and provide a description of objects of interest to the user.

• The method involves accessing sensor signals from cameras on a head-mounted device.
• Machine-learning models determine if criteria are met for analyzing the scene proactively.
• Objects of interest to the user are identified and described based on the analysis.
• The description includes relational information about the objects.
• Instructions are provided to present an audio readout of the description to the user.

Potential Applications: - Augmented reality applications - Assistive technology for visually impaired individuals - Enhanced user experiences in gaming or entertainment

Problems Solved: - Providing detailed descriptions of scenes for users - Enhancing accessibility and user interaction with technology

Benefits: - Improved user experience and engagement - Enhanced accessibility for individuals with visual impairments - Personalized and proactive analysis of scenes

Commercial Applications: Title: "Enhanced Augmented Reality Experiences for Users" This technology could be utilized in various industries such as gaming, entertainment, and assistive technology for visually impaired individuals. The market implications include improved user engagement, accessibility features, and potential partnerships with tech companies.

Questions about the technology: 1. How does this method improve user interaction with augmented reality technology? 2. What are the potential implications of this technology for individuals with visual impairments?

Original Abstract Submitted

in one embodiment, a method includes accessing sensor signals associated with a scene in a field of view of cameras of a head-mounted device associated with a first user from the head-mounted device, determining whether one or more criteria are satisfied for triggering a proactive analysis of the scene based on the sensor signals by machine-learning models, analyzing the scene to identify objects of interest to the first user in the scene based on the determination that the criteria are satisfied, generating a description of the objects based on an analysis of the objects and the sensor signals, wherein the description comprises relational information associated with one or more of the objects, and providing instructions to the head-mounted device to present an audio readout of the description to the first user.