18404111. Simultaneous Controller and Touch Interactions simplified abstract (Meta Platforms Technologies, LLC)

Simultaneous Controller and Touch Interactions

Organization Name

Meta Platforms Technologies, LLC

Inventor(s)

Ali Emami of San Diego CA (US)

Aaron Selig of Watsonville CA (US)

Ahad Habib Basravi of Oakland CA (US)

James Michael-K O'donnell of Pacifica CA (US)

Nathaniel Lane of Wrentham MA (US)

Todd Harris of Woodinville WA (US)

Simultaneous Controller and Touch Interactions - A simplified explanation of the abstract

This abstract first appeared for US patent application 18404111 titled 'Simultaneous Controller and Touch Interactions

Abstract: Aspects of the present disclosure are directed to artificial reality (XR) controllers and hand gestures and corresponding representation of the hand gestures in XR space. Users can benefit from being able to use touch interactions while also holding the XR controller. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things-using touch in conjunction with controllers. Thus, if the user has a controller in her hand and wants to touch a UI element, she can also use her finger, where the system identifies the finger not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera) to allow that finger to provide hand input, without the user having to put down the controller.

• Simplified Explanation:

Users can use hand gestures in conjunction with XR controllers to interact with virtual environments, allowing for touch interactions while holding the controller.

• Key Features and Innovation:

- Recognition of hand gestures in XR space - Integration of touch interactions with XR controllers - Seamless switching between controller interactions and manual touch interactions

• Potential Applications:

- Virtual reality gaming - Training simulations - Remote collaboration tools

• Problems Solved:

- Enhances user experience by allowing for more natural interactions in XR space - Eliminates the need to constantly switch between controller and manual touch interactions

• Benefits:

- Improved user immersion in virtual environments - Increased efficiency in performing tasks in XR space - Enhanced user comfort during interactions

• Commercial Applications:

The technology can be utilized in various industries such as gaming, education, healthcare, and design for enhanced user experiences and improved productivity.

• Questions about XR:

1. How does the technology differentiate between controller interactions and manual touch interactions? 2. What are the potential limitations of using hand gestures in XR space?

• Frequently Updated Research:

Stay updated on advancements in XR technology and hand gesture recognition for improved user experiences in virtual environments.

Original Abstract Submitted

Aspects of the present disclosure are directed to artificial reality (XR) controllers and hand gestures and corresponding representation of the hand gestures in XR space. Users can benefit from being able to use touch interactions while also holding the XR controller. Example implementations allow a user to use controllers for distance interaction and interactions that require controller buttons, but also recognize when the user extends her finger and wants to manually touch things-using touch in conjunction with controllers. Thus, if the user has a controller in her hand and want to touch a UI element, she can also use her finger, where the system identifies the finger not touching the controller (e.g., via capacitance sensors) and that a hand pose is present (e.g., finger extended—from camera) to allow that finger to provide hand input, without the user having to put down the controller.