20240019980. CONTROLLING A 2D SCREEN INTERFACE APPLICATION IN A MIXED REALITY APPLICATION simplified abstract (KONINKLIJKE PHILIPS N.V.)

CONTROLLING A 2D SCREEN INTERFACE APPLICATION IN A MIXED REALITY APPLICATION

Organization Name

KONINKLIJKE PHILIPS N.V.

Inventor(s)

PHILIP JOSEPH Keenan of EINDHOVEN (NL)

MOLLY LARA Flexman of MELROSE MA (US)

BENOIT JEAN-DOMINIQUE BERTRAND MAURICE Mory of MEDFORD MA (US)

ASHISH SATTYAVRAT Panse of BURLINGTON MA (US)

PENELOPE EUGENIA Webb of SOMERVILLE MA (US)

ROMAN WICKY Van Doyer of EINDHOVEN (NL)

CONTROLLING A 2D SCREEN INTERFACE APPLICATION IN A MIXED REALITY APPLICATION - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240019980 titled 'CONTROLLING A 2D SCREEN INTERFACE APPLICATION IN A MIXED REALITY APPLICATION

Simplified Explanation

The present invention is a device for controlling a 2D application in a mixed reality environment, aimed at simplifying the operation of devices and equipment for medical personnel. The device includes a 2D application interface, a processor, a mixed reality application interface, and a control interface.

• The 2D application interface receives data representing a 2D application, which contains interactive elements for user control.
• The processor identifies and extracts the interactive elements from the data.
• The processor translates the interactive elements into user interface features of a mixed reality display application.
• The processor generates data for presenting the mixed reality display application and maps user input to commands for controlling the 2D application.
• The mixed reality application interface provides the mixed reality display application to the user.
• The control interface receives user input for the user interface features and sends commands to the 2D application to control it.

Potential applications of this technology:

• Medical training: Medical personnel can use this device to control 2D applications in a mixed reality environment, enhancing their training experience.
• Surgical procedures: Surgeons can use the device to control 2D applications during surgeries, improving their efficiency and accuracy.
• Rehabilitation: Patients undergoing rehabilitation can interact with 2D applications in a mixed reality environment, making their therapy more engaging and effective.

Problems solved by this technology:

• Simplifies the operation of devices and equipment for medical personnel by integrating traditional 2D applications into a mixed reality environment.
• Provides a more intuitive and immersive user interface for controlling 2D applications.
• Enhances the functionality and versatility of 2D applications by mapping them to augmented reality interfaces.

Benefits of this technology:

• Improved user experience: Users can interact with 2D applications in a more natural and immersive way.
• Increased efficiency: Medical personnel can perform tasks more efficiently and accurately using the device.
• Enhanced training and rehabilitation: The device provides a more engaging and effective platform for medical training and rehabilitation programs.

Original Abstract Submitted

the present invention relates to controlling a 2d application in a mixed reality application. in order to provide further simplification in terms of operating devices and equipment for the medical personnel, a device () for controlling a 2d application in a mixed reality application is provided. the device comprises a 2d application interface (), a processor (), a mixed reality application interface () and a control interface (). the 2d application interface receives data representative of a 2d application. the application comprises interactive elements providing commands for user control of the 2d application. the processor identifies and extracts the interactive elements based on the data representative of the 2d application. the processor also translates the interactive elements to user interface features of a mixed reality display application. the processor further generates data for presenting the mixed reality display application and maps user input to commands for user control of the 2d application. the mixed reality application interface provides the mixed reality display application to a user as mixed reality. the control interface receives input from the user for the user interface features and provides the commands to the 2d application in order to control the 2d application. hence, augmented reality interfaces are mapped to traditional 2d screen interfaces. this allows to integrate applications that only provided the interaction on a monitor with a mouse or the like to be operated in today's augmented reality environment.