20240054794. Multistage Audio-Visual Automotive Cab Monitoring simplified abstract (Blueskeye AI Ltd)

Multistage Audio-Visual Automotive Cab Monitoring

Organization Name

Blueskeye AI Ltd

Inventor(s)

Michel François Valstar of Nottingham (GB)

Anthony Brown of Nottingham (GB)

Timur Almaev of Nottingham (GB)

Steven Cliffe of Calne (GB)

Thomas James Smith of Sheffield (GB)

Tze Ee Yong of Nottingham (GB)

Mani Kumar Tellamekala of Nottingham (GB)

Multistage Audio-Visual Automotive Cab Monitoring - A simplified explanation of the abstract

This abstract first appeared for US patent application 20240054794 titled 'Multistage Audio-Visual Automotive Cab Monitoring

Simplified Explanation

The patent application describes a system for analyzing the mental state of a subject in an automobile interior using video, audio, and context descriptors. The system processes video input of the subject using face detection and facial point registration modules to produce an output. This output is further processed by various modules such as facial point tracking, head orientation tracking, body tracking, social gaze tracking, and action unit intensity tracking. The audio input is processed by a valence and arousal affect states tracking module to produce a second output and valence and arousal scores. A temporal behavior primitives buffer produces a temporal behavior output. Based on these inputs and outputs, a mental state prediction module predicts the mental state of the subject.

• The system analyzes the mental state of a subject in an automobile interior using video, audio, and context descriptors.
• Video input of the subject is processed using face detection and facial point registration modules to produce an output.
• The output is further processed by various modules such as facial point tracking, head orientation tracking, body tracking, social gaze tracking, and action unit intensity tracking.
• Audio input of the subject is processed by a valence and arousal affect states tracking module to produce a second output and valence and arousal scores.
• A temporal behavior primitives buffer produces a temporal behavior output.
• A mental state prediction module predicts the mental state of the subject based on the inputs and outputs.

Potential applications of this technology:

• Automotive safety: The system can be used to monitor the mental state of drivers and passengers, providing insights into their attention, fatigue, and emotional state, which can help prevent accidents.
• Personalized in-car experience: The system can analyze the mental state of the occupants and adjust the environment accordingly, such as playing calming music or adjusting the temperature based on their emotional state.
• Health monitoring: The system can be used to monitor the mental well-being of individuals, detecting signs of stress, anxiety, or depression, and providing appropriate interventions or alerts.

Problems solved by this technology:

• Lack of real-time understanding of the mental state of individuals in an automobile interior.
• Difficulty in detecting and tracking facial expressions, head orientation, body movements, and social gaze in a dynamic environment.
• Limited ability to assess the emotional state of individuals based on audio cues alone.

Benefits of this technology:

• Improved safety on the road by detecting and alerting drivers to their mental state.
• Enhanced comfort and well-being of occupants by creating a personalized in-car experience.
• Early detection of mental health issues and timely interventions.

Original Abstract Submitted

described is a task for an automobile interior having at least one subject that creates a video input, an audio input, and a context descriptor input. the video input relates to the at least one subject and is processed by a face detection module and a facial point registration module to produce a first output. the first output is further processed by at least one of: a facial point tracking module, a head orientation tracking module, a body tracking module, a social gaze tracking module, and an action unit intensity tracking module. the audio input relating to the at least one subject is processed by a valence and arousal affect states tracking module to produce a second output and to produce a valence and arousal scores output. a temporal behavior primitives buffer produce a temporal behavior output. based on the foregoing, a mental state prediction module predicts the mental state of at least one subject in the automobile interior.