Biocybernetic VR/AR Training System for De-Escalating Conflict
Information Technology and Software
Biocybernetic VR/AR Training System for De-Escalating Conflict (LAR-TOPS-377)
Adapts VR/AR environment to trainee ‘s or player’s biofeedback signals
Overview
NASA Langley’s innovative biocybernetic system employs a blend of hardware and software within a virtual/augmented/mixed reality (VR/AR/MR) environment. Its purpose is to provide psychophysiological self-regulation training, specifically geared toward helping individuals de-escalate conflicts in real-world scenarios. In this system, trainees or their avatars engage with simulated social situations involving conflict. The virtual characters they interact with are designed to present varying levels of threat or cooperation. What sets this approach apart is its continuous measurement of the trainee's cognitive and emotional states through biofeedback.
The system's feedback mechanism is distinct, and drives the behavior of the virtual characters. Trainees are rewarded when they effectively achieve the desired mental and physiological states, resulting in desirable character responses. Conversely, ineffective states lead to undesirable character behavior, effectively penalizing the trainee.
NASA has tailored this technology for de-escalation training, aiming to reduce physical responses in situations such as customer service interactions, flight attendant-passenger interactions, or police encounters. The system adapts the behavior of non-player characters based on the trainee's ability to attain the desired state, providing a dynamic and immersive training experience.
The Technology
NASA’s biocybernetic system is a cutting-edge technology designed to cultivate emotional regulation skills. It leverages the concept of biocybernetic adaptation, where the trainee engages with virtual entities, such as characters in VR/AR/MR environments, whose behavior dynamically responds to the trainee's physiological signals. This responsive system provides real-time feedback, incentivizing the trainee to attain a calmer physiological state.
The key components of this VR innovation include:
· Head-mounted display hardware
· Physiological monitoring hardware, tracking heart rate, breathing, sweat, breath, and brain waves
· Software, powered by the Biocybernetic Loop (BL) Engine, integrating physiological data into the VR simulation
· Character response avatars
· Integration of the trainee's biofeedback data with the VR environment
This technology relies on two functional elements working in unison to adapt the behavior and appearance of VR/AR/MR characters. Inference of the trainee's emotional state from physiological signals requires the implementation of advanced machine learning and modeling techniques. A pattern comparator stores templates of physiological patterns and continually assesses the proximity of the trainee's real-time physiological activity to the desired patterns.
The pattern comparator calculates a closeness score in relation to one or more reference patterns, transmitting this data to the VR/AR/MR environment components. Consequently, the level of threat or cooperation presented by virtual characters is dynamically adjusted in response to the closeness score, creating an immersive and adaptive training experience.
Benefits
- Enhances Social Skills: Professionals like law enforcement officers dealing with agitated citizens or soldiers interacting with antagonistic civilians can improve their de-escalation and self-regulation skills.
- Elevates Gaming Experience: In a VR setting, it empowers game characters/entities to adapt and respond to the player's physiological changes, making games more engaging.
- Offers Real-Time Consequences and Rewards for Self-Control: It provides immediate feedback for managing one's mental state, making it a valuable tool for self-improvement.
- Prepares for Sound Decision-Making in High-Stress Situations: Users can train to make effective decisions under extreme stress, which can be crucial in various fields.
- Measures Mental State: This technology enables the objective measurement of physiological responses to stress, allowing for accurate benchmarking and tracking of improvements.
- Proven Proof of Concept: At a Technology Readiness Level (TRL) of ¾, this technology is ready for adaptation into specific use cases, showcasing its viability and readiness for implementation.
Applications
- De-escalation training: Law enforcement, military agencies for soldier training, airline training for pilots, flight attendants, and gate agents, and classroom teacher training
- Virtual and Augmented Reality gaming: Enhancing interaction and system intelligence via real-time adaptations based on detected conscious or unconscious human neurological and other biophysical states.
- Autonomous vehicles: Monitoring the psychological states of drivers to inform the design of the autonomous car experience “feeling of being safe” in their self-driving car to optimize the driver’s mental state.
Technology Details
Information Technology and Software
LAR-TOPS-377
LAR-19961-1
|
Tags:
|
Similar Results
Apparatus and Method for Biofeedback Training
Measured values of physiological signals may be associated with physiological states and may be used to define the presence of such states. For example, in a physiological state of anxiety, adrenaline diverts blood from the body surface to the core of the body in response to a perceived danger. As warm blood is withdrawn from the surface of the skin, the skin temperature drops. Similarly, in a physiological state of stress, perspiration generally increases making the skin more conductive to the passage of an electrical current, thereby increasing the galvanic skin response.
It is well known in the field of performance psychology that the peak performance of a task, such as, for example, putting in golf, foul shooting in basketball, serving in tennis, marksmanship in archery or on a gunnery range, shooting pool, or throwing darts, requires the presence of a physiological state, comprising one or more optimal measured values of physiological signals, coincident with the physical performance of the task. The presence of such an optimal physiological state in athletics is colloquially referred to as being in the zone.
The technology provides:
an apparatus and method of performance-enhancing biofeedback training that has intuitive and motivational appeal to the trainee, by tightly embedding the biofeedback training in the actual task whose performance is to be improved; and,
an apparatus and method of performance-enhancing biofeedback training that is operational in real-time, precisely at the moment when a task or exercise, such as an athletic or military maneuver, is required to be performed.
The feedback behavior of the physical environment provided by the present invention has the added benefit of providing aids to visualization that the trainee can use in the real-world skill performance setting.
System for Incorporating Physiological Self-Regulation Challenge into Parcourse/Orienteering Type Games and Simulations
Although biofeedback is an effective treatment for various physiological problems and can be used to optimize physiological functioning in many ways, the benefits can only be attained through a number of training sessions, and such gains can only be maintained over time through regular practice. However, adherence to regular training has been a problem that has plagued the field of physiological self-regulation limiting its utility. As with any exercise, incorporating biofeedback training with another activity encourages participation and enhances its usefulness.
ZONE (Zeroing Out Negative Effects)
The system uses perturbation feedback to help the athlete get into the zone through an original method of ZONE. The method allows a trainee to learn physiological self-regulation in order to modify the difficulty of the performance task and/or environment in which training is conducted. For example, better concentration leads to a variety of easier conditions on a training putting green.
The technology incorporates software and hardware to provide real-time feedback to the athlete about how close his or her arousal and emotive responses are to an optimal state required to successfully perform the athletic task. This innovation presents the capability to extend current sports training and psychological practices of guided imagery visualization and cognitive reinforcement learning by systematically providing
demonstrable and relevant feedback through the use of closed-loop, cybernetic feedback principles that provide immediate reinforcement of pyschophysiological self-regulation and translate into better skill-based performance.
Game and Simulation Control
The technology is constructed to allow modulation of player inputs to a video game
or simulation from a user interface device based on the players psychophysiological
state. The invention exploits current wireless motion-sensing technologies to utilize
physiological signals for input modulation. These include, but are not limited to, heart
rate, muscle tension, and brain wave activity.
The current capability has been successfully prototyped using the Nintendo Wii
console and wireless Wii remote. The experience of electronic game play may also
be enhanced by introducing a multiplayer component in which various players
collaboratively pursue the goals of the game. The device can also enhance multiplayer
experiences such as a video game tournament, in which the skill set required in
competitive game play is increased by allowing players to interact with the game, and
compete with one another, on a psychophysiological level. This system is compatible
with the Nintendo Wii, and prototypes have been designed and are being developed
to extend this capability to the PlayStation Move, Xbox Kinect, and other
similar game platforms.
Functional Near-Infrared Spectroscopy (fNIRS) Cognitive Brain Monitor
Functional near-infrared spectroscopy (fNIRS) is an emerging hemodynamic neuroimaging brain-computer interface (BCI) technology that indirectly measures neuronal activity in the brain's cortex via neuro-vascular coupling. fNIRS works by quantifying hemoglobin-concentration changes in the brain based on optical intensity measurements, measuring the same hemodynamic changes as functional magnetic resonance imaging (fMRI). With enough probes in enough locations, fNIRS can detect these hemodynamic activations across the subject's entire head, thus allowing the determination of cognitive state through the use of pattern classification. fNIRS systems offer low-power, low-cost, highly mobile alternatives for real-time monitoring in safety-critical situations.
Glenn's specific contribution to this field is the algorithms capable of removing motion artifacts (environment- or equipment-induced errors) from the device's head-worn optical sensors. In other words, Glenn's adaptive filter can determine the presence of a potential motion artifact based on a phase shift in the data measured; identify the artifact by examining the correlation between the phase shift and changes in hemoglobin concentration; and finally remove the artifact using Kalman filtering whenever changes in hemoglobin level and changes in the phase shift are not correlated. Glenn's breakthrough allows the advantages of fNIRS to be used for non-invasive real-time brain monitoring applications in motion-filled environments that could potentially save lives.
