Game and Simulation Control
Information Technology and Software
Game and Simulation Control (LAR-TOPS-88)
Control modification through user
physiological state
Overview
NASA's Langley Research Center has developed a
technology at the forefront of a new generation of computer
and video game environments that train valuable mental
skills, beyond eye-hand coordination, for the personal
improvement, not just the diversion, of the user.
Monitoring and enhancement of operator state
is an objective of the current LaRC Intelligent Integrated
Flight Deck Technology (IIFDT) program. Prior research
by the inventor, Alan Pope, modulate (based on player
physiological signals) the manual inputs that a player makes
to the buttons or joysticks of a video game hand controller.
However, a new type of controller allows a player to make
inputs to a video game by moving the entire controller itself,
allowing the present inventions entirely new approach to
integrating psychophysiological signals into game play.
The Technology
The technology is constructed to allow modulation of player inputs to a video game
or simulation from a user interface device based on the players psychophysiological
state. The invention exploits current wireless motion-sensing technologies to utilize
physiological signals for input modulation. These include, but are not limited to, heart
rate, muscle tension, and brain wave activity.
The current capability has been successfully prototyped using the Nintendo Wii
console and wireless Wii remote. The experience of electronic game play may also
be enhanced by introducing a multiplayer component in which various players
collaboratively pursue the goals of the game. The device can also enhance multiplayer
experiences such as a video game tournament, in which the skill set required in
competitive game play is increased by allowing players to interact with the game, and
compete with one another, on a psychophysiological level. This system is compatible
with the Nintendo Wii, and prototypes have been designed and are being developed
to extend this capability to the PlayStation Move, Xbox Kinect, and other
similar game platforms.
Benefits
- User-Control Interaction: Introduces, for the first time, user-control interaction with the Wii video game system via the users physiological signals.
- Variety: Can be used with several physiological signal-measuring devices (heart rate, muscle tension, and brain wave activity).
- Wireless: Operates with third-generation gaming consoles wirelessly.
- Low Power: Use of LED infrared communication reduces device power consumption.
- Healthy Living: Encourages healthenhancing physiological self-regulation skills and therapeutic amplification of healthful physiological characteristics.
- LED Embodiment: The current invention interacts wirelessly with the Wii remote through infrared signal.
Applications
- Consumer brain-computer interface devices
- Biofeedback equipment
- Third generation video game systems
- Physical therapy
- Third-party video game peripherals
- Athletic training
- Mind-body medicine
Similar Results
ZONE (Zeroing Out Negative Effects)
The system uses perturbation feedback to help the athlete get into the zone through an original method of ZONE. The method allows a trainee to learn physiological self-regulation in order to modify the difficulty of the performance task and/or environment in which training is conducted. For example, better concentration leads to a variety of easier conditions on a training putting green.
The technology incorporates software and hardware to provide real-time feedback to the athlete about how close his or her arousal and emotive responses are to an optimal state required to successfully perform the athletic task. This innovation presents the capability to extend current sports training and psychological practices of guided imagery visualization and cognitive reinforcement learning by systematically providing
demonstrable and relevant feedback through the use of closed-loop, cybernetic feedback principles that provide immediate reinforcement of pyschophysiological self-regulation and translate into better skill-based performance.
Foot Pedal Controller
The Foot Pedal Controller enables an operator of a spacecraft, aircraft, or watercraft, or a simulation of one in a video game, to control all translational and rotational movement using two foot pedals. This novel technology allows control across all six degrees of freedom, unlike any technology on the market. The components of the technology are a support structure, a left foot pedal, a right foot pedal, and supporting electronics. The Foot Pedal Controller is intuitive, easy to learn, and has ergonomic features that accommodate and stabilize the operator's feet. A working prototype is available to demonstrate key technology features to potential licensees.
The Foot Pedal Controller technology could be used in designs for the flight deck of the future, video game controls, drone operations and flight simulators. This technology can be useful in any application where it is preferred or desirable to use the feet to control motion rather than using the hands. A potential market could be foot control of equipment by people with arm or hand disabilities. A unique aspect of the innovation is the consideration of natural foot mechanics in the design and placement of the sensors and actuators to reduce operator fatigue. The axes of rotation of the Controller align with the joints of the foot so the foot moves naturally to control the movement of the craft. NASA seeks collaborations with companies interested in licensing and partnering to further develop and commercialize the technology.
Biocybernetic VR/AR Training System for De-Escalating Conflict
NASA’s biocybernetic system is a cutting-edge technology designed to cultivate emotional regulation skills. It leverages the concept of biocybernetic adaptation, where the trainee engages with virtual entities, such as characters in VR/AR/MR environments, whose behavior dynamically responds to the trainee's physiological signals. This responsive system provides real-time feedback, incentivizing the trainee to attain a calmer physiological state.
The key components of this VR innovation include:
· Head-mounted display hardware
· Physiological monitoring hardware, tracking heart rate, breathing, sweat, breath, and brain waves
· Software, powered by the Biocybernetic Loop (BL) Engine, integrating physiological data into the VR simulation
· Character response avatars
· Integration of the trainee's biofeedback data with the VR environment
This technology relies on two functional elements working in unison to adapt the behavior and appearance of VR/AR/MR characters. Inference of the trainee's emotional state from physiological signals requires the implementation of advanced machine learning and modeling techniques. A pattern comparator stores templates of physiological patterns and continually assesses the proximity of the trainee's real-time physiological activity to the desired patterns.
The pattern comparator calculates a closeness score in relation to one or more reference patterns, transmitting this data to the VR/AR/MR environment components. Consequently, the level of threat or cooperation presented by virtual characters is dynamically adjusted in response to the closeness score, creating an immersive and adaptive training experience.
Improved Ground Collision Avoidance System
This critical safety tool can be used for a wider variety of aircraft, including general aviation, helicopters, and unmanned aerial vehicles (UAVs) while also improving performance in the fighter aircraft currently using this type of system.
Demonstrations/Testing
This improved approach to ground collision avoidance has been demonstrated on both small UAVs and a Cirrus SR22 while running the technology on a mobile device. These tests were performed to the prove feasibility of the app-based implementation of this technology. The testing also characterized the flight dynamics of the avoidance maneuvers for each platform, evaluated collision avoidance protection, and analyzed nuisance potential (i.e., the tendency to issue false warnings when the pilot does not consider ground impact to be imminent).
Armstrong's Work Toward an Automated Collision Avoidance System
Controlled flight into terrain (CFIT) remains a leading cause of fatalities in aviation, resulting in roughly 100 deaths each year in the United States alone. Although warning systems have virtually eliminated CFIT for large commercial air carriers, the problem still remains for fighter aircraft, helicopters, and GAA.
Innovations developed at NASAs Armstrong Flight Research Center are laying the foundation for a collision avoidance system that would automatically take control of an aircraft that is in danger of crashing into the ground and fly it—and the people inside—to safety. The technology relies on a navigation system to position the aircraft over a digital terrain elevation data base, algorithms to determine the potential and imminence of a collision, and an autopilot to avoid the potential collision. The system is designed not only to provide nuisance-free warnings to the pilot but also to take over when a pilot is disoriented or unable to control the aircraft.
The payoff from implementing the system, designed to operate with minimal modifications on a variety of aircraft, including military jets, UAVs, and GAA, could be billions of dollars and hundreds of lives and aircraft saved. Furthermore, the technology has the potential to be applied beyond aviation and could be adapted for use in any vehicle that has to avoid a collision threat, including aerospace satellites, automobiles, scientific research vehicles, and marine charting systems.
Apparatus and Method for Biofeedback Training
Measured values of physiological signals may be associated with physiological states and may be used to define the presence of such states. For example, in a physiological state of anxiety, adrenaline diverts blood from the body surface to the core of the body in response to a perceived danger. As warm blood is withdrawn from the surface of the skin, the skin temperature drops. Similarly, in a physiological state of stress, perspiration generally increases making the skin more conductive to the passage of an electrical current, thereby increasing the galvanic skin response.
It is well known in the field of performance psychology that the peak performance of a task, such as, for example, putting in golf, foul shooting in basketball, serving in tennis, marksmanship in archery or on a gunnery range, shooting pool, or throwing darts, requires the presence of a physiological state, comprising one or more optimal measured values of physiological signals, coincident with the physical performance of the task. The presence of such an optimal physiological state in athletics is colloquially referred to as being in the zone.
The technology provides:
an apparatus and method of performance-enhancing biofeedback training that has intuitive and motivational appeal to the trainee, by tightly embedding the biofeedback training in the actual task whose performance is to be improved; and,
an apparatus and method of performance-enhancing biofeedback training that is operational in real-time, precisely at the moment when a task or exercise, such as an athletic or military maneuver, is required to be performed.
The feedback behavior of the physical environment provided by the present invention has the added benefit of providing aids to visualization that the trainee can use in the real-world skill performance setting.