Fraunhofer Institute for Computer Graphics Research IGD
The work of operational forces in the offshore sector is subject to demanding and constantly changing stresses. It requires a continuous ability to manage risks and handle emergency situations. Current methods to raise awareness of hazards among these forces rely primarily on extensive training scenarios under real conditions, which are associated with significant effort and high costs.
The work of operational forces in the offshore sector is subject to demanding and constantly changing stresses. It requires a continuous ability to manage risks and handle emergency situations. Current methods to raise awareness of hazards among these forces rely primarily on extensive training scenarios under real conditions, which are associated with significant effort and high costs.
The work of operational forces in the offshore sector requires mastery of various emergency situations. Specifically, specialists who are transported to their work site by helicopter must learn how to evacuate themselves and others from the helicopter cabin underwater in the event of an emergency water landing. Current training methods are largely based on extensive real-life scenarios, which come with high costs and effort. Additionally, the high physical and psychological stress placed on participants often leads to training being interrupted.
The "Next Reality" project aims to use innovative Mixed Reality (MR) technologies to enable training of action competencies in stressful situations without putting participants at risk. To this end, Fraunhofer IGD, together with its project partners OffTEC Base GmbH & Co. KG, Meilenstein Digital GmbH, and the Christian-Albrechts-University of Kiel, is researching and developing solutions in the fields of Virtual Reality, biofeedback, and haptic actuation.
The result will be immersive training scenarios that simulate psychological and physical stress factors. This will allow participants to prepare for extreme conditions without being exposed to actual risks. An audio-visual VR application will be combined with additional sensory stimuli, such as a tactile and manipulable environment, simulated shortness of breath, and the transmission of movements, shocks, and vibrations to the body.
Our platform connects the MR environment with a mobile control application, allowing the trainer to monitor and adjust the training scenario. At the same time, vital data (heart rate, temperature, etc.) is measured to assess the participant's physical and psychological stress levels. This data is wirelessly transmitted to a server, automatically analyzed, and fed back to the control application. Based on this, manual or automatic adjustments can be made to the intensity of the simulated emergency situation to guide the trainees toward success.
Next Reality enables the exploration of cutting-edge MR technologies in the context of stress-inducing training situations, making the effects on trainees observable, measurable, and evaluable. The developed concepts and modules can be applied to similar scenarios, supporting the successful learning and practicing of emergency response procedures.
The psychophysical, ethical, and legal implications, as well as the acceptance of these new training methods, are additional research topics of the project. In the future, further components will be developed to enhance the VR experience, and new, AI-based algorithms for assessing stress levels and adjusting the VR environment will be researched.
Scientific Background
Through immersion in Mixed Reality, behavior in situations of psychological and physical stress is to be specifically trained. A key research question is what quality of MR representation is necessary in multi-user scenarios with users of varying qualifications and experience (trainers, trainees, technicians, first responders) to achieve the desired immersion and situational awareness in highly demanding, yet safely altered environments.
