A serious bus accident has occurred in a tunnel with twenty people injured and in need of emergency medical care. What is the best way to handle the situation as a first responder?

This was the training scenario in the initial Field Trial of MED1stMR held at the Johanniter Research and Innovation Centre in Vienna led by Johanniter Austria. From 25th to 28th July, trainers and first responders tested the mixed reality training system that was developed by the project consortium in the past two years.

Four medical first responders of Johanniter Austria stared the Field Trial and practiced on lifelike manikins in a hall of Johanniter’s Research and Innovation Centre wearing VR goggles, headphones as well as sensors attached to the lower back, hands, and shoes. Their every movement was closely watched by members of the MED1stMR consortium via multiple screens next to the training field.

Sensors recorded vital signs to measure stress levels while the movements of the first responders were tracked and transmitted to the screens of the team on the sidelines. On the screen, the virtual avatars of the four paramedics could be seen, imitating every step and every movement of their real counterparts.

This Mixed Reality environment combines aspects from the physical world – such as the medical high-end patient simulation manikins and real equipment- with aspects from the virtual world. Trainees can thus immerse themselves more strongly in the scenarios and perceive the body, limbs, and movements tactilely and visually when examining and treating patients.

“It is common practice to train for major emergencies with real-life exercises. That means half a year of preparation and the involvement of 150 people – for example, role players are needed to play the injured,” says Helmut Schrom-Feiertag, project coordinator at AIT.

With the new solution, “you have a stressful crisis scenario at the push of a button, so to speak, where you can send the team in and train them,” Schrom-Feiertag continues. Not only can training take place more often, but it also saves resources. Another advantage over conventional training exercises is the documentation – the training sessions are recorded. This provides a good data basis afterward, which is helpful for debriefing.

Each day, different groups of medical first responders trained in the MED1stMR solution supported by educated Mixed Reality Trainers. About 50 first responders tested the prototype training solution over the week. Before, during, and after the training, research on different levels was conducted to include these results (stress data, behaviour, the usability of the technology, etc.) in the forward development of the project. 

The biosignals data was analysed and interpreted as individual stress levels in a backend and displayed together with other training information to the observing trainer in a user-friendly dashboard. This dashboard provides the trainer with the virtual scenario flow, the decisions of the trainees, and their performance data. Based on this evidence, the trainer is able to change the scenario to provide the best learning results for the trainees. Additionally, the scenario can be controlled automatically with artificial intelligence (based on biosignals and behavioural data) in order to support trainers and optimise outcomes. 

The following Field Trial will be held in Heidelberg, Germany as a City exercise from 25th to 29th September.