MetaVR and Varjo introduce eye tracking capability for pilot performance review training
Sudbury, MA, 3 June, 2021: MetaVR and Varjo have combined the capabilities of Virtual Reality Scene Generator (VRSG) and the Varjo XR-3 mixed-reality headset to provide a real-time, pilot eye gaze tracking facility to enhance After Action Review (AAR) for simulator-based pilot training.
The Varjo XR-3 mixed-reality headset includes a built-in, real-time 200 Hz pupil-tracking capability that can precisely capture even the smallest eye movement of the wearer when training in virtual reality and mixed-reality environments. MetaVR’s VRSG version 6.6* includes the means to harness this eye-tracking data. At the end of the training mission, VRSG exports this data via DIS as a PDU log and visualizes the pilot’s head position, orientation, and gaze vector (with the gaze of each eye depicted as a color-coded cone) over the events of the training session for AAR.
During training mission debrief, the trainer and the pilot can see where the pilot’s head position and eye movements were directed throughout the simulated air engagement, providing the ability to objectively assess the engagement so as to fully understand what lead to the mission outcome. From a tactical perspective, the eye-tracking playback can help identify missed moments of attention to instrumentation or the direction of an important activity, help identify areas for improvement and track pilot progress throughout the training cycle. It also has application for beyond-visual range systems management review.
In addition to recording the eye tracking visualization for after-action playback, the VRSG session with the eye tracking visualization can also be monitored by instructors in real time during the simulated engagement, to further enhance understanding of the nature of the training engagement.
Among the options for configuring VRSG with a Varjo XR system is foveated rendering. Foveated rendering increases the performance of rendering the entire scene in the headset. This option enables the GPU to render the scene at full fidelity in the area that the headset wearer is looking, while reducing fidelity in the peripheral areas of the scene.
Urho Konttori, Co-Founder and Chief Technology Officer, Varjo, said: “What’s best, pilot training can be enriched without adding to the trainee’s cognitive burden. This can help achieve better learning outcomes and improved live missions down the line. The realistic simulation of flight scenarios in virtual and mixed reality is not science fiction. It’s here today already with Varjo XR-3, already producing tangible value for pilot training.”
“Essentially what we are doing here is turning a normally subjective process into a quantitative review, enriching pilot simulator training and improving learning outcomes for trainees,” Bert Haselden, Mixed-Reality Lead, MetaVR, said. “The Varjo XR-3 gives pilots the most natural and fully immersive training possible; when combined with VRSG the result is a capability that is – we believe – unique to the current simulator training space.”
* Eye tracking visualization is a VRSG version 6.6 feature being offered in advance.
