Virtual Reality & Flight Training

With the advent of consumer priced Virtual Reality (VR) hardware, such as the Oculus Rift, HTC Vive and others, the benefits of this technology has found its way to many applications. The main benefit of virtual reality to flight training is the immersive representation of the flight experience, providing some important learning advantages over traditional flight simulators. Key benefits to flight schools include reduced training costs, cost savings on aircraft familiarization training and faster training of students.

Virtual Reality basics

Virtual Reality (VR) is the concept of being immersed into a computer generated environment with a visual, audible and optionally haptical representation of the environment. This environment, e.g. a room, landscape or a cockpit, may be presented to the user through a screen or a head mounted display (headset). The user may be able to interact with the environment, e.g. through gestures or physical buttons or levers.

VR is used in many fields including entertainment (video gaming), robotic control (controlling ROVs, robotic surgery), therapy (treatment of anxiety disorders and PTSD (Posttraumatic Stress Disorder), meditation and educational purposes. The overall advantage of using VR is the feeling of “being there” or immersion. The level of immersion is determined by the sophistication level of the VR setup. An immersive VR experience requires a high screen frame rate, high resolution displays, a low system latency and sophisticated graphics. The immersion level may be increased by adding 3D audio (binaural audio), controllers for interaction and haptic (touch) feedback. If the VR experience is realistic enough, the user’s mind can be tricked into believing the virtual world is real.

The Learning Process

VR offers new opportunities to greatly enhance teaching and learning as subjects can be taught in a very different way than reading a textbook or listen to a lecturer.

Knowledge retention

One of the most important aspects of learning is knowledge retention – the ability to remember what you learn. This is especially true for flight crew, who must memorize dozens of checklists, rules and procedures. Studies have shown that VR-assisted learning has the potential of improving knowledge retention by up to 400 %, adding value to the time spent studying. Better knowledge retention means less retraining and in the end, better pilots who remember the emergency checklist better thanks to VR-assisted learning.

Referring to Edgar Dale’s Cone of Experience (see figure 1), students only remember 10 % of what they read, but 90 % of what they do. This supports the old saying about learning by doing.

Figure 1: Edgar Dale’s Cone of Experience. Image courtesy: Jeffrey Anderson, CC BY-SA 3.0

Studies have shown that VR engages the student much more in the learning process, thus making the student remember more of what he or she learns. This is an obvious advantage when performing tasks as flight crew where complex procedures must be memorized.

Faster learning

Numerous studies (1, 2, 3, 4) have shown a dramatic reduction in training time when using VR simulations – for flight training application as much as one year down to four months.

Benefits for flight training

Depth perception

The ability to judge distances correctly is a major learning objective in flight training. An experienced pilot may judge distances better due to his or her vast experience of judging distances. A new pilot student lacks this experience and needs to build it from flying experience. This aspect of flight training is difficult to train in traditional simulators, where there is no depth perception. This is because the screen, onto which the outside world is projected, is placed at a fixed distance from the eyes of the student pilot, and every object projected onto the screens would appear to be at the same distance from the pilot – whether it’s the runway 5 meters from the pilot or a tower 5 kilometers from the pilot.

Virtual Reality goggles offer stereoscopic screens that present two slightly different images of the same scene. This gives the sense of depth and distance in the same way we are able to judge distance with our natural, stereoscopic vision – i.e. our two eyes. Thus, VR is able to accurately and intuitively represent distances in a flight simulation where this aspect is crucial, e.g. when practicing landings where the altitude above the runway in the flare is important to judge correctly, or hovering a helicopter half a meter above the tarmac.

Figure 2a and 2b show the difference in distance perception between a curved screen typically used in traditional flight simulators, and VR goggles. Three targets marked by red, green and blue X’s are represented at the same distance on a curved screen, while they in VR appear to be at their correct distance from the observer. Targets further away from the observer than the screen appear closer (the blue X), while targets closer to the observer than the screen appear further away (the green X). Only when the target is at the same distance from the observer as the screen will the distance perception be correct (the red X).

Figure 2a: Perceived distance of three targets (red, green, blue) seen on a curved screen.
2b: Perceived distance of three targets (red, green, blue) seen through VR goggles.

360 degree vision

Another important lesson every pilot must learn is to perform a proper lookout. Lookout must be performed to watch for traffic and when performing a landing circuit. Traditional flight simulators rarely have a field of vision of more than 180 degrees (see figure 3a), which severely limits the possibility of performing a proper lookout. In these simulators, pilots who train must often resort to alternative methods of reference, such as timing their turns, because they cannot use the lookout procedures they would use the the real aircraft. Virtual Reality goggles allows the student pilot to look in any direction using accelerometers and gyroscopes (see figure 3b). This means the student may look beyond the 180 degree field of view provided by traditional flight simulators, and is able to practice lookouts the same way he or she would do it in the real aircraft.

Figure 3a: Field of view on a curved screen.
Figure 3b: Field of view with VR goggles.

Scalability and modularity

A major cost of flight simulators are the physical cockpit itself. The components used to built the cockpit are often the same, qualified aircraft parts used on the real aircraft, making these simulators quite expensive. Depending on the fidelity of the VR simulation, only the basic flight controls are physically available while the rest of the cockpit is represented purely with computer graphics in the VR goggles. This is not only much cheaper to make, but it also allows the same simulator to take the appearance of a completely different aircraft in a matter of seconds. For flying schools operating more than one type, this has the benefit of the flying school only having to invest in one simulator for all the aircraft types they operate.

Conclusion

In the pursuit to reduce flight training costs, the use of virtual reality as a key part of the flight training process has great cost saving potential due to the reduced need for large hardware components and modularity possibilities. Not only can VR training reduce costs for flight schools, but the students are more likely to learn faster and more efficiently, as well as remembering the lessons taught better.

References

https://xd.adobe.com/ideas/principles/emerging-technology/virtual-reality-will-change-learn-teach/

https://fortune.com/2015/08/17/virtual-reality-hospitals/

https://trainingindustry.com/articles/learning-technologies/3-ways-virtual-reality-training-is-producing-better-outcomes/

https://www.airforcetimes.com/news/your-air-force/2019/10/15/cracking-the-da-vinci-code-virtual-reality-accelerates-helicopter-aviator-training/

https://www.stripes.com/news/us/virtual-reality-heavy-course-speeds-up-air-force-helicopter-pilot-training-by-six-weeks-1.603374

https://www.airforcetimes.com/news/your-air-force/2018/09/30/the-air-force-is-revolutionizing-the-way-airmen-learn-to-be-aviators/

https://www.unity-studios.com/news/oplaering-grundfos/