ChatGPT and YouTube are two of the most frequently used services on the internet (per similarweb.com). To determine how their efficacy compares to VR sim learning, Transfr conducted a study with 24 community members in the Louisville, KY area. Participants used either a VR sim or a webpage which consisted of text generated by ChatGPT and two YouTube videos (which had close to 21,000 combined views). The participants had no prior experience performing diesel engine analyses.
Participants either completed the Transfr sim twice or reviewed the webpage and video content twice. Study time on the webpage was self-paced; participants could decide for themselves when they wanted to stop. Participants in both groups left the testing center and returned 1-7 days later (M = 2.5 for both groups) to attempt the engine analyses using real tools and materials.
Playing the sim led to significantly better real-world performance than studying the webpage, t(22) = 4.05, p < .001, d = 1.65. The likelihood that a random person in the sim condition scores higher than a random person in the webpage condition is 88%.
The effect we observed in this study is the kind learning scientists dream of. Obviously, we want to help people learn, but most interventions aimed at increasing learning produce only modest effects. In contrast, the effect we found here is huge! Playing the sim gave people a much better grasp on the material than studying the ChatGPT and YouTube content.
Participants rated how much they enjoyed the learning experience after each time they played the sim or studied the webpage. They also rated their enthusiasm for using a similar teaching tool (sim or webpage) to learn other tasks. Enjoyment and enthusiasm were statistically equivalent for the sim and the webpage (though they were numerically higher for the sim), which is encouraging given how popular video platforms and generative AI tools like YouTube and ChatGPT are. Scaling up VR learning faces many hurdles, but user enjoyment and enthusiasm when compared to other forms of learning are not among them.
Results from this study suggest that immersive VR training offers a more effective learning experience compared to other popular, readily-available resources. The superior performance observed days later implies a more enduring impact of VR training on skill retention.
