One of the most basic questions we get asked here at Transfr regarding our product and VR, in general, is: “Is VR learning effective?”
The shortest answer to that question is “Yes!” but the longer answer is so much more interesting and satisfying! To get to the bottom of this vital question, we convened a special episode of Upward, the Transfr podcast featuring two members of our Learning Intelligence team: Yun Jin Rho, PhD — Director of Learning Intelligence, and Keith Lyle, PhD — Director of Learning Science. These two experts (and their teams of fellow researchers), represent a wealth of knowledge and expertise that they are aiming squarely at the question of “How effective is VR?”
In part one of this robust, engaging chat, we discussed Yun Jin and Keith’s backgrounds as well as what exactly their team does. We also dig into the nature of our efficacy studies and why we’re doing them. Part two digs into the discoveries we’ve gotten back from the studies themselves and our hopes for future studies.
Studying VR effectiveness to build better sims
Creating robust classroom-to-career pathways for people from diverse backgrounds and pools of undiscovered talent is a tall order: It takes multiple crews of simulation builders, seasoned SMEs and instructional designers, and lots of research from the Learning Intelligence team to build the best products and make sure that users are getting the most out of them. We can’t just release simulations out into the world and hope they’re effective, so we’ve assembled an industry-leading team of researchers to apply “academic rigor,” as Keith puts it, to determine their effectiveness.
“We leverage learning research, data science, and machine learning to understand how we can help our learners learn better. That allows us to create better and more impactful experiences for them,” explains Yun Jin. “We put our learning products under a constant evaluation process to figure out what works and what doesn’t.”
Information about which parts of the simulations are most effective and working smoothly — as well as areas for improvement go back to the Product Team and often result in incremental improvements over time in subsequent versions of the sims. The ultimate goal is sims that are effective at conveying the necessary information and enjoyable for the learner.
“Learning should be a fun part of our journeys,” says Yun Jin.
Keith Lyle calls this sweet spot “the joy of doing.” In various experiments, it’s come up that users find interactive VR more enjoyable than other, more static forms of learning.
“I’ve been studying memory for 25 years,” Keith says. “I’ve done countless classroom studies, but what was missing was a more interactive element, like you have with VR.”
Industry-leading experts uncover learning insights
“For me, it’s a continuation of my previous work and experience,” says Yun Jin. “We conduct research and analytics so that we can create a better learning experience for our learners. Learning and exploring different career options should be a fun part of our journeys.”
Research data is gathered in numerous ways: Some come directly from the Transfr system, some from volunteer surveys, and some from supervised laboratory studies. However, wherever the information is gathered, researchers handle it with the utmost care and are always focused on how they can use the data that comes back to deliver the best experience for future users.
Every company wants to know that its products are effective. When dealing with a product as complex as Transfr’s, a skilled, sophisticated team of experts is needed to truly understand how people are using our VR sims, what their outcomes are, and how the product can be improved in successive versions. Yun Jin and team represent decades of experience in learning sciences and have the skills, background, and commitment to excellence (and the highest ethical standards in research) that are necessary to uncover deep insights about Transfr:
“We have a whole spectrum of outcomes that we measure,” says Yun Jin. “The way that we measure the outcomes is that we follow student learning journeys from enrollment to job placement. So every milestone here is very critical for student success and we measure outcomes and metrics at each point and build a collection of evidence.”
Keith elaborates on the laboratory studies that he and others are conducting:
“We’re doing studies where I’m trying to figure out what users are capable of after one session in the headset — that is, what can they do if given real tools and objects? Transfr has a laboratory space where we bring people in, train them in the simulations, then put them in front of a workbench with tools and say ‘Can you show me now what you learned?’ We measure very precisely the amount of learning that’s happened because we’re observing and scoring the performance of individual steps within the process.”
According to Keith, having that kind of precise information is extremely valuable — both for reporting back to customers and other stakeholders, but also as a driving force for product improvements.
Better analytics, better products, better future
Every team here at Transfr is committed to helping build a better world for ourselves and the generations of learners we seek to serve. Yun Jin, Keith, and the company are in a unique position to understand both the product and our users in more detail and dig into which aspects of Transfr are delivering the most impact to users and where we can keep improving.
Of course, providing prospective customers and current clients with deep insights into the life-changing potential of VR learning is also important. Humans crave knowledge and understanding, so it’s only natural that once we build something like Transfr, we’d want to do all we could to understand how it works in the hands of actual humans! The time, energy, and resources that Transfr has put into our research team indicate just how important this understanding is to us here and the lengths we’re willing to go to truly help train the future of vital industries.
“Research has shown that engagement is highly correlated with student school achievement,” Yun Jin says. “So knowing that our products also provide high engagement for our students is a very good confirmation. We want to help students — including those with disabilities — to explore opportunities and have fun, engaging, exciting learning experiences in a safe, bias-free environment.”
About Keith and Yun Jin
Dr. Keith Lyle, Director of Learning Sciences at Transfr, has been studying learning and memory for 25 years. He is the author of nearly 50 scientific papers and has received multiple research grants from the National Science Foundation. His many hobbies include going to the movies, running, reading comic books, and practicing the peaceful martial art of Aikido.
Yun Jin Rho, Ph.D., Director of Data Science at Transfr, has been in the EdTech world since 2010. She is an inventor who holds four patents regarding learners’ skill estimation and predictive recommendation in adaptive learning. When she’s not helping build the future of VR training, she enjoys playing piano and listening to her son singing.
Listen to the full episode!
Want to delve into VR efficacy studies?
