ChatGPT is one of the most frequently used services on the internet (per similarweb.com). To compare learning gains from ChatGPT-generated content to VR sims, Transfr conducted a study with 24 community members in the Louisville, KY area. Each participant learned two construction and manufacturing tasks (multimeter voltage reading and fire extinguisher usage). Participants were randomly selected to learn one task via VR sim and the other by studying a webpage* with ChatGPT-generated content. Participants had little to no prior experience performing either task. Participants returned 1-7 days after training to attempt the tasks using real tools and materials. Playing a sim led to significantly better real-world performance than studying a webpage**. After controlling for individual differences, the likelihood that a person scored higher after sim play than after webpage study was 74.5%. Participants also reported significantly more mind wandering when studying webpages than when playing sims***.
*Webpage study time was self-paced. The fire extinguisher webpage also included an embedded YouTube video.
** t(23) = 3.29, p = .003, d = 0.67
*** t(22) = 3.33, p = .003, d = 0.69 — due to a data collection error, rating data from one participant is missing
Our findings suggest that experiencing tasks in a well-designed VR simulation is a better way to learn than trying to read up about them online. Learning scientists have known for a long time that most students don’t get much out of just reading and looking at diagrams. Fortunately, VR sims appear to be considerably more effective.
Participants reported extremely high engagement with VR as well as enjoyment, enthusiasm for future VR learning, and willingness to replay the specific simulations used in the test. Participants enjoyed playing sims significantly more than studying webpages*, felt significantly more engaged when playing sims than studying webpages**, and were significantly more enthusiastic about using sims for future learning versus studying webpages***. A much larger percentage of participants were willing to replay a sim versus restudy a webpage with 83% of participants saying they would replay the sim compared to only 13% of participants who would study the webpage again.
*t(22) = 6.56, p < .001, d = 1.37
** t(22) = 8.23, p < .001, d = 1.72
*** t(22) = 6.02, p < .001, d = 1.26
**** “Maybe” responses have been omitted, therefore percentages do not sum to 100
