At the start of every school year, I get the same feeling: the hallways fill back up, the lesson plans get dusted off, and somewhere a school board is asking a vendor "does this thing actually work?" This year, that question has an answer key.
Last month, the U.S. Department of Education issued its long-awaited Dear Colleague Letter on edtech — the first real federal signal in years on how schools should evaluate the tools in their classrooms. It didn't come with new regulations. It came with something more useful: a standard. As Assistant Secretary Kirsten Baesler put it, "Responsible design is the floor. Not only do we need to know if the tool works, but if it works; why does it work, for whom does it work, and under what conditions."
I've read a lot of edtech guidance over the years. This is the first one I've wanted to frame.
The bar just moved from "logged in" to "learned something"
For too long, ed-tech success has been measured in logins, seat time, and screen minutes — proxies that tell you a product was used, not that it worked. The Department's letter calls that out directly, pushing districts to evaluate tools on outcomes and instructional value, and to weight procurement toward products with real evidence behind them — randomized controlled trials where possible, but the opportunity for so much more.
This is a welcome shift, because it's the exact question our Learning Intelligence Team has spent years answering for VR-based training: not "did people use it," but "did it teach them the skill, and did they retain what they learned?"
So in the spirit of showing our work, here's some of what we've found.
VR clears the bar the Department is setting
In controlled lab studies, we've measured what actually happens when someone learns a hands-on skill in VR versus a more traditional method like video or written instruction:
- Learning gains that hold up under observation. Participants who trained in VR improved their performance scores by 36.8%, compared to 24.5% for those who trained on slides — and we didn't just take their word for it. We watched them perform the task afterward, in a controlled real-world setting.
- Gains that don't depend on who you are. One line in the guidance stopped me cold: edtech "increases learning opportunities for rural students, students with disabilities, and home-bound students," and shouldn't be evaluated by screen time alone. Our data backs up why access matters — we found no significant difference in learning gains across gender, income level, or age. Learners went from a 25% pre-test average to 62% post-simulation, consistently. Good training should work for everybody in the room, not just the students it was quietly designed for.
- Real engagement, not just "engagement." The letter draws a sharp line between recreational screen time and instructional value. That distinction is basically our whole thesis. Learners rated VR simulations 86% for enjoyment and 86% for attention, versus 57% and 60% for non-VR methods. Something key: learners also reported far less distraction with VR than with a traditional lecture, across every module we tested, from circuits to lifting devices to technical drawings.
- Time well spent, not just time spent. Here's a stat I cannot repeat enough: when we measured actual VR session length against how long participants thought they'd been in the headset, perceived time ran up to 25% shorter than actual time — 22 perceived minutes for every 29 actual ones. Learners weren't clock-watching - they were doing.
- It beats the tools that are eating everyone's attention right now. Compared to studying AI-generated explanations and TikTok videos, VR training improved real-world performance by 68%. In a moment when districts are exploring how AI-generated content and video fit into instruction, that's a useful data point: explanation and practice aren't interchangeable, and the gap between them shows up in real-world performance.
The "why" and the "for whom"
The Department didn't just ask whether tools work. It asked why, and for whom, and under what conditions, which is the tougher question but one that deserves to be asked. Our answer, in short: VR works because it lets people practice before they're evaluated, in an environment that's honest about mistakes and consistent about feedback. It works across demographics because the simulation doesn't know or care about a learner's background, it just responds to what they do. And it works under the condition that matters most to a school procurement officer: when someone checks the results afterward.
What this means walking into the new school year
I don't think every ed-tech company needs to be nervous about this guidance. I think the ones who've already been doing the unglamorous work of measuring outcomes, not just usage, should be glad the quiet part has been said out loud. Evidence isn't a compliance checkbox. It's the difference between a tool that keeps kids occupied and one that gets them somewhere.
So here's my back-to-school ask, both to my own team and to the districts and training providers we work with: keep asking the harder question. Not "are they logging in," but "are they learning, and can we prove it." That's the assignment this year and it's the one we've been doing our homework for.
Transfr's Learning Intelligence Team studies of VR training efficacy referenced above are available at transfrinc.com/efficacy-studies.
