From Myths to Principles Part 5: Myth: Learners Learn More

From Myths to Principles: Navigating Instructional Design in Immersive Environments
Part 5 Myth: Learners learn more

The
foundation of all learning, or child’s first book, by which a child
will learn more in one month than by many others in twelve, Author Unknown, Date 1800. Source: Compositor, University of Birmingham

Myth: Learners learn more in immersive experiences

This myth shrouds itself within a cloak of research. Citations will state that learning in immersive experiences is somehow greater when pitted against an implied traditional learning approach. The claim could appear as retention, but it is related to how well the learning was accomplished when measured up against learning objectives or a final goal.

When referring to the PricewaterhouseCoopers (PwC) report results mentioned earlier in this article series, Scott Likens claimed, “We found the realism and performance feedback in virtual reality simulations helped people learn faster and retain more information around soft skills,” (Zielinski, 2021, para. 9). He claimed they retained more information. This specific claim has been repeated in academic literature, which stated, “Studies have found that students who use XR training are more engaged with the content, display more confidence with the material, and retain more information than students who use traditional training methods.” (Rode, 2024, para 2.)

A close examination of the PwC report, however, reveals that the claim was unsupported within the report’s own data. When comparing information retention in VR versus an e-learning course, the authors “quickly discovered retention scores were inconclusive, as the delta between pre-and post-assessments in each modality was not significant” (Eckert & Mower, 2020, p. 44). Thus, there was no statistical difference between VR-based, e-learning, and traditional classroom learning. 

Claiming
something happened but your instrument didn’t pick it up is the knomes-did-it territory of cause-and-effect, dudes. Watch out.

 

The report therefore does not provide statistical evidence of more or greater learning within VR, yet it has been cited in academic publications (O’Dwyer, 2021, Etienne et al., 2022,  Jelki et al., 2022; Bäckelin, 2023; Etienne et al., 2023; Lønne et al,. 2023) and touted in media outlets (Murad, 2023; Schwantes, 2020). For the dubious claims, the report has been debunked as untrustworthy (Neelen & Kirschner, 2020). 

There are similar claims about greater learning retained from immersive experiences. Advocates for digital twin campus environments claimed that they “create greater retention of the information that is learned” (D’Agustino, 2022, para. 5) and “students’ grades go up” (Victory XR, 2024, Who We Are). 

 

While this is just one tiny sentence, keep in mind how much money VictoryXR makes from these claims.

 

In another example, the CEO of the Miami Children’s Health System touted that learners had 80% retention after one year after using VR, but traditional learners had 20% retention. A close look at the supporting documentation shows that the CEO actually said that the difference between VR learners and traditional learners can be the 80% to 20% difference. 

 


However, the CEO’s statement did not refer to any published results; it was opinion. The CEO explained their claim by saying that, “The level of understanding through VR is great because humans are primarily visual, and VR is a visual format” (Gaudiosi, 2021, para. 4). Cue learning styles!

Nonetheless, the quote of 80% retention has made its way into academic research (Iacono & Vercelli, 2019; Mathew & Pillai, 2020; Ternès, 2018). Some claims are extreme. One keynote speaker, Alvin Graylin, speaking as a leader at HTC (a VR headset maker) declared that as a result of VR use in the classroom, “Every single child has the potential to be a genius” (Educators in VR, 2020, 23:33). 


“Every single child has the potential to be a genius.” You just need to buy a (HTC) headset.

Justification for the greater learning or retention claims seems to be conjecture. Claims refer to how real an immersive experience feels to a learner. Returning to the PwC report, Likens credited “the realism and performance feedback…helped people..retain more information.” (Zielinski, 2021, para. 9). But long term studies measuring retention are hard to find within the body of academic literature. Some studies measure retention three weeks after the immersive experiences. Given that many of the examples of immersive experiences relate to hands-on disciplines like nursing or construction, things learned in immersive experiences would be needed on the job more than three weeks after the training.

Hamilton et al. (2020) stated that finding “learning outcomes, intervention characteristics, and assessment measures associated with immersive virtual reality has been sparse” (p. 1). Beck, Morgado, and O’Shea (2023) pointed out that details of methods are missing so that outcomes become questionable, “Very few literature reviews focus on the educational practices and strategies used in immersive learning environments. Thus, the problem is that we are evaluating outcomes without a comparable way to describe the educational approaches that led to those outcomes” (p. 2). Lawson et al. (2024) completed a systematic review of immersive experiences and found that research studies rarely isolate instructional methods and conditions when describing research studies and thus impact real world classroom decisions.

Meta analyses are starting to illuminate this area. Akgün and Atıcı (2022) observed that there was only a moderate effect on learner achievement after surveying 31 studies. Kaplan, Cruit, Endsley, Beers, Sawyer and Hancock found that “XR does not express a different outcome than training in a non-simulated, control environment. It is equally effective at enhancing performance” (2020, p. 1) Some researchers are starting to incorporate machine learning and artificial intelligence into this challenge in order to determine what the published records states about immersive experiences. Markowitz et al. (2024) recently surveyed 196,734 paper abstracts with this method.

Basically
this graph says that because the numbers are so small and so close to
zero, there is no discernible effect of VR on learning, regardless of
immersion (Kaplan et al., 2020).
 

From Myths to Principles Part 4 Myth: Learners learn faster

 

From Myths to Principles: Navigating Instructional Design in Immersive Environments
Part 4 Myth: Learners learn faster

Credit: Burst and Canva


Dispelling Myths


With some background established on boom and bust cycles in the hype for immersive experiences (Parts 1, 2, and 3), we need to dismiss the rather rampant myths about learning within immersive experiences. In the hype, learning advantages have been overstated and over simplified. Web pages post outrageous claims (and I’ll show you!). Keynote presentations banty incredible promises (yup, it was recorded). This series addresses the four primary myths about learning within immersive experiences: that is, that it is faster, greater, active, and induces empathy.

In this Part 4, I’ll address the “learners learn faster in immersive experiences” myth. For those of you that follow my writing, you’ll know that this is Round 3 of me taking on this myth. My argument has not changed; remember this article series is an update, but not every point needs updating. However, I continue to communicate about this because the “VR learning is faster” myth continues to circulate– mostly in the reference to “4 times faster” and the PwC report. So, TLDR, the VR experience was designed to be 29 minutes long. That’s it. No longer. The classroom equivalent in content experience was designed to be 2 hours long. That’s it. 29 minutes is ¼ of 120 minutes. Someone inverted ¼ to 4x (which is factually true) and PwC who appears to have had a cozy contract with Oculus/Meta at the time, went out to trumpet the ‘four times’ from the rooftops. But students do NOT learn faster. They experienced a learning event that was designed to be faster. Had the learners spent 120 minutes in the headset, someone would have probably greedily snatched the headset off their heads and told them that they overstayed their welcome (and wondered what they were doing for the extra 91 minutes).


Myth: Learners learn faster in immersive experiences


The first myth asserts that learners learn faster with immersive experiences. Particularly, the phrase “four times faster” has taken root in the publications and in public discourse. A google search on the phrase “VR is 4 times faster” returns a plethora of results repeating the myth.

4x in the wild. And it’s not hard to catch, yo.

The source of this phrase is suggested to be one non-peer reviewed industry report by PricewaterhouseCoopers. Within the report, VR-based learning was “4x faster than classroom training on average” (Eckert & Mower, 2020, p. 8). The results of this report were then repeated in academic literature.

Pie graph showing classroom training took 2 hours, e-learning training took 45 minutes, and VR training took 29 minutes. Text: We were able to train employees up to four times faster in VR than in the classroom and 1.5 times faster than e-learn.
Do not make pie graphs that do not add up to one whole thing.


Referring to the same report, D. Clark (an educational researcher not known for getting data wrong, but he did) wrote enthusiastically that “VR was x4 faster than classroom and x1.5 faster than e-learning” (2021, p. 190). Claims that learning is completed faster attempt to represent immersive experiences as a more efficient learning method, i.e., less time to learn equals learning faster.

Tracking down how many academic papers have cited the PwC report is difficult. I’ve seen numbers as little as 4 citations and much higher numbers if I start flexing my search. Part of the problem is that folks have not cited the report (even though it calls itself a study) correctly. Some credit PwC, a few find the Eckert and Mower authors, but in general the hand wave approach to referring to the 4x data is very prevalent.


The cause of this supposed faster learning was attributed to how a VR headset isolates the learner’s perception, so that the learner is focused on the learning task at hand. In other words, less distraction equals more focus. In the PricewaterhouseCoopers report, Likens seemed to hypothesize that, “A lot of courses that normally take an hour could be completed in 20 minutes through VR because people are so immersed in scenarios, there are fewer distractions and the learning is very concentrated” (Zielinski, 2021, para. 10).

To be clear, in the PwC case, classroom learning which covered the same content was designed to take two hours to complete. The immersive experience was designed to take 29 minutes. Given that 29 minutes is approximately one-quarter of two hours, PwC inverted the time ratio and touted the line that the immersive learning was four times faster. The problem is that it is not true that learning in the immersive experience was faster. The VR-based learning took less time because it was designed to be a 29-minute one-on-one learning experience designed for a shorter total time duration. When compared to classroom learning, it is already known that one-on-one personalized learning is generally faster; it moves at the speed of the learner, not at the speed of the class. Perhaps, this is how myths begin. A kernel of truth gets extended to something with no context. Lack of context is a noted and rising problem in educational research (Williamson, 2024).

Learning faster can be confused with greater efficiency. Efficiency could have a wide range of meanings beyond just taking less time. It could also mean wiser use of resources or less teaching burden on the instructor. Another example of the loose wordplay is on an industry webpage that displayed that VR training was 50% faster than a traditional in-person medical simulation. Not stated in the distilled summary of that study is that learners scored worse in the VR training than the traditional in-person medical simulation (Katz et al., 2020).


If faster equals worse performance, this might not be the efficiency that educators are looking for.

The myth that learning happens faster continues when educators fail to acknowledge that a different instructional method was being used. When supporting using virtual reality for chemistry studies, Muhsinah Morris, a chemistry professor and metaverse program director at Morehouse said “You can’t see molecules, but in my virtual reality classroom where I taught advanced inorganic chemistry, you can. You can actually build three-dimensional representations of molecules … The learning tends to happen faster. They go on to the real situation faster.” (D’Agostino, 2022, para. 5).

Side point: Learning Chemistry in Three Dimensions


Since this is my publishing space, I am going to spend some time on Mushinah Morris’ instructional and learning point here. Again, I was involved in the online teaching of chemistry for 14 years and my research speciality was science in VR, so I’ve got thoughts. If you would like to see her talk on video on this, she’s here in this video published by VictoryXR.

She is correct that molecules cannot currently be seen in everyday life. It makes chemistry, as a field, a more abstract or conceptual field along with physics when compared to the “you can see it before you” fields of biology or earth science. Teaching that something unseen exists and engages in reactions has always been the uphill battle of chemistry teachers. So she’s describing an accurate problem.

There is a tiny fly in the ointment, however, in that not many students at the college level fall into a chemistry course completely unfamiliar with chemistry at all. So learners in college chemistry probably were exposed to atoms in some other prior learning experience, be it high school, a museum, or a summer-camp like situation. So do her students need to learn atoms from the very beginning? I somewhat doubt that. 


HCHE Advanced Inorganic Chemistry


To massively further complicate her argument, she names and shows her chemistry course “Advanced Inorganic Chemistry”. That’s HCHE 421 at Morehouse University, which in 2021 had a prerequisite:

HCHE 322 Elementary Physical Chemistry, which itself has 3 prerequisites:

CHE 321/321L, Elementary Physical Chemistry and Lab (which has 4 prereqs: CHE 232, PHY 154, and MTH 161 and 162)

PHY 253 Electricity & Magnetism, which has 2 prerequisites: PHY 154 (C or better) and MTH 162

MTH 271 Introduction to Linear Algebra, which has 1 prerequisite: MTH 161

You see where I’m going here. It’s highly doubtful that students arriving in an advanced chemistry class after what is years at college, whose content focus is actually math (that’s what inorganic focuses on) and not spatial abilities (which arguably organic chemistry DOES focus on) have a substantial problem with visualizing atoms and molecules to the point where it is disturbing their learning performance. And therefore VR could make a difference. No. Not buying it. 

Covered in the mentioned course’s lab. That’s math, yo.

Nonetheless, I’ve known very smart and exposed people have trouble visualizing atoms. So it IS remotely possible and let’s pretend she is articulating only the beginning of the trouble of understanding for a lay crowd…not the only problem. Said another way, she’s speaking about VR’s affordances overall, maybe not specifically for her students in her aforementioned class. For example, some chemical reactions are easy to understand (like cooking) and some are difficult to understand (like how hair coloring works or cell electrical potentials).

It is interesting that she said “The learning tends to happen faster.” It’s a couched statement, for sure, with the word “tends”. In science that cannot be pinned down. So she gave herself an out. But what was she describing? At this point, we have to think about the instruction of chemistry.

How To Show Atoms and Molecules

Within the history of chemistry itself is the continuing saga of how will atoms be depicted? As in, how do you draw them? How are they really? And how does a teacher relay that ‘realness’ to the learners– and why? 


So we’ve had our:


Atoms are indivisible tiny units, folks. Thank you to the Greeks! There are no pictures from that time.


We’ve had Bohr’s heliocentric-like model folks wherein the atom looks like a solar system or set of concentric rings. To be fair, the heliocentric model really does help explain things like electron energy levels.

We’ve had our Thomson plum pudding folks– which never translated from its culture. Which is probably a shame. I like plums.


We’ve had our ‘cloud model’ folks – which are like the postmodern philosophers of chemistry. Truth for me, truth for you, we all get a truth, which isn’t true. But they told us that electrons cannot be pinned down and measured, they could be anywhere at any time but when we set about measuring them, that’s when they run away from us. Yes, I’m nodding to Heisenberg here. And wave/particle theory.

Cloud model of what an atom looks like.


After the heliocentric model, however, depictions of atoms needed to be displayed as three-dimensional, not just as two-dimensional on flat paper. By far, I’ve only selected some of the atomic model theories here. If you want to know more, study chemistry! It’s not hard.

But, now, going against Mushinah Morris’ arguments now, educators HAVE been working on that educational problem for years (with success, mind you).


First of all, delightful molecular (and atom) kits exist with physical manipulatives. Yeah, they look like tinker toys. I love them. They are good for at least 30 minutes of instruction, maybe more. They are usually plastic (boo, although there is nothing stopping them from being made of wood) and the kits would have to be purchased, stored, and de-germed from time to time. So they have their minor downsides.


Second of all, 2D screens can show 3D objects…that’s entirely possible.


Third, programming VR to follow mathematical principles – like, voila, chemistry DOES!– is actually not that hard. The first uses of VR in education that I know of were in the “physical”—that is mathematical sciences, physics and chemistry. Let’s face it. A computer understands 9.8 meters per second per second MUCH easier than a person does. (<- that’s one gravitational force).

And get this, purchasing a simulation to teach atoms is so drop-dead cheap that it’s actually free by now. I have recommended those simulations for courses before and seen learning scores do quite well, thank you. 

Looks pretty 3D to me



I seriously bet that if I had been able to place that counter proposal before her administrators, I’d win the budget proposal. Ha! Bonus points that I could prove that my students would score equally to her VR students on the final exam.

So in all, did she make a good point here? I’d say no but that’s because I recognize the instructional problem and I realize that the problem can be solved in a much cheaper and equally as efficient way. Also, she showed no data that “the learners learned faster”.

Side point to the side point: Mushinah Morris on YouTube is highly associated with VictoryXR, the vendor that she is using when referring to her VR-for-education accomplishments. Close association with XR vendors makes for suspicious conclusions. I’m not picking on Mushinah Morris unfairly. She’s gone on the record multiple times for her claims. I could easily pick (and will in the future) other education influencers that are selling the VR-for-education snake oil.

Back to my article

Further, there is at least one study (so far!) that refutes this focusing-causes-faster-learning claim. Makransky, Terkildsen, et al. (2019) found that immersive metaverse environments could be sensory overload for learners and therefore decrease the learner’s focus. On the whole, claims for increased speed can often be attributed to more efficient instructional methods. Immersive experiences can allow for the utilization of comparatively faster instructional methods.

The author finds this myth, that immersive experiences cause learners to learn faster, false. 

Seeking Integrity In VR Educational Research 2: PwC VR for Soft Skills

Decorative image of a cloaked woman going through paper archives

 Credit: Me and Midjourney

My first article in this series garnered so much attention! But many folks tried to pass me Mirjam Neelen & Paul A. Kirschner’s Truth or Truthiness? Analysing a VR Study Using Gorard’s Sieve article on the PwC report entitled “The effectiveness of virtual reality soft skills training in the enterprise: a study” and all of its associated webpages like this one. I was like, I know! Mirjam & Paul wrote their article 2020 and I wrote about it in 2021. What’s cool is that separately, we both came to the same conclusions. That’s a good sign for our conclusions!

Short version: we both cast strong doubt on any conclusions.

Still, I realize the world does not revolve around me (sigh!). Some folks might have missed my long stream-of-consciousnesses article about the PwC report. I decided that the second article in this series should be an abbreviated and updated critique. Bear in mind that to reach the LinkedIn audience, I have to leave much nuance by the side of the road. If you have questions, just ask!

As Mario says “Here we go!”

What is Said About The Report

This infographic summarizes the dominant conclusions:

  • 275% more confident to act on what they learned after training
  • 4x faster than classroom training on average
  • 4x more focused than e-learners
  • 3.75x more emotionally connected to the content than classroom learners.

nfographic: 275% more confident, 4x faster, 4x more focused and 3.75x more emotionally connected to the content.

 

LinkedIn post that mentions 4 times twice and nearly four times once.

Capture of how the PwC report is being talked about on LinkedIn.
 
A few more quotes, thanks to Google and a search on “VR 4x faster.” What seems to be a pattern about all of these results?

 

 

 

 

What do these Google results have in common?

They are all companies that sell some sort of VR product or service.

Because I was curious, I checked out that vrowl dot io link (“Virtual Reality training is not effective”) just to see if it was presenting an alternate opinion. It’s a strawman argument; it puts up “not really real” protests against VR for learning and then explains them away. I’m telling ya, Beware the VR Strawman.

What the Report Says

Eckert, D., & Mower, A. (2020). The effectiveness of virtual reality soft skills training in the enterprise: a study. https://www.pwc.com/us/en/services/consulting/technology/emerging-technology/assets/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf

Let’s ask Google Scholar what it thinks. It’s coming up with 11 cites. That’s not much at all. But as I showed above, the money shot is on the Internet, not in academic articles.

Truly, the 4x faster learning quote is the runaway train of this report.