Does ChatGPT Enhance Student Learning? Not in 2026, it doesn’t.
“ChatGPT enhances academic performance. ChatGPT boosts affective motivational states. ChatGPT improves higher-order thinking propensities. ChatGPT reduces mental effort.” Source:…
Category: Poor
The predicted downfall of ENGAGE XR
Photo by Megan Lee on Unsplash Given that it looks like EDUMetaverse is headed to the turf, I might as…
XR for education propaganda – EDUMetaverse
propaganda: 1 – ideas, facts, or allegations spread deliberately to further one’s cause or to damage an opposing cause also…
When Tech Platforms Donate The Resources
Jeremy Bailenson As much…
Seeking Integrity in VR Educational Research 3: It keeps on happening
It keeps on happening If I were you, by now, I’d be asking, “Heather, why are you doing this?…
Seeking Integrity In VR Educational Research 2: PwC VR for Soft Skills
Credit: Me and Midjourney
My first article in this series garnered so much attention! But many folks tried to pass me Mirjam Neelen & Paul A. Kirschner’s Truth or Truthiness? Analysing a VR Study Using Gorard’s Sieve article on the PwC report entitled “The effectiveness of virtual reality soft skills training in the enterprise: a study” and all of its associated webpages like this one. I was like, I know! Mirjam & Paul wrote their article 2020 and I wrote about it in 2021. What’s cool is that separately, we both came to the same conclusions. That’s a good sign for our conclusions!
Short version: we both cast strong doubt on any conclusions.
Still, I realize the world does not revolve around me (sigh!). Some folks might have missed my long stream-of-consciousnesses article about the PwC report. I decided that the second article in this series should be an abbreviated and updated critique. Bear in mind that to reach the LinkedIn audience, I have to leave much nuance by the side of the road. If you have questions, just ask!
As Mario says “Here we go!”
What is Said About The Report
This infographic summarizes the dominant conclusions:
- 275% more confident to act on what they learned after training
- 4x faster than classroom training on average
- 4x more focused than e-learners
- 3.75x more emotionally connected to the content than classroom learners.
Capture of how the PwC report is being talked about on LinkedIn.
A few more quotes, thanks to Google and a search on “VR 4x faster.” What seems to be a pattern about all of these results?
What do these Google results have in common?
They are all companies that sell some sort of VR product or service.
Because I was curious, I checked out that vrowl dot io link (“Virtual Reality training is not effective”) just to see if it was presenting an alternate opinion. It’s a strawman argument; it puts up “not really real” protests against VR for learning and then explains them away. I’m telling ya, Beware the VR Strawman.
What the Report Says
Eckert, D., & Mower, A. (2020). The effectiveness of virtual reality soft skills training in the enterprise: a study. https://www.pwc.com/us/en/services/consulting/technology/emerging-technology/assets/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf
Let’s ask Google Scholar what it thinks. It’s coming up with 11 cites. That’s not much at all. But as I showed above, the money shot is on the Internet, not in academic articles.
Truly, the 4x faster learning quote is the runaway train of this report.
The Salem Witches of ID OR Cancel culture has arrived in Instructional Design
Photo by Erica Marsland Huynh on Unsplash
What do the Salem witch trials and woke cancel culture have in common?
Both established rogue thoughts as truth.
I’m sorry to report that over the past few months, cancel culture has arrived in Instructional Design.
Personally, I’ve seen “Andragogy” and “Brain-Based Learning” attacked and discarded on public LinkedIn posts, threads, and some blog posts. I’ll be collecting them here below as I find them. However, if there get to be too many (and already collecting these was depressing and exhausting. In one case there were 30 replies and that was not even to me!) I’ll stop collecting.
It is as if a newer ID hears of brain-based learning, says to
themselves “huh, where else is learning supposed to happen?” and then
calls brain-based learning stupid because of the name.
I’ve
tried to point out that what’s happening is that less educated
Instructional Designers are approaching these concepts as words only or
with very little in-depth research and are tossing out the concept
entirely.
In
the case of Andragogy, I tried defending it. It’s an established
section of education with a depth of history of more than 50 years (in
popular Education studies, longer in lexicon). Attacking it, to me, is
the equivalent of attacking Black History. Why would you do this? It
makes no sense. The arguments against andragogy always seem to equate little children with adults.
For example:
- According to andragogy, adults want to know why.
- My child asks why.
- When doing so, my child behaves as an adult.
- My child is not an adult.
- Therefore, andragogy does not exist.
Rinse & repeat with a lot of cognitive elements (my child can do this, my child can do that…) Always exceptions. Piaget gets dragged into this (he does below). Perhaps then begin the “Well, if one part of false, then all parts are false” arguments…which themselves are logical fallacies. Duh.
It’s tough out there. Note in this first example, the author is the post is ALSO the author of the article hosting place called The Learning Scientists— which is a point that I make; that the OP is putting on a aura of authority that is, perhaps, inappropriately authoritative to the audience. Said another way, readers might not understand that the writing, all inclusive here, was opinion.
Andragogy
Academics Getting Paid To Write
 |
| Photo by 愚木混株 cdd20 on Unsplash |
I once was in the position to see the editing of an international research position paper; how the sausage is made, so to speak. I was able to see contributors, editing, and writing styles first hand by academics from my industry.
What I saw was shocking.
Writer #1
I saw one writer who is a famous name in my industry who could not write their way out of a wet paper sack. Literally one paragraph that was meant to contain a definition of “immersion” contained five sentences that were the same sentence reworded. It was exactly like watching a Freshman college student open, work on a paper by writing one floppy sentence, then hitting save, put the paper away for a day, then re-open it, start again in the same paragraph and forget that they should be adding thought and simply re-work the same thought they had the day before, and then think that they were profound. The result is that they sounded like a bullshitter. Their work needed major rewriting; every sentence.
This is a doctoral holder who is director of a major research/production house with the US. [EDIT: Update from December 2025. This writer’s university is collapsing.]
Writer #2
Second, a writer who is famous — like hits the podcasts weekly for their opinions in my industry– contributed a page and a half of writing and was paid US$5,000. (Actually all three examples I give here were paid $5,000 each for their writing). It didn’t need much editing but…hey…could you sleep at night knowing you gave that kind of contribution? The final paper was 75 pages.
$5,000 for 1.5 pages of writing?
Nice work, if you can get it.
Writer #3
The last writer actually gave good writing and a decent amount of it. Unfortunately, this person does not get off completely free from problems.
I personally tech supported the person as they were supposed to present in an admittedly difficult XR platform. They didn’t attend a rehearsal. The time of the event, their sound didn’t work. I brought them outside of the auditorium to troubleshoot. But as I went through the standard list of fixes (which all centered on the user and their Windows settings), I got a healthy heaping of sighs and “I should be inside presenting” statements. Eventually, the writer gave up on me and the tech and decided to go in on cell phone patched in.
The writer was still paid $5,000. But they had an attitude that would not fit through the door sideways.
All doctoral holders, all famous names in the industry. All unpleasant to work with, to put it mildly.
Disclosure: I contributed to the early stage of this same writing project. However, there is no proof that any of my contributions, or indeed much of any of the contributions of the 54 unpaid, early stage writers showed up in the final project other than just in name. The early contributions were supposed to be read, condensed, and included in the final version. There is absolutely no proof that actually happened.
Even at one point, I had heard an incident that a quote attributed to an
early writer had never happened; they didn’t write/say that. That person strongly objected to the quote for obvious ethical reasons. The editors removed the mistaken attribution.
It seems much more likely, from what I witnessed, that the three writers above plus the editors simply restarted a new document to their own preferred liking.
The further editing was a mess as editors worked with different writing programs
and had distinctly different approaches to what was acceptable academic
writing. In the end, the main editor was so disgusted by the writing
that they refused to have the document qualified as academic writing and
it was spun as a “market report” (interesting, given that it was
written entirely by academics) instead. It didn’t really work.
The document hit the presses
more than 6 months late to little acclaim or notice.
Everything’s Coming Up Metaverse
Photo by Sophie Grieve-Williams on Unsplash
December 1, 2021:
Hey, we’ll be having a Christmas party on December 18. You’re invited.
December 14, 2021:
Actual party invitation
Now it’s a St. Nick’s Frosty Metaverse Party. I wonder if they served Metaverse cookies? 😣
January 24, 2022:
Email:
Recently the concept of Education Metaverse is
sweeping the world. Lots of people are developing products and
researching on this topic. I have just co-published a concept paper on
Education Metaverse. If we can add Metaverse to our site, that would make it
more timely and trendy. Immersive Learning is the pedagogy underlying Education Metaverse. We just added Metaverse for Learning as a theme to (Anonymized Conference) 2022.
That’s an academic researcher advocating that an academic group should use a trendy word.
February 17, 2022:
Metaverse Conference (not it’s real name)
Opening day (ahem, one day, but still). Opening seconds after 30 second countdown.
Camera goes on. Live microphone.
First word broadcast: “Shit.”
Presenter leans back in chair, takes earphones off, and says to someone back off camera, “What?”
Sighs quickly. Strips off headphones, jumps up out of chair, and runs out of room and off camera.
Dead air.
This is 9 a.m. local and 11 a.m. Eastern US time zone on a Thursday in mid-February. I’m just reminding you of that because I’m stating the point, as a conference organizer myself, that these folks were right in the sweet spot of being able to handle an emergency no-show. They had a 30 second jazzy countdown graphic, with music! They could have had a “green room” concept. They could have published their one-day schedule earlier before the conference so that each presenter had plenty of time to know exactly when they were on schedule. Their admin staff could have expressed the presentations times to the presenters in their own home/local time zone (that’s only polite). There are many support steps to prevent what happened—it’s not like this presenter was woken up at 3 a.m. to suddenly go on camera. That’s what I’m saying…
Presenter comes back into room.
With no apologizes starts talking about the difference between Mountain time and a new made-up phrase ‘Metaverse time’ (which turns out to be an old allusion, I think to Second Life time which was Pacific, where Second Life was headquartered, because the other co-presenter was an hour off…late, which can only be construed as Pacific time with reference to Mountain time.)
The next 30-40 minutes are a bit of a jumble as the original presenter patch-presents a ~15 minute introduction and then the intended host shows up and also presents at a first compacted and then expanded set of concepts (which is recycled from other conferences). The intended host, to my chagrin because I like him, doesn’t even know who is coming next on the schedule and when.
The rest of the day has only one session that I found grounded in research and practice. The other sessions wildly pushed the “technology” button for the Metaverse. ‘Want to get started in the Metaverse? Build something!!’ I was actually surprised how much the host college pushed their own technology classes, but then again, what are you going to get for free…
It appears that many attendees were newbies to Metaverse because they kept saying that the concepts were “new” to them.
A definite low-lite was when a fight broke out in the chat about the WHO. Yes, that WHO, (not the band or the Doctor.) I took a break at that point. It’s clear that politics is coming into the Metaverse with us.
There was one high-lite but I can’t say what it is without saying which conference it, therefore, was so I’ll let that go.
But over and over, institutions wonder why women and minorities do not feel welcome.
Revisit that first few seconds of broadcast. You had your chance, after weeks of social media buzz, to make an impact.
You did.
You advertised your quality.
In one word.
Analysis of The Total Economic Impact™ Of Mixed Reality Using Microsoft HoloLens 2
Analysis of The Total Economic Impact™ Of Mixed Reality Using Microsoft HoloLens 2
(Image: Crowds gathering outside New York Stock Exchange Source: https://commons.wikimedia.org/wiki/File:Crowds_gathering_outside_New_York_Stock_Exchange.jpg)
This report, The Total Economic Impact™ Of Mixed RealityUsing Microsoft HoloLens 2, dated (November 2021) came through my LinkedIn feed in this post:
What most caught my eye was the fourth bullet:
Mixed reality increased
training efficiency by 60%, saving $1,440 per trainee while improving
knowledge acquisition and retention.
I posted my response:
“With
respect, a company that sells a product that then claims the product
increases training efficiency with these kinds of numbers is to be
doubted.
In the case of the 60% increase, my eyes land on the
table on page 21 and the source of the 60% increase in task efficiency
data is “Interview data”, not quantified data. That means the
interviewees that Microsoft picked provided verbal estimates to the
interviewers. While the report takes great pains to describe how the
results were calculated– and I respect that– this is a “commissioned
report” (page 1)– that was paid for by the very company selling the
product.
Long term, results like these do not hold up.”
To which, an Associate Fellow at Lockheed Martin replied:
While the numbers seem outlandish they are very real based on our findings.
We
use mixed reality to build spacecraft and see well over these numbers
(~93% reduction) in touch labor. This was analyzed across 10
manufacturing sites and numerous spacecraft manufacturing programs.
Technicians consistently finish 8 hours worth of traditional work in 45 minutes, and other companies are seeing similar results.
For
training we see an 85% reduction in training time. Mixed reality also
offers greater comprehension than traditional methods.
While the numbers seem incredible, they have been validated with more than four years of shop floor implementation.
If you would like to chat details, just let me know. I would be happy to provide insight to our findings.
I did not reply further at the time to this post because I was not actually arguing time-based measurements. I was arguing improving knowledge acquisition and retention via task efficiency– admittedly their own squirrely wording. Said another way, claims that users learned more.
This blog post is, in part, my response.