Seeking Integrity In VR Educational Research 2: PwC VR for Soft Skills

Decorative image of a cloaked woman going through paper archives

 Credit: Me and Midjourney

My first article in this series garnered so much attention! But many folks tried to pass me Mirjam Neelen & Paul A. Kirschner’s Truth or Truthiness? Analysing a VR Study Using Gorard’s Sieve article on the PwC report entitled “The effectiveness of virtual reality soft skills training in the enterprise: a study” and all of its associated webpages like this one. I was like, I know! Mirjam & Paul wrote their article 2020 and I wrote about it in 2021. What’s cool is that separately, we both came to the same conclusions. That’s a good sign for our conclusions!

Short version: we both cast strong doubt on any conclusions.

Still, I realize the world does not revolve around me (sigh!). Some folks might have missed my long stream-of-consciousnesses article about the PwC report. I decided that the second article in this series should be an abbreviated and updated critique. Bear in mind that to reach the LinkedIn audience, I have to leave much nuance by the side of the road. If you have questions, just ask!

As Mario says “Here we go!”

What is Said About The Report

This infographic summarizes the dominant conclusions:

  • 275% more confident to act on what they learned after training
  • 4x faster than classroom training on average
  • 4x more focused than e-learners
  • 3.75x more emotionally connected to the content than classroom learners.

nfographic: 275% more confident, 4x faster, 4x more focused and 3.75x more emotionally connected to the content.

 

LinkedIn post that mentions 4 times twice and nearly four times once.

Capture of how the PwC report is being talked about on LinkedIn.
 
A few more quotes, thanks to Google and a search on “VR 4x faster.” What seems to be a pattern about all of these results?

 

 

 

 

What do these Google results have in common?

They are all companies that sell some sort of VR product or service.

Because I was curious, I checked out that vrowl dot io link (“Virtual Reality training is not effective”) just to see if it was presenting an alternate opinion. It’s a strawman argument; it puts up “not really real” protests against VR for learning and then explains them away. I’m telling ya, Beware the VR Strawman.

What the Report Says

Eckert, D., & Mower, A. (2020). The effectiveness of virtual reality soft skills training in the enterprise: a study. https://www.pwc.com/us/en/services/consulting/technology/emerging-technology/assets/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf

Let’s ask Google Scholar what it thinks. It’s coming up with 11 cites. That’s not much at all. But as I showed above, the money shot is on the Internet, not in academic articles.

Truly, the 4x faster learning quote is the runaway train of this report. 

Continue reading “Seeking Integrity In VR Educational Research 2: PwC VR for Soft Skills”…

The Search Wars

 

Sichuan Zhongshuge Store

No one remembers the Search Wars! Dang!

They occurred between 1997 – 2003 or so. It was a time when a dominant search engine, Google, had not yet risen to power. Multiple search engines vied for top spot. What was top spot, you say? It was “most pages indexed”.

How were the Wars fought? Each week, researchers would depth-check each search engine. Back then Yahoo was really the one folks thought would win, AskJeeves was in there, I think some Microsoft ones were in there.

Image of Ask Jeeves search engine logo.

 

Method:

They would search for something that does not exist. The results, therefore, would contain the entire pool of sites that the search engine HAD indexed and said, essentially, NOPE, not there, we don’t have that and we’ve checked everywhere.

How do you search for something that does not exist? Easy. Put your fingers over a keyboard and starting hitting keys.

Example:

fhdakfjklpoiovpoie

That ^ does not exist.

A return number on that from a search engine (yes, they used to return numbers, not today’s hopeless signal of ‘Next: 2, 3, 4,…27 million’) would tell you how many total web pages that engine had indexed.  It was tracked week to week.

Yes, Google eventually won the Search Wars.

But even now, there are pockets that Google just can’t get to.  See below.

Good hunting, Rebels.

Continue reading “The Search Wars”…

Manifesto: XR will not cause lasting improvement in education

 

Photo of clear glass sphere on a beach reflecting the sunny scene upside down. Keyword: Clarify.

Photo by Dan DeAlmeida on Unsplash

I’ve received some questions on my video and transcript posted here: https://heatheredodds.blogspot.com/2022/09/xr-will-not-cause-lasting-improvements.html

So I’ll add some clarifications:

1. There are weak points in my argument:

A. I argue that the learner is the still as-yet undiscovered cause of the flat lining of learning objective results media to media.  I have NO data to back that up. That is a supposition by me. I suspect the data will have to come from brain studies.

B. My argument that learners in previous generations were NOT dumb is a bit of low…err…high?…blow. Certainly, there were dumb learners in the past.

However, I do not buy the modernist argument that when technology gets “better”, learning gets better.  Nope. No. As I mentioned in the video, humans appear to have a learning speed limit. Said another way, the neural pathways of learning in a human brain are set. (Yup, I’m referring to brain-based learning theory here. You might know it as neuroscience.) Short of something like “Lawnmower Man” or a “Flowers for Algernon” royal technology/drug-induced fuck up, I don’t see humans getting smarter.

2. Let me be clear on my argument about results flat-lining and there being no “lasting improvement”. The “lasting improvement” that I’m mentioning are ONLY learning objectives. So said another way, if there was an exam covering X taught with media Y where students score Z right now….in 10 to 30 years, learners will still score Z even if XR is the media.  I’m sticking to apples to apples comparisons. I’m NOT talking about other things like XR affordances, which would introduce apples to kiwi to melon comparisons….which are not comparisons and are not fair.  

So I’m not talking about XR doing things like increasing access to resources due to manipulations of time, space, geography, physics, etc. Those things are affordances, the characteristics that belong or sort-of stick to a media form.

The conversation about affordances is fascinating and I’d love to have it! As a designer, knowing the positives and negatives about each media is my specialty! (See my XR platforms writing.) However, I’m also bound as designer to not force any decision about the “best” media upon a client. The clients decides what they will select, what they will pay for, what they will invest in long-term and thus the client accepts both the positive and negative consequences of their decision, their “opportunity cost”. So by default, I almost never like to say this is “the best” when it comes to an XR platform.

3. Timeline = I used smartphones as an example in the video but I’m really brief about it.  But it is in somewhat recent memory that smartphones went from a new technology to everyone having one.  How long did that take?  Hmm… lemme check:

First arguable smartphone: 1992.

2022: as shown in the video there are enough smartphones in the US for every adult to have one. Translation = the US market is saturated. Smartphones are ubiquitous. 

1992 to 2022. So that took 30 years.

I’m fine with adding in Moore’s Law here.  So the adoption of XR until the point of it being ubiquitous and saturated– how long will that take?

Hmm… I’m guessing but I’m more comfortable saying closer the 10 years from 2022 than 5 years.  That puts my guess at 2032.

Now now, you pro-XR folks out there! I heard your cry! 10 years!!  Don’t be sad.  Remember what is between HERE and THERE: a great big increase, an expansion, a bubble, GROWTH.  It will be a good 10 years.  (Imagine what the first 10 years was like for smartphone manufacturers Nokia and Apple, whoohoo!)

Continue reading “Manifesto: XR will not cause lasting improvement in education”…