Category: VR (page 2)

Seeking Integrity In VR Educational Research 2: PwC VR for Soft Skills

Decorative image of a cloaked woman going through paper archives

 Credit: Me and Midjourney

My first article in this series garnered so much attention! But many folks tried to pass me Mirjam Neelen & Paul A. Kirschner’s Truth or Truthiness? Analysing a VR Study Using Gorard’s Sieve article on the PwC report entitled “The effectiveness of virtual reality soft skills training in the enterprise: a study” and all of its associated webpages like this one. I was like, I know! Mirjam & Paul wrote their article 2020 and I wrote about it in 2021. What’s cool is that separately, we both came to the same conclusions. That’s a good sign for our conclusions!

Short version: we both cast strong doubt on any conclusions.

Still, I realize the world does not revolve around me (sigh!). Some folks might have missed my long stream-of-consciousnesses article about the PwC report. I decided that the second article in this series should be an abbreviated and updated critique. Bear in mind that to reach the LinkedIn audience, I have to leave much nuance by the side of the road. If you have questions, just ask!

As Mario says “Here we go!”

What is Said About The Report

This infographic summarizes the dominant conclusions:

  • 275% more confident to act on what they learned after training
  • 4x faster than classroom training on average
  • 4x more focused than e-learners
  • 3.75x more emotionally connected to the content than classroom learners.

nfographic: 275% more confident, 4x faster, 4x more focused and 3.75x more emotionally connected to the content.

 

LinkedIn post that mentions 4 times twice and nearly four times once.

Capture of how the PwC report is being talked about on LinkedIn.
 
A few more quotes, thanks to Google and a search on “VR 4x faster.” What seems to be a pattern about all of these results?

 

 

 

 

What do these Google results have in common?

They are all companies that sell some sort of VR product or service.

Because I was curious, I checked out that vrowl dot io link (“Virtual Reality training is not effective”) just to see if it was presenting an alternate opinion. It’s a strawman argument; it puts up “not really real” protests against VR for learning and then explains them away. I’m telling ya, Beware the VR Strawman.

What the Report Says

Eckert, D., & Mower, A. (2020). The effectiveness of virtual reality soft skills training in the enterprise: a study. https://www.pwc.com/us/en/services/consulting/technology/emerging-technology/assets/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf

Let’s ask Google Scholar what it thinks. It’s coming up with 11 cites. That’s not much at all. But as I showed above, the money shot is on the Internet, not in academic articles.

Truly, the 4x faster learning quote is the runaway train of this report. 

Continue reading “Seeking Integrity In VR Educational Research 2: PwC VR for Soft Skills”…

XR Will Not Cause Lasting Improvements In Education

 
 

This post accompanies my XR will not cause lasting improvement in education video and contains a few more details. I wrote this blog post first, then made and remade the video and I’ve come back to finish the blog post with the final script and my notes.

XR will not cause lasting improvement in education.

That’s an interesting statement to start a video

when I’m known for being pro-XR.

That’s right, I am pro-XR in education.

But I have expectations that learners will not perform higher.

With respect: Rephrased
from the Cambridge Handbook of Multimedia, (2005, pp. 7-9) and Cuban’s
1986 book: Teachers and machines: The classroom use of technology since
1920 (pp. 9-26) and Mayer, R. (2020). Multimedia Learning (3rd ed.).
Cambridge: Cambridge University Press. doi:10.1017/9781316941355.

Generally, educators are on the lookout for what causes learning and we want to encourage more of it. We realize that content is hard to learn and we want as many learners as possible to successfully learn it. This is given– a belief in the general positive well-being of the learning process, the educators and not least, the learners. It IS important to say that because somewhere along the way, one of the counter arguments against that fact that we don’t find learning gains is “the technology was poorly implemented” or “the leaders don’t care for change” and I wanted to cut both of those excuses off right at the beginning. Nope! Educators IN GENERAL are implementing the technology well and leadership IN GENERAL is pro-change.

Next we need to visit the scientific experimental model as it is the basis for the experimental models used in education. That means that we observe an effect, some data, some phenomena, and we ask “What caused this?”

Remember, we are looking for cause and effect.

This is the scientific experimental model.

Controlled variables – things hold them constant so that they don’t change.

Independent variable – what we purposely change to test cause and effect.

Dependent variable – what we measure as the result.

There are other models to gain information from; naturalistic…meaning anything outside of a lab

Or cultural ways of knowing. This could be indigenous or religious knowledge.

Regardless, the Experimental Model is one of our strongest logic systems and it comes through more times than not at finding cause and effect.

We can isolate variables down to determining the cause (a deductive reasoning approach, a la Sherlock Holmes), or we can simply start with as few variables as possible to find the cause.

This is the same experimental model as it appears in educational research.

We have our learners, we add a technology, and we measure the results.

And it’s not like we just started this research.

For the purposes of this video, I’ll go back just over 100 years and use the word technology to mean anything powered by electricity.

For example, Radio

And here are the results: no lasting improvement.

Projectors – no lasting improvement

Television – no lasting improvement

Computers – no lasting improvement

Internet – no lasting improvement

and in the future, cloud-based learning by robots or whatever.

But in all seriousness, this video is about XR, extended reality, cross-reality, mixed reality or whatever you want to call it.

Graphic of learners plus XR equals results.

Now RIGHT HERE, some will become upset. They say:

But this is different!

This is learning in 3D!

No, you don’t understand, this is a computer stuck to your face!

We need to implement it correctly and THEN we’ll see the results!

I have a study right here that shows it better when putting VR up against a textbook or a human teacher!

OK for that last one, I toss that right out as non-comparable methods, but that’s a topic for another day.

So let’s look at the results, shall we?

No improvement.

Now for those that are hearing me right now having a really hard time taking this in, I understand that this is not fitting into your schema. What you are feeling is bias. You want the results to be a certain way, and even when the results are not turning out the way you want them to, you want to reject all of the previous results as not predicting what will happen next. Remember that bias, in research, is a bad thing. We don’t want it. So I need to ask you to check your bias and leave it behind.

I’ll give you an example that should be in the recent memory of XR enthusiasts. I’ll use 2022 words to explain a 2022 real world example.

How many studies do we hear of right now that show a spectacular increase in learning with a smartphone (mobile)?

How many times do we hear from learners that they love learning on their smartphone? “Oh it’s so cool!” “Oh it’s the best!” Oh I love that I can learn from a computer in my pocket! Oh, I love that I can learn on this tiny screen!”
~ Oh I love that I’m
being forced to do my workplace learning on my own device (that I paid
for, pay for the internet subscription for, and pay the insurance on, to
say nothing of being tracked by my workplace VIA my own phone!

What’s that?

No one says this?

You’re right.

Why?

Said another way, smartphones are ubiquitous. Actually if you listen closely, there is a STRONG amount of conversation about how learning on the smartphone is boring, forced, poorly designed and/or at least equivalent to learning in the classroom—thanks to COVID and 2020.

So learning on a smartphone is ubiquitous. The learning results have flat-lined.

I’ve made my case that history predicts that XR will also flat-line after it has become ubiquitous.

But….why?

We still didn’t answer that.

I have 2 reasons. One I’ll share, the other, not yet.

Let’s go back and look at that experiment model again.

We said that every technological improvement has proved to produce zero overall learning gains. Learners are simply NOT DOING BETTER.

We can slip in and out all of these technologies and we keep getting goose egg results, nothing. But…look closely at the model. What other variables are there?

We said that technology was a variable and our proposed independent variable– we are purposely changing it).

The results are the dependent variable – they are the output, the effect, or the result of what we are changing and frustratingly, they are NOT CHANGING.

So what else is there?

Look. One more variable is present…

The learners!

Matching my technology examples: 1920s learners

1940s learners

1960s learners

1980s learners

Learners from the year 2000

2010 learners

I mean, everyone knows that 1920s learners were dumb, right? I mean…

Oh, you mean the time when Einstein discovered his E = mc(squared) hypothesis? We were dumb?

1940s? The start of the discovery of the polio vaccine? Saving thousands if not millions of future lives?

We were dumb then?

1960s? Early computers being built? Remember…going to the moon?

1980s? Well no comment from me, I’m from there.

Many smart well-respected people that I acknowledge, say it is a mistake to assume that older generations were not, at least, as smart as us, and in some ways, we can find evidence that they excelled (for example, try learning entirely by oral tradition, no shared writing, READ: no books).

So we can’t say that those learners, educators, and leaders were dumb. They were trying to implement the latest, greatest technology in the best way and certainly there’s been plenty of time to try MANY iterations of the technology. For example, radio for adult learning, radio for kids,

radio for cows. Heh heh, just kidding about the cows, let’s leave them out of this.

~I included cows because there is some research already about there about putting VR headsets on cows and I’m totally befuddled by that. I’m like “Why? Just stop it.”

But the humans are there.

The humans are the same.

I’ll repeat that for emphasis.

The humans are the same.

So we have experiment after experiment; we change out the technology thinking THAT will cause changes in the learning. But the results come out the same.

Could it be the OTHER variable– the humans – causing the non-increase in learning?

I posit, yes it is.

Brain-based learning science (OK, use the word neuroscience if that makes you more comfortable) gives this as it’s prediction.

The humans are the cause of why the learning results are always turning out the same, flat-lining, goose egg in improvements. Humans seem to have a “speed limit” when it comes to learning. We all have it. We can’t break past it. (Why? that’s my second shhhhhhh reason.)

So that’s why I’m so confident that XR will not cause lasting improvements in education.

As long as we are using humans as our test subjects, the results will peg even.

To be clear, I’m all for the improvements in AFFORDANCES that VR will bring; for example, safely learning inside a VR volcano, or added safety information with XR glasses. But those will not cause an overall lasting improvement because eventually everyone should be able to learn inside of a VR volcano or with XR glasses at work. Eventually, VR will be ubiquitous and not…

not the domain of the rich kids.

Continue reading “XR Will Not Cause Lasting Improvements In Education”…

2021 Bests and Worsts

 

I drew up my list of Best and Worst for 2021 and to make it balanced, it has 3 on each side. Here we go:

Best

1. Meeting Sriya Chintalapalli.

I count meeting Sriya as a golden moment of 2021.  I actually haven’t had long chats with her. But I was given a small heads-up for a student XR conference that I was supporting that a speaker was coming that was going to be amazing. I think the ‘knock socks off‘ phrase might have been used. I was under FERPA regulations to know that she needed extra protection at the conference and I volunteered to give it. That means I stood on the virtual stage with her, playing the role of direct tech support but also crowd control if necessary.

But what did happen meant something much more to me.

Sriya gave her presentation. It was a great topic and very forward looking. Then, she took questions from the audience. Because the topic was on brain-computer interfaces (BCI), it didn’t take long before questions of invasion of privacy questions came from what were obviously professors in the audience.  

I’ve seen these verbal examinations before. I’ve seen them break college seniors and Master’s Degree students. It’s just enough questioning to find where the student does not know the answer. That’s the push point. Several men in the audience were going right for her, directly and academically.

Standing on stage with her, without her knowing it, I would have thrown up a shield if she needed it and blocked those men from getting to her/embarrass her/humiliate her by making some excuse that we’d run out of time, audio wasn’t working, etc.

But, she held the stage. She held her ground. More than once she said “The data doesn’t say.”  

Good line! Don’t let them pin you where you have not staked a claim.  She’d been trained well to enter an academic fight.

When she was done, I let out my breath.

Were those men plants in the audience? Not sure. Maybe. Either way, my hackles were real.

And the lesson for me that day was: if I can do anything to help women like Sriya…even if it is only shouting “Make a path!“, I will.  It’s very hard to be a woman in the technological sciences. The road ahead will shape her in ways I’m sorry to contemplate. May she always find a woman like me standing by, ready to help.  

Please follow her. Great things are ahead.

2. A small unheralded research paper, HMD Type and Spatial Ability: Effects on the Experiences and Learning
of Students in Immersive Virtual Field Trips.

I was able to meet the first author, Pejman Sajjadi, at the IEEE VR conference in March/April 2021 in avatar form here. This small piece of research stayed in my mind all year as a great example of the piecemeal way that scientific research works its way slowly towards practitioners and teachers.
 

The write up of this study is pay walled behind IEEE, I believe, and Pejman would be the first to point out the small sample size. Therefore, there was no fanfare and no social media on this paper. If you look at his research background, what you see is this paper is just one of several papers generated from one research event, so it’s pretty generic par-for-the-course research.
 
Taking into account all those discount factors, this tiny study investigated something that teachers do really want to know:   
 
Are expensive VR headsets worth it?
 
The answer is no.
 
There is much more to the no, of course, related to content, learning objectives, scalability, etc. But more so than ever in 2021, educators turned to VR as a more realistic mainstream learning choice. The price drop of the Oculus Quest 2 to $299 and further, the Facebook push for the work use of Workrooms to bring VR use directly into the workplace show that we are going to have to get more and more comfortable with VR headsets and quality will be a question.
 

 (Image source: https://about.fb.com/news/2021/08/introducing-horizon-workrooms-remote-collaboration-reimagined/)

Quietly researched, small sample size, no social media presence.  
 
But bit by bit, researchers are answering these questions. I hope teachers are listening to the work of Pejman.

P. Sajjadi, J. Zhao, J. O. Wallgrün, P. C. La Femina and A. Klippel,
“HMD Type and Spatial Ability: Effects on the Experiences and Learning
of Students in Immersive Virtual Field Trips,” 2021 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2021, pp. 546-547, doi: 10.1109/VRW52623.2021.00155. https://ieeexplore.ieee.org/abstract/document/9419337

 

3. Equal Entry and XR Women 

It’s a tie! Both organizations work for similar goals: 
  • Equal Entry has a strong drive for accessibility and has a section of work dedicated just for VR, AR, and XR.
  • XR Women‘s mission is dedicated to getting women’s voices up on stage as part of the narrative about the ongoing and future directions of XR.  
  • Both organizations stay focused on their task and welcome listeners, newcomers, and allies.
Both groups alike are working on accessibility into the coming metaverse for all.  I applaud their efforts.
Now time for the worsts.  Is worsts a word?  You will notice a theme from the Bests that carries through.  Here it comes…

Worst

1. Not necessarily restricted to 2021 sadly, say the phrase “Women in XR” and you will likely get this image:

Or this one.  That’s not even a woman on the right. #dehumanizing means you treat women like animals.

Actually, as I prepped for this article, I went to find one screen capture of a woman in a short skirt playing Beat Saber so that I could use it as a example of a poor behavior.  I thought finding one image of a woman in a skirt would be hard. I had remembered seeing one.  
 
Much to my shock and horror, it turns out….it was drop dead easy.  So easy, nearly EVERY image on YouTube for playing Beat Saber is of a young female scantily dressed.  Check it out:

I counted ~9 images of women playing with either bare legs, bare midriffs, sports bras, etcs, for every 1 man.
Think that’s a coincidence?  Oh no. It’s BY REQUEST.  Look video info at the bottom of this image I just posted above again.
 
 
It says:
 
“Song + Outfit per George T’s request! To request songs & outfits/costumes become a Patron at…”

This woman is taking money to have herself videoed/green screened playing Beat Saber in a short skirt.  Don’t tell me that the Patron isn’t begging for that skirt to fly up at some point. I know what you can see through that black skirt by outline.  In these videos, women have not only lost body space control, they are selling it.

 
It’s disgusting. And this is ALL OVER YouTube.  There’s a research project in there to count the views of Beat Saber videos without skirts versus those with.
Remember that the Quest 2 was a major Christmas gift for 2021 and your daughters are now –January 2022– watching YouTube videos to learn how to get better at Beat Saber.  Is getting better at the game the only thing they are learning?
Think that this is just about fun, though?  Really? Did you read what happened at late 2021 a technical conference ad?  Reminder: Major “Game” conference, no women speakers on the ad, and a sexbot prominently featured. This is what women in tech are facing when we “go to work.”

Women have been getting groped at tech conferences during large standing-room only keynotes. It’s real that women feel less comfortable in HMDs because they give up body space control. 
 
At any conference right now, by putting on a headset, women take a risk that men do not.

2. Major immersive learning researcher responds to an accessibility question with “I don’t know why a blind person would ever use VR.”

I was running tech support. I was on mute. I sputtered.  But the researcher’s mic was hot. The video caught that…I think. It’s out there.  
 
But what does that matter if it’s on video or not, if the researcher truly thought that?
 
I don’t even know what to do with that.
 
Major. US. Immersive Learning Researcher.  
 
😔
 
By the way, for you, reader,  in answer to the question, contemplate this:

Screenreader Experience of a Virtual Reality Conference by Rhea Althea Guntalili

and  

Virtual Reality in the Dark: VR Development for People Who Are Blind | Accessibility VR Meetup Recap by Aaron Gluck (YouTube link and transcript available at this link)

3.  Microaggressions against women in the XR industry
 

I left 3 organizations in 2021 and am no longer associated with them. It’s apparent now that I could not stand up for the rights of women and for accessibility in XR without being targeted myself.
 
“A microaggression is a subtle behavior – verbal or non-verbal, conscious
or unconscious – directed at a member of a marginalized group that has a
derogatory, harmful effect. Chester Pierce, a psychiatrist at Harvard
University, first introduced the term microaggression in the 1970s. ” https://www.thoughtco.com/microaggression-definition-examples-4171853
 

The last organization I left questioned if I was a dues-paying member, so they used an institutional rule to execute an exclusionary move.

We’ve heard about headset straps that do not adjust for varying hair styles. Women and people with disabilities that are not recruited into research studies so that research results are invalidated when applied to major populations, conferences that not only host but advertise manels with sexbots, and the list keeps going already 7 days in 2022…
😔
 
Continue reading “2021 Bests and Worsts”…