There you are, eating your morning bowl of cereal. You glance at your social media feed. Up pops these images:<\/span><\/p>\n <\/p>\n<\/p>\n <\/p>\n Wow. Even the infographic people seem amazed.<\/span><\/p>\n But My Executive Summary:<\/span><\/p>\n <\/p>\n<\/p>\n Let’s start with the nuts and bolts. <\/span><\/p>\n Mower, Despite that What you find at that web Something Disclosure is right up front (first sentence!) that:<\/span><\/p>\n “supported by Oculus for Business and Talespin,”<\/span><\/p><\/blockquote>\n Good Their study started in 2019 and ended in February 2020, so they indicated when it was run. <\/span><\/p>\n Note: They had a hypothesis: Our Remember that a And PwC defined \u201cmore effective\u201d with:<\/span><\/p>\n That definition of “more effective” is a little murky. Usually time<\/b> plays a very definitive role in “effective” measurements. For example: widgets produced over time<\/i>. Here, time is not actually mentioned and yet time is prominent<\/i> in the infographic stats later. Hmm..<\/span><\/p>\n In their experimental design, PwC appears to think that they have made comparable training: <\/span><\/p>\n Oo, bust here<\/b>. While I hat tip to the innovative thought process expressed<\/i> here, to take advantage of what VR can offer…<\/span><\/p>\n \u201cThe \u201cHowever, we determined this linear approach would not … but, owch, non-comparable methods!! <\/b> If you literally taught the information differently, you cannot compare the methods and thus, you cannot compare the results<\/b>. Now to parse out the difference between 360 Number of participants: 1600 possible. Good. But they never disclose how many learners they actually had in each group (their Ns)<\/u><\/b>. Experiences: 5-7 minutes long<\/span><\/p>\n Did use Oculus Quest (ahem)<\/span><\/p>\n Did use Oculus for Business for remote device management<\/span><\/p>\n Honorable One more small note: The report does a nice I Fly in ointment? VR can cost a great deal of money<\/i> for development and for the equipment. READ: the set up.<\/span><\/p>\n Counter to that<\/i>? I still don\u2019t like comparing \u201cclassroom\u201d to VR, such a not-fair<\/i> comparison.<\/span><\/p>\n \u201cWhat In the report, they shared numbers in minutes. <\/span><\/p>\n Classroom: E-learn: 45 minutes<\/span><\/p>\n V-learn: 29 minutes<\/span><\/p>\n 29 * 4 = 116<\/span><\/p>\n 116 is close to 120. Therefore, saying V-learn is 4 times faster is accurate. [Edit: In future writing, I clarify that stating “4 times” or “4x” is not a data lie, but it expressing a truth in a deceptive way. Learners didn’t learn four times faster, the literal training session was 1\/4 as long in time. So they sat through LESS training time. That doesn’t mean that they learned faster.}<\/span><\/p>\n This is a poor item to measure when we are focusing on learning outcomes. The Dunning-Kruger effect says that those least<\/i> able to accurately self-measure something are actually the worst at that thing.<\/span><\/p>\n They But alas, this is all future prognostication and not actual data. We can hope<\/i> for something but that doesn\u2019t mean that our hope leads to actual results.<\/span><\/p>\n<\/p>\n <\/p>\n 166% and 275% of what? Where is the 100% in this diagram? Said another way, what is the baseline? Zero?<\/span><\/p>\n I still can\u2019t find this in the report. <\/span><\/p>\n I\u2019m FURTHER, newer data<\/a> is showing that types of empathy count…not just general empathy or emotional connection.<\/span><\/p>\n There![\"Screen](\"https:\/\/blogger.googleusercontent.com\/img\/b\/R29vZ2xl\/AVvXsEgiWYi2CiVRMhjLntxB_I8gpYRB8j1oxdvIjzD1E0y2ThyQ8G88GNjkn75PpQ6yW7h9za_TfnSexMOwnwC0ln5peyCsAFe55iJJFr-VxX9lVLBqM16bh33raX5ny9JMuqH5Jd4C9fNbAWQSAtAAmZNk4aNAeYo-APtja6S_OgLNVnmQWj6o6qynneIMFgE\/w640-h360\/PwC%20%20Soft%20Skills%20Study.jpg\")
<\/a><\/div>\n
\n if it is too good to be true, it probably is, at least where research
\nand marketing intersect. Not to worry! I’m here to tear apart this and
\nsee what’s inside. Can virtual reality (VR) teach soft skills training?
\n Verdict: Maybe.<\/span><\/p>\n<\/a><\/div>\n<\/p>\nCitation & Report<\/span><\/h2>\n
\n Andrea. \u201cThe effectiveness of virtual reality soft skills training in
\nthe enterprise: a study\u201d. [Place of publication not identified]:
\nPricewaterhouseCoopers, 2020. Online. Internet. 21 Aug 2021. .
\nAvailable: https:\/\/www.pwc.com\/us\/vlearning.<\/span><\/p>\n
\nacademic-y looking citation, the writing does profess itself to be a
\nreport, not research. That’s good because reports are not held to the
\nsame standards of rigor as research.<\/span><\/p>\n
\nlink, however, is NOT the full report. You are looking at the corporate
\nsummary. Remember that technically, PwC is not in the VR business (a
\nplus) so they are not selling you something about VR<\/i>. They are only telling about how some VR training went at their company.<\/span><\/p>\n
\n quoted the report as “73 pages” but the website is not 73 pages long,
\nso I had to find the actual report. That took a little more digging but I
\n found it here: https:\/\/www.5discovery.com\/wp-content\/uploads\/2020\/09\/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf<\/u><\/a><\/span><\/p>\nExperimental Design<\/span><\/h2>\n
\n disclosure; it\u2019s good practice. While I’m not loving that it is
\nsponsored by a VR headset manufacturer and training creator, knowing
\nthis lets me view this with the appropriate amount of critical thinking.<\/span><\/p>\n
\n Every piece of research that touches 2020 and forward into the near
\npandemic future, should clearly mention WHEN the study was run, because
\nthe COVID pandemic is impacting every part of our lives<\/i>. COVID does impact \u201creports\u201d and we need to know if these are \u2018at-home-stressed-but-sent-a-headset\u2019 users.<\/span><\/p>\n
\n hypothesis was that training using VR is more effective in achieving
\nlearning outcomes than traditional training methods (classroom or non-VR
\n digital experiences).<\/b><\/span><\/p>\n
\nhypothesis in experiments is good. Hypotheses guide us to our data and
\nresults. Bias in experiments is bad. Bias makes us ignore our data and
\n results.<\/span><\/p><\/blockquote>\n\n
\n
\n classroom and e-learn course experiences were linear: A video was
\nshown, the learners asked some questions, then the next scenario was
\npresented.\u201d<\/span><\/p>\n
\n leverage any advantages of the VR modality. We hypothesized that
\nplacing the learner directly in the scenarios covered in the curriculum
\nand giving them the ability to act as they might in real life would be
\nmore rewarding for them.\u201d (p. 16)<\/span><\/p>\n
\n The key to making a good design that includes VR (and I\u2019m giving you
\nthe $64,000 answer here for anyone designing “compare” research with VR)
\n is to put it up against something very, very cognitively similar.
\nRight now, that technology is 360 video where you put in branching
\ndecisions. And add haptic bodysuits & controllers. But that convo
\nis for another day.<\/span><\/p>\n
\nvideo and VR, you\u2019ll need thousands of users. Not many investors right
\nnow have the willingness to spend thousands of dollars in equipment and
\ntime to collect that much data. And even when you get there, you
\nprobably won\u2019t find much a of difference. Why? Well, take a look at what
\n you designed. If you make a cognitively similar experience and run
\nhumans through it, it actually makes sense that no significant difference in the data will arise.<\/u>
\n Remember at this point, I\u2019m talking about learning outcomes ONLY, no
\nother characteristic. Also, go back and look at the hypothesis. They
\nalready thought that the ‘learning outcomes’ would be the same (READ:
\nsame scores on tests) but that they could achieve those learning
\noutcomes “more effectively”. That’s interesting. I wonder how much of
\nthis report was written from hindsight and how much was written before
\nthe study started. (This is why you write your hypothesis first–before
\nyour study.) That hypothesis is now possibly showing a bias…did they
\n*think* it was going to turn out…the way it did?<\/span><\/p>\n
\n Therefore, I’m going to call this a strong negative because they could
\n have included that number (I don’t see a business reason to conceal
\nit). <\/span><\/p>\n
\n mention in the report: Using the phrase \u201cnot the most gratifying\u201d when
\ntalking about tagging and inventorying what must have been more than 300
\n pieces of equipment.<\/span><\/p>\n
\njob explaining how they decided to buy 100 headsets and what the cost
\nand time of developing the v-learning training was. It is outside of my
\nscope to analyze that but I will recommend it as good to read. Remember
\n that if the cost of v-learn is dropping, it becomes a better and better
\n choice over time. They do those calculations and find that it becomes
\nthe better choice at 3,000 users\/learners. <\/span><\/p>\nLet\u2019s see what the data shows:<\/span><\/h2>\n
\n
<\/a><\/div>\n<\/h2>\n4x faster to train than in the classroom<\/span><\/h2>\n
\n might have to cede this one right off the bat because VR does provide a
\n 1:1 experience that most classrooms cannot beat. How many classrooms
\ncan provide– minute for minute– the same 1:1 attention of the teacher
\nto the student? Erps. Few.<\/span><\/p>\n
\n VR can be done for very little money and because training can be
\nreplicated a billion times when a 1:1 teacher cannot be replicated, VR
\nwins the day on this claim. READ: it CAN save time but after
\ndevelopment is done.<\/span><\/p>\n
\n took two hours to learn in the classroom could possibly be learned in
\nonly 30 minutes using VR. When you account for extra time needed for
\nfirst-time learners to review, be fitted for and be taught to use the VR
\n headset, V-learners still complete training three times faster than
\nclassroom learners. And that figure only accounts for the time actually
\nspent in the classroom, not the additional time required to travel to
\nthe classroom itself.\u201d <\/span><\/p>\n
\n 2 hours (Watch that crossfire, boys! They didn’t express this as 120
\nminutes. Heather gives PwC a strong look for that. Don’t be mean. Or
\nget a better editor.)<\/span><\/p>\n275% more confident to apply skills learned after training<\/span><\/h2>\n
\n make an argument that particularly with soft skills, confidence AFTER
\ntraining would help implement the soft skills in the workplace. I liked
\ntheir design inside the V-learn module. Learners had to say<\/i> their lines in the simulation. Nice touch!<\/span><\/p>\n<\/a><\/div>\n3.75 x more emotionally connected to the content than classroom learners <\/span><\/h2>\n
\n not going to pick this one apart much because the errors should be
\napparent. Often VR presents training as first person– meaning the user
\n looking through the headset is often the protagonist of the adventure.
\nTherefore, a story happening personally<\/i> to the user creates more emotional connection than the same story in a classroom. This comes from the bucket labelled obvious<\/b>
\n and is frivolous data. It also blatantly shows what\u2019s wrong with using
\n non-comparable instructional designs. This is an apples to oranges
\ncomparison.<\/span><\/p>\n4x more focused than their e-learning peers<\/span><\/h2>\n
\n are some great accessibility studies and autism spectrum studies coming
\n out that are showing us some very interesting research (really, it\u2019s a watch this space<\/i>
\n stuff) that VR can be more cognitively overwhelming for some learners
\nand hence every measurement that says learners have more brain
\nengagement could actually be learners overwhelmed (yeah, obvious bucket
\nagain). <\/span><\/p>\n
<\/a><\/div>\n<\/p>\n
\n