Cogitate and Percolate

Category: Bad Research

Does ChatGPT Enhance Student Learning?
Not in 2026, it doesn’t.

ChatGPT enhances academic performance.

ChatGPT boosts affective motivational states.

ChatGPT improves higher-order thinking propensities.

ChatGPT reduces mental effort.

Source: Does ChatGPT Enhance Student Learning? A Meta-analysis (Deng, et al., 2025).

All of these statements, however, are ‘bent’ and are not necessarily true. Why? Watch the video below.

TL:DR
- It’s too early to conduct an AI meta-analysis.
- Effect size is actually 0.25, with no statistical significance.
- Authors did not include papers that show ChatGPT caused harm.
Thus:
- Not all research is created equally.
- Not all data are created equal.
- Knowledge takes time.
- Lying with data is super easy.
I’m sharing this because many folks disregard reading research papers altogether and will only hear the headline. Others will only read abstracts. Others will not recognize that the published paper’s research was essentially bad.

Sources matter.

Legitimate sources matter.

Research methodology matters.

It’s a tough world to navigate, instructional designers.

Let’s be careful out there. 👮♂️
February 23, 2026
The predicted downfall of ENGAGE XR

Photo by Megan Lee on Unsplash

Given that it looks like EDUMetaverse is headed to the turf, I might as well get a blog post up prognosticating that ENGAGE XR is going down too. I might as well. I’m not even early to these thoughts; I heard through the grapevine that ENGAGE XR was laying off folks within the past 12 months. I did research that and it is true.

LinkedIn shows negative 41% employee growth in the past 12 months.

Negative 41% employee growth.

The Irish Times reported a stock slippage of 8% on January 6, 2025 due to contracts with Middle Eastern partners that had not come through or realized yet.

By June 2025, there seemed to be further delays and possibly more loss, although this article from Sharecast from 6 months later seems to have the same figures as the January article. So it is unclear to me if this is more of a tumble or a new tumble. With the dramatically increasing interest in AI during this time, it would not surprise me at all if the investors were looking elsewhere for returns.

I had heard “layoffs” but didn’t know of anyone specifically. It appears that Glassdoor did. 😕

To be clear, I’m not kicking a man when he is down here. I’ve had a beef with ENGAGE XR for years due to their false learning claims using virtual reality. I’ve written before about their contract with Meta and Stanford University. I’ve written about their foolish “metaversity” concept. I’ve written about their strange use cases and the evidence that they use to push their snake oil use of VR for learning.

Not aging well? Morehouse College, ENGAGE XR, and the Metaversity

So this is a case of the chickens coming home to roost. One can advertise all one wants to about how great one’s XR is, but if one is propping up untruths, failure will follow.

BTW, in the same bucket with ENGAGE XR is VictoryXR. I might write this out more later. For sure, with the early evidence of what AI is doing to education (impact on learning), it’s going to get ugly.

Note: be aware that there is more than one company with the name “Engage” in the virtual reality space.

February 14, 2026
XR for education propaganda – EDUMetaverse
propaganda:

1 – ideas, facts, or allegations spread deliberately to further one’s cause or to damage an opposing cause

also

: a public action having such an effect

2: the spreading of ideas, information, or rumors for the purpose of helping or injuring an institution, a cause, or a person

Merriam-Webster Dictionary entry

I was so tempted to make a reaction video to this video within Andrew Wright’s LinkedIn post. But no. Instead, I’ll just point out the XR for education problems therein.

Post:

If you’re wondering what #immersivelearning looks like. Watch the clip of today’s onsite session till the end. Innovate, Engage, Inspire
This is not a ‘flash over substance’ experience, at EDUmetaverse, this is the real deal!

Consider that this is one lesson of ten, from one world out of a hundred, you’ll then get some idea of what we’ve spent five years creating.

Designed by teachers, for teachers. Available now as part of our education bundle for 2026.
All you need is a browser..

✅ Immersive Worlds
✅ Engaging PBL
✅ Relevant Content
✅ Curriculum aligned
✅ A.I Literacy embedded
✅ STEM based
✅ Teacher Created

What are you waiting for? Get in touch today.
www.edumetaverse.com.au

#educationevolved #ai #vr #mixedreality #3ddesign #stem #webxr #pbl #generativeai Frame MeshyAI Lauren Main Andrew Google Flow Apple

Strong Words

Wondering what immersive learning looks like?
Not ‘flash over substance’?
The real deal!
1 of 10, 1 of 100!
Spent 5 years creating

Designed by teachers, for teachers (ouch. I apologize to teachers on EDUMetaverse’s behalf, cause EDUMetaverse has a tendency to throw y’all under the bus, regularly.)

Then a bunch of key phrases: STEM, relevant, immersive! Probably written by AI. 🙄

I did not find the same video posted to any other EDUMetaverse social media (huh? 🙄) . I’m going to show screen captures with my written descriptions.

Opening scene, upbeat music: It’s Avatar Andrew inside of a FrameVR/Virbela world that looks like a stadium during winter. Avatar Andrew is standing on a blue running track looking towards empty wooden spectator seats where a real world ski jumping video clip plays and 2D picture of a 3D model of a ski jump is displayed.

Students are watching a flat screen monitor 🙄, where an EDUMetaverse world is shown and inside that world there is what looks like an EDUMetaverse produced video and some Olympics mascots. (I searched for matching clip or 3D model, didn’t easily find anything but it isn’t hard to guess that they could have been made by AI.)

Capture of students looking into EDUMetaverse world

An interface shows a 3D ski jump model. I’ve never used EDUMetaverse, so I’m guessing this is a compose or build-type of interface. Interestingly, we can see that AI is doing the building because there is text: “Prompt: An olympic ski jumper in jump mode leaning forward over skis…” and “Generating 70%”. In my experience with VirBELA, this looks like a VirBELA-like interface. Note: this supposed result doesn’t appear anywhere in the video. (cough, AI fail? 🙄 cough)

A little more video of Avatar Andrew watching real world ski jumping video in world. 🙄

Then what must be a post-production edited still shot (NOT video) because an innocent student appears to be pointing to something that doesn’t exist but a ski jump has been placed into the shot. This is AR-like. In my opinion, this is a faked video shot and it is poorly done. 🙄 For fun, I noticed the colored bracelet. Can we see it elsewhere in the video OR was the student’s hand a new creation from somewhere else? (Spoiler: yup, the bracelet is on a student later).

Nomination for worst AR faked video shot

Then a slick EDUMetaverse video clip of a ski jumper.

I asked Google Image to find this image as I thought it might have been a clip produced by the Olympic organizers or broadcasters. Result: “No exact matches found. This could mean the image is unique or has not been widely shared yet.” Technically, that is one hell of a ski jumping video clip if it was based on ANY form of real reality cause the top of that ski jump is literally as tall as mountains. 🙄

Then back to videos of students sketching a ski jump. At this point, I don’t know why since I thought this was a pro-VR video. But I have had a great deal of fun with The Sum of All Thrills where one designs a roller coaster so I’m aware that working on design is a fun step.

Students are creating a ski jump from a cardboard box. Imagine my surprise. Is this a middle design–like between the drawn designs and the 3D one? Looks fun…but…why are they doing this? 🙄

Quick shot of a student navigating inside of the VR world that is simultaneously displayed on a bigger screen. I don’t know why the student is doing this. 🙄 Displaying it on a bigger screen is intriguing, though.

Final scenes with students show them letting a marble roll down and off their cardboard ski jump models. At this point, I’m like “OK, let’s take these skills into VR somehow or…what?” No joy. 🙄 It doesn’t have to go back into VR, I know that. But this is a VR company so I’m looking for them to clinch the promo.

Lesson had a claim to be related to STEM (overall EDUMetaverse website claims that their lessons have ‘PBL packs’, problem-based learning) but I’m not sure I ever saw any math, anything measured or calculated. 🙄

But the piece de resistance that threw me over the edge was the post-production video edit of a ski jumping going the wrong direction on to/ off of the cardboard ski jump.

Yup, ski jump is definitely going UP the jump, left to right across the screen.

Dear Jumper, that is not the correct way to use our ski jump.

Executing a truly miraculous pivot 90° to the right at the height of the jump. Impressive for a ski jumper, that is. To be fair, less impressive for a freestyle skier. 😒

Really nailed the landing well. On the desk. Which wasn’t really part of any of the students’ designs. This is one prescient ski jumper.

Who is Veo anyway? I just noticed their watermark in the corner.

I won’t link to Veo here because when I surfed there, it took over my browser dominantly. I would steer clear.

Veo makes AI-generated Clips.

Educational value

So…how does this product (which provides no prices upfront, you need to ask for a quote and hope for your educational discount…from a company with EDU in their name 🙄) actually add to the educational experience where students made ski jumps and rolled marbles off of them?
- The students watched a video about ski jumping inside a virtual world.
- Then they did something with generative AI about making a ski jumper?
- Then they made their own ski jump models out of cardboard and rolled marbles off of them.
I didn’t see any measurement of angles or distance.

I even think the students’ faces look a little disappointed as their marble doesn’t sail up into the air much like a ski jumper does.

before and after

So where’s the learning added? Where is the advantage of using the product?

where’s the beef?

Students could have watched that 2D ski jumper video outside of the Olympic world. Technically, everything I saw happening in world was unnecessary.

Yeah, it would be a tad more boring but when the immersive Olympic world doesn’t add anything, it is a distraction. Unnecessary information should be removed (Mayer’s Principle of Coherence).

BTW, who’s going to tell them that PBL is falling out of fashion?

So let’s score them against their words

Wondering what immersive learning looks like?

No, but that’s because I’m a specialist in immersive learning. What you’ve shown ain’t it.

Not ‘flash over substance’?

The video and supposed learning has no real substance. You might want to re-think using the phrase ‘not flash’.

The real deal!

😆

1 of 10, 1 of 100!

When in doubt, dazzle them with statistics!

Spent 5 years creating

what. a. waste.

BTW, your YouTube says you went AI crazy 6 months ago. Sure you want to stick with 5 years?

EDUMetaverse reputation

Interestingly, EDUMetavere’s YouTube account is empty! What?

This channel doesn’t have any content. You’re telling me.

And Andrew’s YouTube account is full! huh? (No comment on this Jess Jones AI agent…but…let’s just say there is a LOT of content with her.)

Did you know that you can spin up ‘blank’ avatars, basically avatar bots, in VirBELA based products?

I got $5 💸saying this image contains bots

Summary

I don’t begrudge the students. Poor souls having to be dragged into this. They remind me of the poor HTC Vive students. I’m glad the students made their cardboard ski jumps IRL. But somebody get them a tape measure. Get a physics teacher in there!

But for the love of God, please have your fake AI video have the ski jumper going down and then up OFF OF THE SKI JUMP, not the opposite. That is, if you are going to highly produce your propaganda about how your VR helps learning, have the ski jumper go from right to left, not left to right.

Here’s how to goes:

Notice how the ski jumper slides down and jumps up off the ramp?

What’s my main problem with the video/post?

It does a terrible job of portraying a possible way to use XR for education. Even if one looked past the faked video shots (and I don’t have a direct beef against using AI for video clips, even though I mourn for the proper actors put out of jobs with this), teaching this way with XR is awful. I see no educational benefit at all.

All in all, posts like this (and Andrew posts like this very often) do more harm than good to the XR for education industry.

Over and out.

Post script

I usually add more to my blog posts after publication; don’t be weirded out. But this one is quite the eyebrow raiser.

The blogger records show that I published this blog post on Saturday February 7, 2026 at 11:27 A.M. EST.

On ~Sunday February 8, 2026, Andrew Wright published on LinkedIn that he was leaving EDUMetaverse, a company by his own LinkedIn tagline he “created” saying that “the project is now in safe, capable hands.”

I’m not implying that Andrew lost his job with EDUMetaverse because of my blog post. Far from it. My read stats of this blog post immediately upon posting/sharing it and all up to this very moment of writing this post script on February 14, 2026 show that there have been at best ~2 views. I highly doubt that Andrew was one and EDUMetaverse (whomever that is) was the other.

But the coincidence is A-MAZ-ING.

And if you followed my inference in my blog post, I immediately wondered if Jess was taking over at EDUMetaverse. 😖

An interesting idea: build AI…and it takes your…job? 😕
February 7, 2026
When Tech Platforms Donate The Resources

[EDIT: This post was originally written in January 2023 and lightly edited in 2025]

Jeremy Bailenson

As much as I admire Jeremy Bailenson’s research work (really!) his Communication 166/266 Virtual People course in June 2021 had some real problems. In its defense, it was a first-of-its-kind course, even if it wasn’t the very first course in VR. Depending on how to define VR versus XR, groups of this size, 263, have met synchronously in other platforms.

Boast Much?

Bailenson defends: “To the best of my knowledge, nobody has networked hundreds of students
(with) VR headsets for months at a time in the history of virtual
reality, or even in the history of teaching.”

Further, he states:

The scale of this course is what sets it apart compared to other “in-VR” courses. In addition to having a relatively large number of students enrolled in the course, we also had a large number of sessions taking place in VR over time, many of which were in a networked virtual environment. To our knowledge, prior courses that have used VR in an educational setting have rarely accomplished all three of these criteria.

Here is a YouTube video, Stanford “Virtual People” class in the Metaverse posted by Bailenson.

The ENGAGE Platform

In the video clips, we see the ENGAGE platform.

Why ENGAGE? It was not deeply explained, here:

In addition to the headsets, the course also needed software to connect
the students and teachers. For this, Bailenson said the university
decided to use the ENGAGE virtual communication system. ENGAGE is used by major companies and educational organizations to hold virtual meetings and events.

A Big Problem

I looked at some of the film clips closely. I searched and the early clips appear to be deleted off of YouTube. I have facilitated small and large events in XR.

In the video clips of this course, I can detect that sound appeared to be a somewhat major problem in the platform; getting users to hear, signal that they could hear, or having multiple groups in one space (like a lab) and hear over top of each other.

The Headsets

Learners in the course received the Quest 2 headsets.

“Virtual Reality is becoming mainstream, with more than ten million
systems being used in the United States alone. This class examines VR
from the viewpoint of various disciplines, including popular culture,
engineering, behavioral science, and communication. Each student will receive an Oculus Quest 2 headset, and the bulk of our learning will
occur while immersed in VR.”

Each student was given the headset:

Each was given an Oculus Quest 2 headset

According to another course from 2022, headsets were to be returned at the end of the semester:

Facebook Meta provided a “workaround” for the forced use of Facebook accounts in the headsets:

The Facebook login requirement had sparked complaints and privacy
worries, leading some organizations to seek a workaround. Stanford
University uses Meta’s headsets in its courses on VR, said Jeremy
Bailenson, the founding director of the institution’s Virtual Human
Interaction Lab. To ensure student privacy, the lab had to seek Meta’s help in creating anonymized accounts for classroom use.

This article comes right out and says this:

And money for the project—as well as donated VR headsets for students at
the participating colleges—comes from Meta, the company that owns
Facebook.

The connection between Facebook Meta and Stanford has been documented.

While the experience was good in that, at the beginning of trying out any new technology, there will be false starts. Said another way, it is good to learn that bringing in 30 learners to one large-ish lab space to teach separate labs of 5 people each won’t work if there is flat sound. That has be learned. I think his course showed that.

But overall, conducting a course with donated technology and then turning around and saying the learning was great* is a conflict of interest.

I found a written summary here, but it’s light on conclusions. There a few glimmers, but otherwise, they did seem to hint that the groups versus sound problems that appeared in the video did happen.

* What does “the learning was great” actually mean? Bailenson and Han claimed better presence, enjoyment, motivation, and transfer. While I could let you consider if any of those deserve merit, I railed against the conclusions of the course in my The Immersion Delusion post. This post, being written more than 2 years before I hit publish, focuses on the hype just as the course was starting. Therefore, obviously, this particular post does not hit hard on hype versus results. It only focuses on hype and the conflict of interest of hitting the airwaves with how amazing your course must be, to be a first of its kind, learning about VR in VR, yada yada yada.

[EDIT: I decided to publish this post on 12/26/2025. I’ve done quite a deeper dive on that course and the publications around it. I feel even more confident and I edited this article to come right out and say that Bailenson had a conflict of interest, rather than a “dis-authentic event in research” around the entire course and following publications.]

Sources

Stanford course allows students to learn about virtual reality while fully immersed in VR environments

Stanford Now Offers A Class Held Entirely In Virtual Reality Using Quest 2

Into the Metaverse: Designing the Future of Virtual Worlds

Meta plans to make VR headset business-friendly

With Money From Facebook, 10 Colleges Turn Their Campuses into ‘Metaversities’

US University Launches First Full Class in Virtual Reality

Learning About VR in VR

Video of spaces from Victory XR (Unsure if these were used in the Stanford course or not)

December 26, 2025
Seeking Integrity in VR Educational Research 3: It keeps on happening
It keeps on happening

If I were you, by now, I’d be asking, “Heather, why are you doing this? Why are you stirring the pot? You claim to be pro-XR for education but you are reviving research from long ago just to pick on it. It’s old news.”

[To protect identity, I am PURPOSELY going to change some things by asking AI to rename and reword some of these statements.]

Heather steps up the microphone and says “Within the past 3 weeks…

New Journal

We’ve seen the launch of the International Journal of Emerging and Disruptive Innovation in Education : VISIONARIUM

Title proper: VISIONARIUM :

Abbreviated key-title: Visionarium

Other variant title: iJEDIE

Other variant title: International journal of emerging and disruptive innovation in education

Original alphabet of title: Basic roman

Subject: Dewey : 371

Subject: Education, teaching, training of special groups of persons. Special schools

Corporate contributor: Lindenwood University.

Publisher: [St. Charles Missouri]: Lindenwood University, 2023-

Dates of publication: 2022- 9999

Description: Began with: Volume 1, issue 1 (2023)

Frequency: Three times a year

Type of resource: Journal

Language: English

Country: United States

Note: Volume 1, issue 1 (2023) (digitalcommons.lindenwood.edu viewed Aug. 8, 2023).

Note: Volume 1, issue 1 (2023); title from cover image (digitalcommons.lindenwood.edu viewed Aug. 8, 2023).

Medium: Online

Indexed by: ROAD

Journal summary:

The journal provides a diverse, interdisciplinary forum for the
publication of original peer-reviewed scholarship, data, and research
addressing intersections of education and technology. Education in all
domains increasingly incorporates emerging technologies and their novel
use in learning environments, such as current pedagogical explorations
of gamification, mobile and adaptive learning, digital humanities,
machine learning, blockchain, Artificial Intelligence (AI), and
Immersive Realities, to support innovative teaching methods and engaging
learning experiences. With the rise of new educational platforms and
metaverses, iJEDIE focuses on emerging trends in research to bridge the
artificial divide between scholarship and innovative pedagogical
applications. Submissions to iJEDIE will include, but are not limited
to, the following themes of interest:
- Emerging technology and pedagogical application in specific disciplines or learning environments
- Issues and applications in secondary education
- Issues and applications in post-secondary education
- Application of education technology in enterprise, industry, and nonprofit environments
I’m sorry, could you hit me over the head with the word application one more time?

Published by Lindenwood University -a NOT regionally accredited
institution, however, their Teacher education program (which this would
appear to be under the auspices of) is CAEP accredited. Unfortunately,
it’s not a strong tie to claim that a particular university or
institution’s reputation applies to the people within. It’s very
possible (and I’ve seen it!) but it’s a weak link, IMO, as great
researchers can be within poor institutions and vice versa.

Interesting how the journal description looks like the panel it was derived from…

“April 21, 2023, the Senior Editorial Board and organizing committee of the
International Journal of Emerging and Disruptive Innovation in Education (iJEDIE)

hosted a panel of speakers on Emerging Technologies and the Future of
Education. The session invited researchers and practitioners from a wide
range of fields, including Education Technology, Digital Humanities,
Extended Reality (XR), Artificial Intelligence and Machine Learning, and
more. Speakers will discuss their recent research into how emerging
technologies may be used to disrupt, enhance, and/or revolutionize
traditional approaches to education for the benefit of both teachers and
learners.

I italicized and/or bolded the similar wording between the panel and the journal.

I don’t know about you, but I’m getting a very strong “applications” vibe here. Notice how application is contrasted to research. Hmm…can you say “chip on shoulder”?

Speaking of the journal

This is a quote from volume 1:

“There is now clear evidence that virtual reality can greatly enhance academic performance and educational attainment for students in both academic and higher education institutions” (Rephrased via Microsoft AI).

This sentence came from the end of a literature review section, which in fact, did NOT make this particular statement CLEAR with EVIDENCE.

Hello? Editors? A good editor would catch a claim like that NOT being substantiated in writing. You do plan to have editors in your edited journal, right?

But wait there’s more

No need to wait to read articles that contain this like of non-editing! You can just read a special issue coming out next year that is dedicated to, ahem, utilization of XR. Membership in the parent organization is US$150/year.

JAID special issue, sponsorted by AECT ($150/year membership):

Journal of Applied Instructional Design (JAID)
Special Issue: Designing Extended Reality (XR) for Authentic Learning

Watch how the highlights are nearly all the same concept:

For this special issue, we are interested in presenting current research in applied
instructional design methods for utilizing VR, AR, MR, and other immersive
technologies to foster authentic learning experiences. We are inviting articles that will
provide readers with practical ideas, strategies, methods, and techniques on topics related
to designing, implementing, and evaluating instruction using XR for authentic learning
experiences. Furthermore, we seek contributions that provide evidence about the efficacy
of XR technologies, including the challenges encountered during their application in
authentic settings. The articles should inform the study and practice of immersive
learning in preschool, K-12, higher education, or work-based contexts. We invite
scholar-practitioner perspectives as a means of disseminating and developing new ideas
in instructional design. We aim to share expertise, success stories, and lessons learned
from failure.

Everything old is new again

Oh and do you think the PwC thing is old news?

5 days ago on LinkedIn:

and here is the luscious “4 times” quote! (If you’ve been reading along, you know this is the key phrase to look for.)

And there there was this comment, saying “That study is gold”

It’s a report. It’s not research. It’s marketing. Say it with me “MAR-KET-TING”

And the commenter is using it for their dissertation on “the potential impact of IVR learning platforms on teaching presence”?

The PwC report did not measure presence, in any of the academically accepted ways, nor any of the man-on-the street ways. The word presence is in the report zero times.

Summary

Falling into the trap of thinking that just because it is published means that it’s fact-checked is false.

Most of the volunteer reviewing jobs I’ve been on contain 2 reviewers and 1 editor. Rarely do I ever run into anyone else with an educational psychology research background that knows about research models that do not stand up to publishing scrutiny (methods like comparing non-comparable instructional methods or exposing learners to novelty effect). I know a source that ran a 91-93% acceptance rate on articles. Owch! That’s the “write your name at the top of the paper and you get an A” publication standard. Cringe!

A person’s biases show up in their writing and editing– this happens to me just the same— no stones being thrown in glass houses here.

But there has been an undercurrent that I’ve detected running for the past 3 years:
1. Most folks are generally skeptical about learning in VR. It looks like a game.
2. Pro-VR people realize that “published research” is a way of adding validity & gravitas to their pro-VR stance.
3. Pro-VR people have been slipping pro-VR pieces of research into low publication standards sources and getting their overblown and hype statements like “staff learn 4x faster” flown right under radars.
4. Pro-VR people sit back and say “The research proves it! Come and buy some VR for education!”
This all happened in the past 3 weeks. August…August of 2023. Can you see way this Seeking Integrity series must continue?

I just can’t face palm enough.

#VirtualReality #VR #XR #VRForLearning #Technology #Future #edtech #learning #education #InstructionalDesign #research #ComparisonResearch #Media #MediaForLearning #ImmersiveExperience #Design #ResearchIntegrity #publishing #review #editor #provr #journal #specialissue

This blog post was updated on April 11, 2026 with an improved font.
August 28, 2023
Misleading Headline or I snagged a live one!

Oh, I snagged a live one! I really didn’t think this image would garner much attention as my posts rarely do on these topics. 2022 has been a bad year for keeping XR friends.

But I’ve been contacted directly by a nice LinkedIn link.

Incoming message

(DM’ing instead of commenting on your recent post here.)

Why are you distorting the facts?

The article says:

“The survey revealed that students showed an average of 68 %
improvement in a test taken after learning with Futuclass VR lessons
compared to the results before the 30-minute VR lesson.”

This is very different from “started from zero and learned 68 out of 100
percent”.

The source is here, wasn’t hard to find. https://futuclass.com/blog/how-VR-lessons-increased-chemistry-test-scores-by-68-percentage/

And looking up the teacher involved, quoted in the article, she has
multiple publications at hard science institutions at Tallinn
University. I somehow doubt she’d agree in your assessment on her
scientific rigor. https://www.etis.ee/Portal/Persons/Display/95da33ee-9b4c-4e72-9cab-3e72e3791c0c?

I’m all for criticizing messaging and methodology of these “research”
reports, but when you skew the actual facts you’re undermining your own
point.

So, my response:

Good sleuthing! I like that you want to look at the actual numbers.


I did so. I did read the article before I made my post.

May I direct to a
few more sentences lower down in the article?

Some of these will be confusing, so by the end of this, we’ll have to
look at the entire article and not just stick with exact sentences.

Here is the one you quoted and I do see it:
“As a result, students showed an average of 68 % improvement in a test
taken after learning the metal oxides during a 30-minute lesson using VR
compared to the results before.”

This one is not clear what “to the results before” is actually referring
to. Could it be referring to: other teaching, to non-teaching, to
“before” teaching? I read further down in the article to find out.


This appears to be describing more of the method: After testing
different approaches, the best results proved to be with group work and
paper worksheets. Students with VR headsets were expected to communicate
what was going on and solve the worksheets together with the students
without the VR sets.


I thought that was interesting because it implies other approaches that
but that this one was settled on and it included a non-tech component
(paper worksheets).


The next sentence is problematic: “Several studies have revealed that
first-hand experience is four times more effective than traditional
theory learning. “I believe that we should mix the experience from VR
equipment and practical work and this would be effective learning,”
admits Katrin Soika.”

It quotes “four times more effective than traditional learning theory”.
The “four times” is clued me in. The statement implies that VR is 4
times as better than traditional teaching. This is debunked research (by
me). If it referring to the PwC study (https://www.5discovery.com/wp-content/uploads/2020/09/pwc-understanding-the-effectiveness-of-soft-skills-training-in-the-enterprise-a-study.pdf
), that study had learning take place in 1/4 the number of minutes that
a classroom equivalent would have taken. Those are not apples to apples
comparison, but it was published by PwC that they thought VR was, then,
4x as effective.

Here is where we get to what the 68% was actually measuring: “With VR
education, learners are inspired to discover for themselves. Students
have an opportunity to learn by doing rather than passively reading or
listening. After the VR chemistry lessons students were surveyed about
their experience and the results were significant. The survey revealed
that students showed an average of 68 % improvement in a test taken
after learning with Futuclass VR lessons compared to the results before
the 30-minute VR lesson.”

In that paragraph, it becomes clear that the 68% was the “average”
improvement **compared to the results before the 30-minute VR lesson**.


That’s why I wrote: Learners learn after learning.

The 68% was the
score that they received (on average) after engaging in a learning event
– in this case – the 30 minute VR lesson.


So that’s why I wrote what I wrote.

Do you think I missed anything else quantitative in the article?

_ _

The source isn’t here

About “the source isn’t here”

That’s purposeful from me. I’m not out to point out the company or publisher’s name or link my comments with them on the LinkedIn network – especially when I have something negative to say. I let my hashtags carry the weight of passing my opinion around my network:

#XR #VR #VREducation #VRForGood #edtech #Chemistry #ChemistryIsHardIGetIt #VRForLearning #ReadCarefully #WriteBetter

Note: this article came to me via my LinkedIn network – via 2 friends, actually. So I’m only recirculating my opinion back out and NOT pointing out the article, company, or researchers.

Additionally, I posted to LinkedIn one image and one line of text “Let’s be careful out there” (a reference to the TV show Hill Street Blues). That’s it. I didn’t include a link.

_ _

The researcher’s qualifications

About the comment about the researcher’s qualifications: I don’t actually think the researcher/educator whose experience is IN this article wrote this article. There is no author quote, so I’d have to go with someone at Futuclass wrote this. I say that because some of the quotes for the researcher are put up close to sentences on something different– and thus, it doesn’t make sense in terms of writing. Usually evidence supports a claim, not says something very different and off tangent. By the way, when I see this, I always try to figure in possible changes in translation between languages. For example, could this article have been originally written in Estonian and then translated to English and thus some things just don’t sound like native English? Yes, that’s possible. However, the problems I see in the writing are more along the lines of illogical ideas or “pushed” writing (advertising) than straight research. So that would not invade actual word-for-word translations, but it would remain at the sentence & paragraph level- which is what I think I see.

Here is an example:

Several studies have revealed that first-hand
experience is four times more effective than traditional theory
learning. “I believe that we should mix the experience from VR
equipment and practical work and this would be effective learning,”
admits Katrin Soika.

So the quote shows that Katrin wanted a mix of technology and practical work to create effective learning. That seems to be her point. But the sentence just before makes the (wild) claim of the four times (but I’m not sure what of four times). So the ideas don’t connect.

BTW, the link to her qualifications is in another language. Regardless, I respect that she might have done some nice work here. Actually if you look at her quotes in isolation AND look at the “other properties” section – particularly with reference to chemistry education – I think that this educator is cooking with gas, as in, she’s doing well.

Conclusion

Overall my opinion is that I wish the “other properties” section would have actually made the headline and NOT the quantitative data. There is some really good stuff there. It has been 5 days since this interaction and no follow-up response from my link. 😟

This is just another case of if it looks too good to be true (68% improvement!!), it probably is.

September 26, 2022
I’ve Quit With Zero-Day Notice 3 Times Now. Sorry, Not Sorry.

Photo of a beautiful spot in Ukraine by Maksym Tymchyk on Unsplash

Zero day notice. Same day notice. Resigning and walking. Notifying your boss that you quit on the day you quit.

I’ve given zero-day notice 3 times. This blog is about why I found it to be the right choice those 3 times. I’ve been told I write long articles too (yes, I know!) so I’ll keep this moving along.

Time #1

I was working for an institution that was famous for quick & sudden firings, Western Governors University. On June 2, 2010, they fired 14 employees, one every 15 minutes, between 9 – 12:30. No severance. Their famous line was “Your position has been eliminated.”

WGU was fire happy like this

The institution was an At-Will institution in an At-Will State. I define that as employment can be terminated for any reason without notice.

The At-Will clause applies to both the employer and the employee. Notice that? The employee is employed at the employee’s will. Folks often miss that.

I was being retaliated against for being a whistleblower. I was in a horse race; they would fire me or I would quit. I was not sure which would happen first.

I consulted with two Human Resource experts over my zero-day plans. Both of them advised that given the institution’s reputation for firing and the at-will status, it would be acceptable to give zero-day notice in this instance. One even said that I’d be a champion for those that were previously fired because they might have wished to walk out on their own terms.

I calculated my departure day ~3 months in advance.

The unanswered instant message

The day arrived. At 9 in the morning, I sent my boss an instant message. “Hey, it’s important that I meet with you today. Can we find some time?”

No response. 9-10 goes by. 10-11 goes by.

This is just like her. Ignoring me was her management style.

Her schedule showed a “Leadership” meeting from 11 – 1 (ironic! 😂). I wait and continue to complete my work.

By 12:54 I really cannot wait much longer. I intend to give verbal notice but I can’t get to her to deliver it.

So I click send on an email written to her and Human Resources simultaneously.

Three minutes goes by.

Then my instant messenger goes off. “Oh Heather!”

My boss begs for a phone meeting. I put her off until 3:30 p.m. Guess who’s too busy now? 😏

I communicated the status of my projects by email. Because this institution runs on the Amazon 1:8 leadership model, I had 8 or less direct reports and I did not have much on my plate.

Once our phone call happened, it was a stilted meeting. I can feel that she wants me to state an reason for leaving suddenly (i.e. “I’ve been diagnosed with cancer, so I’m starting the treatments tomorrow”). That might absolve her of her guilt.

But I don’t give her a reason. I know I don’t have to. My employment was at my will.

I shut down the work laptop for last time at 6 p.m.

I never looked back.

(more…)

March 4, 2022
The Failure of Technology-Centered Approaches To Multimedia Design

Photo by Birmingham Museums Trust on Unsplash

Within the same morning, I had scanned The Total Economic Impact™ Of Mixed Reality Using Microsoft HoloLens 2, A Forrester Total Economic Impact Study Commission by Microsoft, headlined by the Senior Mixed Reality Specialist at Microsoft. I found the numbers inside dismal and took screen captures of the most egregious numbers so that I would not forget what jumped out as the most ludicrous (60% increase in efficiency in learning as an verbal report given in interviews by interviewees selected by Microsoft).

I also had been invited to a group that will “build a community of practice around applications of learning experience design in XR modalities.” But I had watched this community do a series in 2021 where they picked individual pieces of research and tried to derive principles for design in XR. I gave them feedback for the first 3 days. They kept hand-picking research and trying to establish large principles.

Err, that’s ethically wrong.

Plus, when I pointed out that some pieces of research– while fine as independent pieces of research, could not be applied broadly because of problems like cognitive load, comparative design, sample size, novelty effect etc. they would give me the hand wave response of “Oh yes, we saw that” but they never retracted or stepped back from the total theme and they had the ability to.

So….

I don’t see much hope there.

Therefore, I was in a pit of despair. Everyone around me is in some sort of technology-haze thinking it will solve all of their problems. Come to think of it, much of the field of instructional design for the past 18 months has been soaking in a technology tools fantasy. And yet, not a word about learning gains. Funny, that.

(more…)

January 24, 2022
2 out of 4

Twitter post dated November 29, 2021.

“It has been proven that people learn better through an immersive experience. The training tools powered by Virtual/Augmented Reality enable users to retain the material better while cutting costs & eliminating safety risks. Read [link]. #VirtualReality #VR #AR

So there are 4 claims here:

A. people learn better through an immersive experience

B. enable users to retain the material better

C. cutting costs

D. eliminate safety risks.

And here’s my vote on these claims:

A. people learn better through an immersive experience – No. 👎

B. enable users to retain the material better – No, because ‘better’ is flaky. I’ll shoot at “retain” too. 👎

C. cutting costs – Yes. 👍

D. eliminate safety risks.- Yes. 👍

The link provided goes to a business website that is selling developer services to make things in Unity. On the front page, there are NO claims about learning that I can find at all. So the “Read:” doesn’t seem to invite you to read more about facts supporting those claims. They are asking you to read all about how their business is cool.

Overall. that’s a score of 50%. Still, failing.

Remember that I’ve pointed to how dangerous & misleading “hand waive” language is. As soon as this started with “It has been proven that…” my hackles go up.

😦

November 30, 2021
Next stop: Bad VR Implementation
In my published writing of August 2021:

“Especially with decreased technology prices and increased access to XR,
campus administrators might want to buy the technology first and think
about use second. Instructional designers are obligated to advise on the
best use of the technology even if that advice is sought after the
purchase.”

Immersive Learning Environments: Designing XR into Higher Education.

It feels like the ink is still drying…and this pops up. An instruction designer asks how to best use VR headsets, after the boss has committed to buying some.

That set off my “bedonkers” filter 😜, so I replied:

After I posted the phrase “@ss backwards” that thread stopped for almost a day. Then one more post has arrived talking about a suggested resource.

But here we are folks.

Bad Implementation is our next stop on this train. Other stops on this train include:
- Overspending
- Results the same as other forms of learning
- Bosses disappointed in results
- Bosses reluctant to invest in the next big thing
- VR learning becomes laughable but slowly adopted.
Does all this seem familiar? If you are over 30 years old, it should be. It’s the e-learning adoption story. Go further back and it is the Internet-in-all-schools adoption story. And DVD-adoption story. And TVs in classrooms…

And film strips…

And radio…

And “moving pictures”

And individual textbooks

And chalkboards.

Think I’m kidding? I wish I was.
November 18, 2021