Analysis of The Total Economic  Impact™ Of Mixed Reality Using Microsoft HoloLens 2

(Image: Crowds gathering outside New York Stock Exchange Source: https://commons.wikimedia.org/wiki/File:Crowds_gathering_outside_New_York_Stock_Exchange.jpg)

This report, The Total Economic Impact™ Of Mixed RealityUsing Microsoft HoloLens 2, dated (November 2021) came through my LinkedIn feed in this post:

What most caught my eye was the fourth bullet:

Mixed reality increased
training efficiency by 60%, saving $1,440 per trainee while improving
knowledge acquisition and retention. 

I posted my response:

“With
respect, a company that sells a product that then claims the product
increases training efficiency with these kinds of numbers is to be
doubted.

In the case of the 60% increase, my eyes land on the
table on page 21 and the source of the 60% increase in task efficiency
data is “Interview data”, not quantified data. That means the
interviewees that Microsoft picked provided verbal estimates to the
interviewers. While the report takes great pains to describe how the
results were calculated– and I respect that– this is a “commissioned
report” (page 1)– that was paid for by the very company selling the
product.

Long term, results like these do not hold up.”

To which, an Associate Fellow at Lockheed Martin replied:

While the numbers seem outlandish they are very real based on our findings.

We
use mixed reality to build spacecraft and see well over these numbers
(~93% reduction) in touch labor. This was analyzed across 10
manufacturing sites and numerous spacecraft manufacturing programs.

Technicians consistently finish 8 hours worth of traditional work in 45 minutes, and other companies are seeing similar results.

For
training we see an 85% reduction in training time. Mixed reality also
offers greater comprehension than traditional methods.

While the numbers seem incredible, they have been validated with more than four years of shop floor implementation.

If you would like to chat details, just let me know. I would be happy to provide insight to our findings.

I did not reply further at the time to this post because I was not actually arguing time-based measurements. I was arguing improving knowledge acquisition and retention via task efficiency– admittedly their own squirrely wording. Said another way, claims that users learned more. 

This blog post is, in part, my response.