Why Google and Apple Will Rule Mixed Reality

The future of all new technologies is based on convenience, accessibility, affordability and ease of use.

Which means they will be rooted in devices and related services delivered by the one piece of technology with which consumers are already comfortable, if not obsessed: their smartphones. Advantage mobile providers. JL

Mike Elgan reports in Computerworld:

While Microsoft, Magic Leap and others are creating powerful, exciting mixed-reality products, they lack the one quality that Google and probably Apple have -- the intention to develop mixed reality that's affordable, mobile and mainstream. The mixed reality you'll actually use isn't the heavy, expensive, nonmobile version from Microsoft and Magic Leap. It's the one that comes free with your phone.
The brilliant future of mixed reality will come to us from the likes of Microsoft and Magic Leap, according to the conventional wisdom.
Microsoft has grabbed mindshare with awesome demos of its Hololens product, like its Project XRay game demo at its October 2015 Windows 10 event.
Magic Leap, which has raised more than $1 billion from investors, has dazzled YouTube users with simulated experience videos.

Other contenders in this space include ODG and Meta.

There's no question that the demos are cool and that several or all of those products will appear on the market and thrill users with a new experience that blends the real with the virtual.
But the future of mixed reality won't be dominated by Microsoft, Magic Leap, ODG or Meta, but by Google and Apple. Here's why.

The reality about mixed reality

Unlike virtual reality, which involves immersion into a fully computer-generated world, mixed reality blends the real with the computer-generated.
There are two kinds of mixed reality:
1. Head-up display (HUD): This kind of mixed reality simply places computer-generated words and images into the user's field of view without attaching those virtual objects to things or surfaces in the real world. Google Glass offers a HUD, as do fighter-jet displays and next-generation car windshields.
2. Object- or surface-aware augmented reality: Any mixed reality where text or computer-generated objects appear to attach or interact with real-world objects. This can range from simple applications, such as the Sphero BB8 toy robot smartphone app, to complex systems, including Hololens, Magic Leap and others.
The most thrilling mixed reality experience involves real-time, 3D mapping of the environment, which enables virtual objects to interact with surfaces and objects in the real world. For example, a computer-generated creature that can stand on a table -- or hide behind it.
Here's the challenge: 3D-mapping capability is compute-intensive, meaning it is expensive, power-hungry and heat-generating. So is the real-time rendering of 3D objects.
With the richest mixed-reality experience, you need both 3D mapping and image rendering. As a result, products like Hololens and Magic Leap require heavy, bulky headsets and cost a lot of money. (The Hololens developers edition weighs more than 1.25 lbs. and costs at least $3,000 -- roughly the same weight and price as four iPhones.)
This class of mixed-reality product is exciting, but these systems aren't for casual use. I'm sorry to break it to you, but only businesses, universities, military operations and very dedicated gamers will buy them in the near future.
And that's where Google and Apple come in.

Google

I told you about Google's Tango 3D mapping technology more than a year ago. The platform is capable of quickly mapping interior spaces, including depths and distances; it's even capable of accurately determining the size of objects in a room.
Since then, the first Tango-capable phone has shipped (Lenovo’s Phab 2 Pro) and a second has been demonstrated (the Asus ZenFone AR).
Dozens of Tango-supporting apps are now available, and there should be hundreds by the end of the year.
The American Museum of Natural History is using Tango to bring back the dinosaurs, as demonstrated at the most recent Google I/O developer conference.
BMW is using Tango to sell cars.
And MIT researchers are using Tango to create virtual-reality spaces based on real-life spaces. This is a nascent approach to virtual and mixed reality where such environments could be created on the fly using low-cost phones and headsets.
Some of the most interesting Tango apps are for shopping, such as Home AR Designer, which lets you see virtual furniture in place in your living room before you buy.
The best Tango products are yet to come. These will appear in the form of new phones, new apps and new peripherals, such as mixed-reality glasses that work with a smartphone.
What's amazing about Tango is that Google has already demonstrated sophisticated mixed-reality-capable mapping that's inexpensive enough to be offered in a $500 smartphone.
The ZenFone supports not only Tango, but also Google's Daydream VR headset -- but not at the same time. I spoke with Tango chief Johnny Lee this week, and he told me that they can theoretically work together now, but not well enough for public use. Expect the union of Tango and Daydream support (as well as support for lighter mixed-reality glasses) in future Android phones.
Lee also said that the Tango team has frequent meetings with Google's Pixel smartphone group. He said they're well-versed on all aspects of Tango and that there's "a lot of interest and enthusiasm" as Tango technology gets smaller and cheaper. I think it makes enormous sense for future Pixel phones to get Tango functionality built in.
Tango can harvest 3D data the could be useful in the future for identifying objects, which would provide artificial-intelligence-based augmented reality, Lee said. That means Tango itself could be a major tool for identifying objects, buildings, appliances and products and labeling or providing additional information about them in your field of view. Everyday use of this feature could enable Google to crowdsource data on everything in real time, much as Google's Waze app maps traffic.
The most important thing Google and Tango are doing is bootstrapping a smartphone-based mixed-reality ecosystem. This wide effort is happening behind the scenes and encompasses the development of smartphones, peripherals, apps, content -- even a "large library of 3D assets that you can place in the world," according to Lee.
Google is building a world of smartphone-friendly mixed reality that's mainstream, portable, affordable and practical.
Apple may be doing the rough equivalent

Apple

During a visit to the U.K. last week, Apple CEO Tim Cook told The Independent that he is "excited about augmented reality" (AR). (Cook uses the term "augmented reality," but "mixed reality" is more accurate.)
That's interesting, because Cook rarely uses the e-word -- excited -- unless he's talking about Apple products.
(Apple rarely discusses future product plans, and the company declined to comment for this column.)
Goldman Sachs analyst Simona Jankowski and KGI Securities analyst Ming-Chi Kuo both believe Apple's 10-year anniversary iPhone 8, will support mixed reality.
Combine all this tea-leaf reading, guesswork and rumor-mongering with Apple's numerous mixed-reality patents and acquisitions, and it all adds up to a strong possibility that Apple could embrace mixed reality in commercially available products. Chief among Apple's acquisitions is Primesense, the company behind the hardware Microsoft based its Kinect system on.
If Apple does offer something in the mixed-reality space, it will probably be for the mainstream consumer market. Cook told The Independent that mixed reality is a "big idea, like the smartphone" — something "for everyone."
I've predicted in this space that Apple will get into the smart glasses business. I wouldn't be surprised if Apple's first foray into mixed reality was a simple HUD display built into fashion glasses, a 3D-image sensing feature for the iPhone 8, or both.
Either way, when you think about mixed reality, understand that while Microsoft, Magic Leap and others are creating powerful, exciting mixed-reality products, they lack the one quality that Google and probably Apple have -- the intention to develop mixed reality that's affordable, mobile and mainstream.
The mixed reality you'll actually use isn't the heavy, expensive, nonmobile version from Microsoft and Magic Leap. It's the one that comes free with your phone.