How charities could soon harness the immersive world of ‘mixed reality’

First virtual reality, then augmented reality, and now mixed reality is all the hype – but what’s it all about, and what could it have to do with charities?

Chloe Green | 23rd Aug 19
Image from HoloLens, Microsoft's mixed reality headset Image credit: Microsoft

Back in July, Microsoft opened its first physical store in the UK on London’s Oxford Street. Among the technology products and services being demoed to the public was Microsoft’s HoloLens – a ‘mixed reality’ headset that Microsoft says could have applications in a wide range of industries.

But what exactly is mixed reality and how does it different from virtual reality and augmented reality?

First of all, a quick look at what virtual and augmented reality are. Virtual reality, or VR, involves wearing a head-mounted display to interact with an entirely similated 3D environment, in the form of a game or video. It’s been a hyped technology since at least the 1990s but only recently has it become available and affordable on the consumer market in the form of Facebook’s Oculus Rift, Sony’s Project Morpheus and the Samsung Gear VR. You can even plug VR into a mobile phone using apps and a low-cost headset such as Google Cardboard, and the slightly more sophisticated Google Daydream View.

> See also: Five ways charities are using virtual reality

Charities are beginning to explore how Virtual Reality can bring people closer to their cause, such as the recent Alzheimer Research UK’s app that lets people experience life through the eyes of a person with dementia, or the National Autistic Society’s immersive experience that puts people in the shoes of someone with autism.

It’s also becoming a popular tool in fundraising: charity: water took 400 people on a VR trip to East Africa at an annual black tie fundraising banquet at the Metropolitan Museum of Art, raising over $2.4 million.

 

AR

Augmented reality, on the other hand, overlays computerised elements onto a real-world environment. While VR completely covers and replaces your field of vision, AR apps are only viewable through your smartphone or tablet screen, integrating digital information with real-world objects. The best-known example in recent years is probably the Pokemon Go game which saw people running around in the real world chasing virtual creatures by holding their phone up to a certain location.

Charities have been experimenting with augmented reality apps over the last few years. In 2015, for example, Oxfam kicked-off a campaign with an augmented messaging app called Traces that allowed people to engage digitally with the charity by placing an AR portal at over 650 UK retail locations. And in 2018 the Royal British Legion launched an AR-based campaign that brings stories from the Battle of Passchendaele to life in a bid to engage with a younger audience.

Most recently, WWF combined mirrors with augmented reality to bring the issue of sustainable fashion home to Wimbledon fans.

 

MR

Mixed reality (MR) combines elements of virtual reality and augmented reality, blending the physical world with the digital, and VR and AR capabilities together. Users wear a headset, such as in VR, to interact with an environment where physical and digital objects coexist. MR programming allows digital objects to interact with physical objects and people to interact with digital objects as if they are physical.

Microsoft’s HoloLens, for example combines eye tracking, voice recognition and 3D display, combining data and artificial intelligence hosted in the cloud.

MR might sound difficult to understand, but picture this: a construction engineer who is able to have ‘x-ray vision’ of a building by overlaying holographic blueprints on top of the real world; A mixed reality interior design app that allows a designer to paint real walls, countertops or floors in different colors and patterns through a holographic overlay, or insert objects at real scale into a shop floor to visualise what it will look like; A mixed reality app that allows an automotive designer to layer new design iterations for an upcoming car refresh on top of an existing car; An aircraft engineer who can view the schematics of the inside of a jet engine they are fixing, without having to remove the engine.

Philips is piloting HoloLens in a healthcare setting, allowing doctors to view live x-ray, ultrasound and other information so that they can “see” inside a patient in real-time. By integrating and displaying this rich 2D and 3D data in mixed reality, combined with intuitive gestures, eye-tracking, and voice control, the healthcare pracitioner can increase their focus on the patient and the procedure.

> See also: Augmented reality app to drive charitable fundraising

Microsoft says: “Mixed reality is the next evolution in human, computer, and environment interaction and unlocks possibilities that before now were restricted to our imaginations.”

 

MR for charities

The headsets themselves are beginning to come down in price, but the current price of MR software is likely to mean in the short term it’s mainly targeted at high value engineering applications. In due course, however, these technologies will inevitably trickle down to the charity sector.

Jonathan Chevallier, Charity Digital CEO recently visited the Microsoft Store in London, and his verdict was: “I can see it becoming more mainstream. It can do impressive things like scroll a section of text as you read it (automatically), respond to voice commands and enable you to push a button you can see on the headset display. I can foresee applications for the disabled, those with learning disabilities and many other areas.”

Much like VR and AR, there is plenty of room for new ways to bring causes to life for potential donors and supporters and stand out from the crowd. But the real game-changing applications could lie in service delivery.

For example, an MR system could allow service users with limited mobility access to services, games, work environments and the internet, through the use of their eye movements and voice commands.

In an education setting, MR could make it easier to learn subjects by making them interactive and engaging through digital overlays – something which could be particularly transformative for people with learning disabilities.

We are about to see an explosion in MR products that will bring many new options. Apple just announced a new patent for its mixed reality technology, which includes enhances facial tracking with lots of potential for the mobility limited, whereas vendors like Adobe are working on bringing digital creativity into the real world with its new suite of apps – watch this space.