How Augmented Reality is Revolutionizing Museums, Schools and Jobs
While still at its infancy, AR is dramatically changing the way we learn, get inspired and train.
Barcelona, 2015. As I queue to visit Casa Batlló, one of the houses designed by Antoni Gaudí, I’m given not an audio-guide but a much heavier and chunky video one. It instantly sparks my curiosity.
There are two ways in which the video-guide device can be used: To see rooms as they were intended by Gaudi, and to explore a scale model that shows the halls filled with tiny digital inhabitants instead of the louder, bigger tourists that surround me.
Gary Gautier, Director General at Casa Batlló, affirms that since the videoguide was launched in 2014, visitors have had nothing but good feedback:
The experience is absolutely positive because thanks to it, we can offer the visitors a tour that escapes from purely visual aspects and turns into an experience for the senses due to the use of the augmented reality.
The way the guide works is as follows: When you enter a room (and there are over five floors of them,) you can lift your device and play an video that overlaps with what you are seeing and lets you explore the boundless imagination of the renown architect. The rooms get quickly filled with symbolism, such as swimming/flying fish and colored organic animations intertwined with the exquisite furniture. It all alludes to the functional and aesthetic brilliance of the building, and it provides a dynamic context that help you appreciate the pieces and the architectural decisions even more.
Casa Batlló is one example of the many augmented reality initiatives that are being installed in museums, galleries and shows. The industry of AR is a blooming one: It’s actually expected to produce more revenue than VR, and to increase even faster than it in the next years (Digi-Capital.)
The starting point for VR is a tabla rasa, a canvas to fill with whatever the creator wishes. Designing for AR, though, means being pinned to this very ground we walk, for it combines virtual and real images, in 3D and in real time. And as such, it is pebbled with challenges — and potential.
What makes for a good AR experience?
Let’s quickly revise what makes an AR experience good. Here are its three essential elements. It should:
- Be engaging and compelling;
- Be intuitive and easy to use, based on physical and virtual affordances; and
- Be anchored in the real world, and combine it with the digital.
Virtual and augmented reality are great tools for enhancing storytelling. The projects that can benefit the most from them are projects where an immersive experience can enhance the content, the story, as the digital is layered onto the existing world. Environment and story should complement each other.
But here’s the catch: First-person experiences are usually non-lineal and non-narrative. How do we set ourselves for success?
A note on Calm Technology
A concept that comes handy is the one of calm technology, which states that the interaction between the technology and its user is designed to occur in the user’s periphery rather than constantly at the center of attention. (Wikipedia).
The periphery should inform without overwhelming. This is particularly important in museums and similar spaces, where one’s attention tends to wander towards the objects that spark curiosity or awe.
The right amount of technology is the minimum needed to solve the problem. — Principles of calm technology, CalmTech
Think of a tea kettle that does its job well, letting us know only when the water is ready. A tea kettle can be forgotten until it sings. And when it sings, oh, when it sings!
And now for some examples!
The Smithsonian: Bone Hall
The Bone Hall is one of the few museum exhibits that can proud itself in having survived across three centuries of visitors gazing upon them. This grand comparative anatomy in the Smithsonian National Museum of Natural History still mirrors the selection of skeletons used in the original exhibit, opened in 1881.
Now thanks to AR, visitors can superimpose a virtual world onto the physical one. The app has a special way to ‘see’ skeletons on display in the Bone Hall, or printed on paper and on a monitor. It lets you peer into the inner lives of animals like bats, giant sea cows, rattlesnakes and more, meet the people who study them, ponder big ideas about the natural world and play a game or two. The best feature, reviewers seem to agree, is putting skin on the skeletons.
The kids actually jumped when they saw the Mandrin! — Bone Hall dad user.
Cleveland Museum of Art: ArtLens
Cleveland Museum of Art is using both award winning augmented reality and Kinect natural motion technology to let visitors interact with and create immersive art experiences.
On the one hand we have the app, called ArtLens 2.0. It uses image-recognition software to scan and recognize artwork, and it then provides additional content of the piece. Galleries in the museum are also color-coded and grouped thematically to facilitate easier navigation and easier identification of areas of interest.
Every single item on display is included in ArtLens, so you just need to scan and inspect. If information needs to be updated, it’s done and distributed automatically.
A second installation completes the experience: Studio Play, the creative process of making the art via customized Kinect technology. 3D Kinect depth cameras and sensor technology augment and enhance the visitors’ experience. They create original artwork by using their own hands to manipulate clay in 3D space and/or throw virtual paint onto a 4K digital canvas.
ArtLens was launched in two phases, the second one in June 2017. Several improvements have reduced the download and loading size, improved the interface and enhanced the synchronization.
Google Classroom
In June, Google announced the opening of its Classroom services, expanding Google Expeditions to include augmented reality. One of these expeditions sent, in 2015, two million participants in virtual school trips to learn about buildings and places around the world.
In mid-May, Google presented Expeditions AR. The use of augmented reality in Google Expeditions will be tested at pilot schools from autumn 2017.
Virtual reality makes it possible for a teacher to offer learning experiences that are engaging, creative and immersive. We have the ability to connect reality and digital content, to access continuous background information through a smartphone.
Some of the things you can do with augmented reality in the classroom are adding a short biography about a person, using geolocation of historical events, creating visual models for math and recreating historical places.
The use of AR in education opens a whole lot of questions, too. Is google setting the tone for the future of digital education? Will AR lessons replace the ones we have? And if they are to be, should they be open, and available in public libraries?
What do Surgeons and Elevators Technicians Have in Common?
“There is no excuse today for the surgeon to learn on the patient” — William J. Mayo, 1927.
For a long time, in medicine, this was the only way to learn. Luckily for doctors and patients, we now have the ability to integrate visualization, computing, performance measures and simulated procedures.
AR can help surgeons become more efficient at surgeries, as well as train them safely without having to use actual humans.
For example, though AR an accurate 3-dimensional reconstruction of the body can be created, empowering surgeons with a sort of x-ray vision (in real time, and without radiation!)
One of the barriers to developing virtual and augmented reality surgical simulation has been the large amount of computing capacity required to remove delays in signal processing. Also fortunately for us, systems that break down tasks are addressing the issue, and as A.I. gets more and more intelligent, we are certainly heading towards a healthcare revolution. Imagine what will happen once quantum computing is here!
So, what do surgeons and elevator technicians have in common? They can both benefit enormously from AR.
ThyssenKrupp, the big elevator company, is trying out the HoloLens with its elevator maintenance teams. Techinicians can use AR to dig into the problems of thousands of different configurations and millions of parts that make the elevators they maintain. And they can do it more quickly and safely.
With AR, you are looking through the headset at the real thing, augmented with additional information. There’s no substitute for learning on the spot. AR puts the equipment in the hands of the person.
On-the-job training can be taken to a whole new level thanks to AR. Imagine a new hire faced with in real-world situations in which they must perform their job duties. Millions of new employees could be trained using these technologies. Seems like hyper-training, for good or bad, is here to stay.
VR at its best shouldn’t replace real life, just modify it, giving us access to so much just out of reach physically, economically. If you can dream it, VR can make it. — MATTHEW SCHNIPPER, “Seeing Is Believing: The State of Virtual Reality”
Augmented reality is truly revolutionizing the way we interact with information, with our history and the world around us. Its availability makes it the perfect tool for small-scale grandeur.
I’m anxious to see what the future might bring, and as happy as hell that I got to live in this time.
