The Medical Futurist | 9 min | 14 November 2019
Augmented reality is one of the most promising digital health technologies at present – look at the success of Pokémon Go – and it has the potential to change healthcare and everyday medicine completely for physicians and patients alike.
By now, it is official: Pokémon Go conquered the world. TechCrunch reported that on the day when the game was launched, it immediately surpassed the daily time usage of Facebook, SnapChat or Twitter by the average iOS user on mobile phones. Tom Curry, a man living in New Zealand quit his job to become a full-time Pokémon hunter. In Central Park, herds of Pokémon Go players almost caused a stampede as they tried to capture a rare type of the imaginary animal.
Rafael Grossmann, the first surgeon who performed an operation with the help of Google Glass, told me that Pokémon Go represents the ultimate gamification of an “activity” app, and that he does not think the inventors of the game such as Nintendo expected nor planned this effect in people.
AR has a place in the future of healthcare, just read our interview with mixed reality expert, Robert Scoble. But now, let’s see what is already possible today.
So why is the game so popular and what does it have to do with the future of medicine?
The response is augmented reality (AR) and the rising interest of people in its use. Pokémon Go is made with exactly this technology: the device (in this case your phone) transmits a live or indirect view of a physical, real-world environment which is augmented by computer-generated sensory input such as sound, video, graphics or GPS data. In the future, augmented reality could be a built-in feature in a glass, headset or digital contact lens.
Augmented reality differs from its most known “relative”, virtual reality (VR) since the latter creates a 3D world completely detaching the user from reality. There are two respects in which AR is unique: users do not lose touch with reality and it puts information into eyesight as fast as possible. These distinctive features enable AR to become a driving force in the future of medicine.
The best examples of augmented reality in medicine and healthcare
1) Augmented reality can save lives through showing defibrillators nearby
What would you do if a person next to you collapsed suddenly? All kinds of thoughts would rush through your head, and no matter whether you would think of calling an ambulance, a doctor or your mom for help, you would definitely reach for your phone.
And I suggest you consider downloading the Layar reality browser combined with AED4EU app to your phone next to the basic emergency numbers so the next time you get into a similar situation, you will be able to help more.
AED4EU was created by Lucien Engelen from the Radboud University Nijmegen Medical Centre, The Netherlands. Its users can add places where automated external defibrillators or AEDs are located and this database can be accessed through this new application. Moreover, with the Layar browser, you can project the exact location of the nearest AEDs on the screen of your phone and it would take a minute to find them and help those in need. So augmented reality brings crucial pieces of information to those in need or danger.
2) Google Glass might help new mothers struggling with breastfeeding
While Google Glass couldn’t conquer the world, it was used to help new mothers with breastfeeding.
The Melbourne office of an innovation company called Small World conducted a Google Glass trial with the Australian Breastfeeding Association that effectively allowed their telephone counsellors to see through the eyes of mothers while they breastfed at home. Through such a way struggling mothers could get expert help at any time of the day and they did not even have to put down the baby from their arms. By sharing the patient’s perspective, consultations get to a new level.
3) Patients can describe their symptoms better through augmented reality
Patients often struggle when they have to describe their symptoms to their doctors accurately. In other cases, people often find themselves overreacting a medical situation or on the contrary, belittle the problem. In ophthalmology, augmented reality might be the answer for patient education.
EyeDecide is one of its kind medical app which uses the camera display for simulating the impact of specific conditions on a person’s vision. Using apps like EyeDecide, doctors can show simulation of the vision of a patient suffering from a specific condition. For instance, the app can demonstrate the impact of Cataract or AMD and thus helping patients understand their symptoms and their actual medical state. If patients can experience the long-term effects of their lifestyle on their health, it could motivate people to make positive changes.
4) Nurses can find veins easier with augmented reality
The start-up company AccuVein is using AR technology to make both nurses’ and patients’ lives easier. AccuVein’s marketing specialist, Vinny Luciano said 40% of IVs (intravenous injections) miss the vein on the first stick, with the numbers getting worse for children and the elderly.
AccuVein uses augmented reality by using a handheld scanner that projects over skin and shows nurses and doctors where veins are in the patients’ bodies. Luciano estimates that it’s been used on more than 10 million patients, making finding a vein on the first stick 3.5x more likely. Such technologies could assist healthcare professionals and extend their skills.
5) Motivating runners through zombies
Imagine that you are walking through a dark and abandoned alley, and you suddenly hear the groaning and the slow movement of a strange creature. I am pretty sure that even the laziest person would speed up after realizing that a “real zombie” is after him. This is the basic idea behind the Zombies, Run! application.
The game monopolizes on the fact that fear can motivate people and the fact that everything seems to be more fun when turned into a game. This app is perfect for those who consider running a boring activity. If you not only hear but also see virtual zombies projected onto your phone or device’s screen, you will not only increase your speed and endurance, but also feel that time is just flying by.
6) Pharma companies can provide more innovative drug information
Have you ever been curious about how a drug works in your body? Even if you got interested in discovering how the distant world of pills and medicaments work, I bet you lost all your enthusiasm after you read the boring and indecipherable drug description. Now, augmented reality is here to change it.
With the help of AR, patients can see how the drug works in 3D in front of their eyes instead of just reading long descriptions on the bottle. Lab workers could monitor their experiments with augmented reality equipment. In pharmaceutical factories, workers could start working without hands on trainings as the device would tell them what to do, and how to do it.
7) Augmented reality can assist surgeons in the OR
Doctors and even patients are aware of the fact that when it comes to surgery, precision is of prime importance. Now, AR can help surgeons become more efficient at surgeries. Whether they are conducting a minimally invasive procedure or locating a tumor in liver, AR healthcare apps can help save lives and treat patients seamlessly.
Medsights Tech developed a software to test the feasibility of using augmented reality to create accurate 3-dimensional reconstructions of tumors. The complex image reconstructing technology basically empowers surgeons with x-ray views – without any radiation exposure, in real time.
The earlier mentioned Grossmann, who was part of the team performing the first live operation using medical VR, told me that HoloAnatomy, which is using HoloLens to display real data-anatomical models, is a wonderful and rather intuitive use of AR having obvious advantages over traditional methods.
8) Hololens changing medical education and the study of anatomy
Case Western Reserve University and the Cleveland Clinic have partnered with Microsoft to release a HoloLens app called HoloAnatomy to visualize the human body in an easy and spectacular way.
With Microsoft’s HoloLens Headset, app users are able to see everything from muscles to the tiniest veins before their eyes on a dynamic holographic model. I believe it will revolutionize medical education, as students will be able to see the human body in 3D instead of the usual working method: black-and-white pictures and written descriptions in books.
Although Microsoft Hololens is far from the “perfect” experience, there is no reason to believe that we will not get there soon. Thus, the most effective way to get used to this future trend is starting to educate ourselves and our children.
9) Teaching kids about the human body
The UK-based company, Curiscope creates immersive learning experiences in virtual and augmented reality. They developed the Virtuali-tee T-shirt, through which you can see the inner parts of the human body through realistic holograms. Virtuali-tee brings anatomy to life to such a point of illusion which already represents Arthur C. Clarke’s famous third law. “Any sufficiently advanced technology is indistinguishable from magic.”
Do you remember which your favorite toy as a kid was? For example, I always had a passion for LEGO. Assembling little LEGO-parts into something new, creating castles, cars, complex cities – that is one of the best activities in the world. It stimulates your fantasy, your creativity, develops your skills for holistic vision as well as your attention to detail. Lately, there have been various videogames which attempt to recreate LEGO in the virtual space – such as Minecraft.
Parents often complain that their kids are just sitting in front of some screens, not learning anything about their environment and themselves, but I do not agree. Minecraft also enhances creativity, develops the way children see the world around them – but in a different way as LEGO. I think that from here, it is only one leap before we reach LEGO with AR, where the advantages of building something in the real world might be combined with virtual imagination.
This way, our kids would be able to know that real is real, but would also be ready to exploit the opportunities AR can provide us with.