7

Vision Five ‒ Virtual Healthcare

As we imagine a world where every doctor, nurse, administrator, and patient wears Spatial Computing glasses, we see five changes: the medical system becomes more efficient, existing patient care gets better, those who provide care are better trained, the patient experience improves, and healthcare becomes more predictive and personalized.

Spatial Computing will eventually redefine what we think of as healthcare and the medical system that delivers it. Every part of it will be touched, from the surgery rooms to the doctors who save your life. They already are starting to use devices like Microsoft HoloLens 2 to prepare for or perform surgeries.

Nurses will be assisted by robots bringing them drugs, needles, or other supplies without forcing them to walk down the hall to gather them. All of this directed by glasses flashing visuals about their job process and their patients into their eyes, and audio into their ears providing information and commands.

Patients will see their waiting room experience transformed thanks to VR, not to mention they might not need to wait at all, thanks to new kinds of telemedicine that they will be able to get from home or work. What we think of as healthcare might turn into a new kind of virtual coach, that is, a virtual being that helps you make better choices in life, for example, directing workouts so you meet your fitness and health goals.

Administrators will finally see efficiencies from the move to electronic health records, and will see nice returns on investment on systems to actually change patient behavior, getting them to exercise more and eat better, thanks to games on their glasses.

We will have a new understanding of healthcare, one that's more predictive and can anticipate your healthcare needs based upon your current lifestyle, all thanks to devices you wear, much like today's Fitbits and Apple Watches. This will bring us a world where healthcare is more readily available, much improved, and cheaper to afford. This will be a particularly radical change for the world's poor, in contrast to their current experience of healthcare.

Although this pipe dream is probably a decade away, our discussions with doctors, administrators, and others demonstrate that R&D is already well underway to bring about a new kind of healthcare, which will bring us new techniques that early studies show is even changing our ideas of how pain will be dealt with. Someday soon, instead of being prescribed opioids or other types of prescription drugs to tamp down your pain, researchers see a new kind of practitioner prescribing that you use VR instead. This may seem an odd way to relieve pain, but we'll describe how promising results are already being observed for pain relief in the Brain Tricks/Virtual Medicine section of this chapter.

Even better, we are about to head into a world where the healthcare system isn't reactive like it is today. Today, we mostly go to a doctor because we aren't feeling well, or to get annual checkups, which mostly include blood workups, weight and blood pressure checks, and some other simple procedures to look over your body. Worse, hospitals primarily deal with patients who are having heart attacks or other health failures. Many of the researchers we talked with see a new, predictive healthcare that will see problems in advance of a major issue, or even work to get you to live in a more healthy way, which would save many dollars and lead to better outcomes for everyone.

If your glasses say "hey, you need to get to a hospital now," that is a far better situation than having to be brought to a hospital by an ambulance later, which costs thousands of dollars and may already be too late. Even in a situation where you might need surgery, the doctors will have your heartbeat and other data from the past few weeks available, allowing them to see much more of your history than if they were to hook you up to monitoring equipment and have to guess what's been going on based on a far smaller amount of data.

With all of these highly significant applications for Spatial Computing in healthcare, we will now give you a sweeping overview of what changes we can expect to see in the coming years, based upon our research and discussions with medical professionals.

The Virtualist

Dr. Sherylle Calder in South Africa is, in our eyes, the prototype of a new kind of medical practitioner who works with Spatial Computing technologies to improve her patient's lives―one that Dr. Brennan Spiegel, director of health research at Cedars-Sinai, calls "the Virtualist." Spiegel cofounded a conference that deals with the changes coming to healthcare thanks to Spatial Computing and is writing a book of his own where he details this new kind of specialty.

Dr. Calder is an eye doctor who has found a new way to improve her patients' lives by manipulating their eyes―all with digital patterns on screens. She is the first sports scientist to be awarded a doctorate in visual motor performance and works with professional athletes, and now everyday people, thanks to the company she started: EyeGym.com. The aim of her product is to improve her patients' eyes and the part of the human brain that deals with visual input from them. She does her work with patients completely on digital screens and has had remarkable results. The owner of the South African cycling team told us that she took its worst rider, in terms of falls per race, and, after spending a few minutes a day with Calder's system, the rider went from the worst to best within a few months. Her digital system fixed a flaw in his brain that caused him to fall during races. While not purely "spatial," we believe her system demonstrates how computing can be used to heal the human brain and we will see applications like this greatly expanded as more people wear computing on their eyes.

She explained that more than 30 percent of our brains have to do with visual processing and that she found ways to identify flaws in that system and "exercise them" with patterns and colors on screens. She's the first doctor we saw using a completely digital approach to medicine, which is why we see her work as being groundbreaking. Working with the 2003 England Rugby World Cup team, her system brought huge changes. They ended up winning that year in the final against Australia. She also told us she worked with Ernie Els during a low period of his golf career, and her work helped him to win a major within six months.

Her story of finding ways to improve human brains with digital screens was verified in our discussions with many doctors who are doing the same to find ways to provide relief to those who suffer from depression, pain, autism, dementia, PTSD, and other maladies, and we see her and her work as a prototype of this new kind of specialist.

She won't be alone―several doctors and medical researchers have told us. If you walk through a modern hospital, you will see dozens of specialists, from radiologists to pediatric surgeons. Dr. Spiegel says that to understand this new field that is in its beginnings, we have to see the Spatial Computing R&D being done in surgery rooms, waiting rooms, burn recovery centers, mental health clinics, and other places in the healthcare system.

Photo credit: Robert Scoble. Dr. Sherylle Calder, left, in her South African office, explaining the work she does to improve professional athletes' perception systems.

For instance, UCSF Benioff Children's Hospital uses VR to do patient consulting with children who have brain tumors, which improves the patient's experience. That kind of usage gets patients to relax. There, Dr. Kurtis Auguste, chief of the department of surgery at UCSF Benioff Children's Hospital Oakland, gives children and their parents a fly-through of their brains. He can zoom in on their tumors and the nerves and blood vessels surrounding them and explains whatever procedure he will use on them. This gives the patent much more detail than can be provided by a standard MRI or CT scan.

"This technology is unlike any visual platform I've ever experienced―it is an actual immersive experience," Auguste reports on the UCSF website. "I can use this to plan a surgery and determine the best roadmap to a target in a way I never could before." The system he uses was created by Surgical Theater, Ohio.

It also helps him see things that he would have missed with earlier technology. It's a win-win all the way around and that visualization pre-surgery is bringing us this new kind of "Virtualist" specialist. We will see if the term sticks, but the healthcare practitioners we talked to agree with him that VR and AR are showing up in many places inside hospitals and they can see a role for doctors and nurses who work only with virtual tools.

These "Virtualists" are the ones setting up 3D sensors and putting HoloLens and other wearable technology into surgery rooms, and the ones who are using VR to relieve pain in burn clinics and using VR and AR to treat phobias, phantom limb pain, eating disorders, and other things. These include surgeons like Dr. Shafi Ahmed, who uses a variety of devices to teach surgery and enable new kinds of surgery to be done.

Dr. Ahmed, who the press has anointed "the world's most-watched surgeon" due to his broadcasting of surgeries on YouTube and other social media, told us about his work and the work of others who are leading the way to democratize surgery and healthcare by making better education available to surgeons. He wears devices like HoloLens during surgery, which lets him share that with others, but he says the changes coming are far deeper.

Other places "the Virtualist" will see jobs is in training and putting new brain research into use, and looking for ways to get efficiencies out of the healthcare system as more Spatial Computing glasses, along with AI running it all, and robots, are brought into hospitals and labs.

Technology coming from companies like MediView, Dr. Ahmed says, will let surgeons actually plan for and perform surgery in whole new ways.

Fixing Humans – Surgery Preparation and Surgery

The crown jewel of many hospitals is its trauma center. Here, patients arrive with various injuries or illnesses, and many are quickly operated on in high-tech surgery rooms. Dozens of machines, with teams of doctors who run around working on saving lives, which is very similar to what you see on various TV shows like Chicago Med.

What they don't show you on TV is the preparation time that surgeons go through to prepare for complex procedures. That used to be done by calling other surgeons who might have dealt with a similar condition. That led to incomplete learning, and no way to practice a procedure over and over until the surgeon gets it right.

Today, surgeons like Dr. Shafi Ahmed are using VR to view the data coming in from MRIs and other scanning systems, planning out their complex moves, and then practicing them on 3D printed "virtual" organs with either VR or AR glasses before they get into the surgery room. In this way, they can call in other surgeons and ask for help learning a complex move that they might never have done before and they can practice it over and over until they get it right.

Then, when they move into the surgery room, new systems are showing them the scanned data wrapped right onto, and in some cases into, the patient's body. One system from MediView in Boston even works like a guidance missile system. It tells the surgeon if they have their cutting tool on the right tumor and at the right depth with a series of video and audio prompts that are showing up the lenses of a Microsoft HoloLens.

A company in Ohio, Surgical Theater, too, gives surgeons a very detailed view of, say, a patient's brain, and the surgeon can plan out a path to a tumor, or a group of cells causing epileptic seizures, and then show the plan to the patient and their family.

Photo credit: Edward Miller. Dr. Shafi Ahmed using VR to prepare for surgery.

In addition, using VR for surgery preparation and during the surgery itself, Dr. Ahmed has made a name for himself by using VR to train other surgeons. "When I train, what other surgeons need to know very quickly is depth perception," he says, while extolling its benefits for showing many other budding surgeons how to do a new procedure. Getting a cutting tool into the right place inside the human body often takes great skill and that learning used to be passed along surgeon to surgeon. "This takes years and years of experience. What Spatial Computing allows us to do is see the depth." This changes how the medical system can share techniques and the benefits go way beyond surgery. As this technology comes down in cost, you'll see Spatial Computing surgery spread throughout the hospital system. Why? Dr. Ahmed says that the secret sauce is that VR can improve retention of learning by 80 percent.

Dr. Daniel Kraft, who is the founder and chair of exponential medicine at Singularity University, tells of how he uses tools like OssoVR to train other doctors (he has a long list of accomplishments and was the first to implement a text-paging system at Stanford University's hospital). He uses it along with Precision OS, Surgical Theater, and FundamentalVR. Surgical Theater lets him plan surgeries. Precision OS and FundamentalVR let him practice it. Osso lets him teach it to others. Currently, a small but growing raft of companies, including Boston Scientific, EchoPixel, and Medivis, provide similar capabilities for the planning of different kinds of surgeries.

Further, there is at least one company that goes way beyond visualization, collaboration, or simulation―to actually helping the surgeon do the surgery itself: MediView.

Photo credit: Cleveland Clinic. Dr. Charles Martin, III, Head of Interventional Oncology at Cleveland Clinic, uses a Microsoft HoloLens to overlay AR images on a patient during surgery.

MediView's technology is like a missile guidance system for surgeons. It joins data from MRIs and ultrasounds and overlays that in three dimensions on a patient's body to show where a cancer tumor is. Then it even guides the surgeon's cutting tools toward the tumor and tells them when they are in the right place to cut it out. This technology will even save the life of the surgeon. MediView's founder told us he lost a partner to cancer caused by ionizing radiation used to locate tumors with older technology and that he knows of lots of surgeons who have gotten cancer from the older techniques, which is what drove him to find a better way to figure out how to visualize tumors on the surgery table. MediView has already passed through one patient trial of five patients and is about to go through another one.

Additionally, MediView is focusing its AR surgery system for use on soft tissue surgery, which is a much harder thing to accomplish than, let's say, a surgery that focuses on the bone that tends to stay stable and stationary during a procedure. Another company, SentiAR, uses AR to overlay 3D images of a patient's actual anatomy during interventional procedures that deal with the treatment and analysis of cardiac arrhythmias, so it does deal with soft tissue procedures, but its system is currently utilized for this narrow use only.

Philips' Azurion Platform concept utilizing the HoloLens 2 was announced in February 2019; its focus is on using AR for image-guided minimally invasive therapies, rather than for major surgeries. MediView and SentiAR, as well as HoloSurgical, are developing their platforms for eventual FDA submission. Philips' Azurion Platform is still in the concept stage.

The first related system that was cleared by the FDA was the Novarad OpenSight Augmented Reality system in October 2018; however, the system is for pre-surgical planning use only.

Other companies besides HoloSurgical with its ARAI system, which have a Spatial Computing system that focuses on surgery on hard tissue such as bone, are Augmedix (xvision spine [SVS] system) and Scopis (Holographic Navigation Platform).

What is common among most of these company surgery systems is the use of the HoloLens AR headset. Most of these companies will have moved forward to use the new HoloLens 2 headset, announced in February 2019, by the time you read this book. The HoloLens 2 offers much better functionality in comparison to the first HoloLens, unlocking far more potential benefit for pre-surgery and in-surgery Spatial Computing systems.

The improved functionality has to do with an increased field of view, a more ergonomic and balanced design, direct and natural hand manipulation of 3D images, improved natural language speech processing, and direct access to the cloud and AI services from Microsoft.

The lesson here is that Spatial Computing is already radically changing surgery, and will soon do the same for the rest of the hospital―and eventually for the entire system. For both pre-surgery planning and in-surgery procedures, the aim of the use of Spatial Computing is to reduce operating time, cost, and complications caused by suboptimal surgical execution. Let's look at the use of Spatial Computing another way―how it actually works with patient's brains to open up new treatment methods.

Brain Tricks/Virtual Medicine

Pain management is one piece of the healthcare puzzle in which Spatial Computing, particularly VR, is showing its most promising results. At the University of Washington, researchers working with Tom Furness and burn victims found that playing in a virtual snowfield was better at managing pain than morphine. You can read their studies at: http://www.vrpain.com. This and other discoveries, which we call performing "brain tricks," have deep implications for healthcare and show us a path to a new kind of healthcare that many can experience outside of a hospital, saving time and stress for patients, saving money for the healthcare system, and solving problems better than the drugs of old.

These new anti-pain brain tricks work through multiple methods―the two most understood are distraction and getting the brain to shut down the nerves delivering pain signals to the brain. Putting light into your eyes is a powerful thing, it turns out. "It is a photonic trojan horse," Dr. Brennan Spiegel says. "If the brain is anxious or upset, it wants to keep track of pain. If the brain is calm and relaxed, it doesn't have time to feel pain."

Dr. Spiegel explained to us how it works. Pain, say, from your finger touching a hot stove, travels from nerves in your finger along nerve highways up your arm and into your brain, passing through several "gates" or "handoffs from nerve to nerve." When those signals arrive at the brain, he says, it is possible to either distract yourself from those signals, or you can go further and get your brain to actually close the gates between your finger and brain so pain signals never arrive at all. A snowfield app mostly works through distraction.

When you are having fun throwing snowballs your brain's attention centers are more focused on the fun than the pain, but Spiegel is seeing several other pain-blocking effects. He says VR actually effects how serotonin gets used by your brain and nervous system.

Photo credit: Cedars-Sinai. Dr. Brennan Spiegel attends to a patient using VR to diminish pain.

People who play Beat Saber, a VR game, note that they go into an enjoyable "flow state" after playing for a while. That effect people experience, Dr. Spiegel says, is deeper than just distraction, and that VR works just like opioids: they block those nerve handoffs.

"Many people are not in pain even after the headset gets removed," Dr. Spiegel says. "That suggests there are other mechanisms than mere distraction at work. Engaging people in a flow-engaging activity is the goal." So, don't be surprised if your doctor prescribes a VR headset soon and asks you to play some games similar to Beat Saber!

This set of discoveries has a deep impact on not just the opioid crisis hitting America, but it provides hope that a whole range of mental issues can be fixed simply by having patients put on a headset.

"We are all searching for safer analgesics," Dr. Spiegel says, explaining that in the United States there are lots of research dollars chasing new ways to deal with pain that don't require using opioids, which are killing close to 40,000 people per year in the United States alone. Some companies that have VR apps that address pain management include AppliedVR, Psious, and Firsthand Technology.

Dr. Spiegel and other researchers say that their understanding of these brain tricks are far enough along that they are starting to ask questions about how to use VR with more patients. How do we pay for it? How do you keep the equipment clean? How do we offer a VR consultation service? How often do you need to use VR to treat various conditions? Who owns the equipment and software to make it happen? The field of VR pain management is developing into a standalone discipline. "VR works. This tool cuts across all of healthcare."

An example of another area where VR is being used in healthcare is dementia. With dementia, the "tape recorder" of the mind has stopped working. Dr. Spiegel says "They have lost their sense of self. That is disorienting and depressing." Often, dementia patients develop severe depression. Here, a different brain trick is used called reminiscent therapy. He explained that often he deals with patients that are barely coherent and are extremely depressed by their fractured mind. Pulling out a photo of a childhood memory reminds them of something before their memories stopped working, which makes it far easier to work with the patient to get them to further fix their brains. He says this process works better in VR, because you can quickly immerse a patient in childhood memories.

Another area for VR's use in healthcare is schizophrenia. Schizophrenics are hearing voices in their head. Most of us hear voices, but we understand them to be ourselves. People with this condition hear other voices that have them living in their own form of VR, Dr. Spiegel explains. "[These voices] aren't real. Ironically it takes Virtual Reality to access their virtual reality." Researchers have found that using avatar therapy helps to climb into the mind of the schizophrenic. Patients are shown recreated faces and minds so they can see their demon―and maybe turn that voice into something that helps them out. Dr. Spiegel says that they are seeing statistically significant results that are much better than the drug therapies currently being used.

Another place where the relaxation technique works is during childbirth. Mothers who wear VR see not just a sizeable decrease in pain but perceive that time moves faster, similar to how a Beat Saber player sometimes loses track of time. Tripp VR, a meditation app, is another good example of using the brain to enhance cognitive flow.

Relaxation isn't the only brain trick that comes with VR, though. We are seeing other researchers unlocking brain tricks that come to play with stroke victims, PTSD sufferers, and eating disorders.

In a visit to Stanford University's Virtual Human Interaction Lab, we saw how researchers are using brain tricks to help stroke victims and other people who have lost control of one of their limbs. Jeremy Bailenson and others are doing a ton of research on other VR effects, from environmental education to studying whether it can be used to increase empathy for others.

Inspired by Bailenson, Dr. Sook-Lei Liew, who is director of the Neural Plasticity and Neurorehabilitation Laboratory, also uses avatars to help stroke victims. They have worked for years on a study where they give stroke survivors who can't, say, move a limb, a virtual avatar controlled via EEG, or electroencephalography signals gathered by sensors placed on a patient's skin. She's found that can help their brains and nervous system to rewire, which would eventually lead them to moving that limb again.

In addition, Bailenson told us about studies that use VR to "mirror" a healthy limb onto their other limb and, often, patients regain some movement and control of the limb that previously didn't move. VR actually can cause new brain pathways to be built, it appears to researchers. This research is exciting for all sorts of other conditions, like PTSD or autism.

"It's very easy to fool the brain," says Dr. Albert "Skip" Rizzo, while explaining his work into using VR exposure therapy to treat PTSD. Dr. Rizzo is the director of Medical Virtual Reality at the Institute for Creative Technologies (ICT) at the University of Southern California. He takes patients into a traumatic event, like a simulation of a battle for a veteran who is suffering. He and clinicians can use that visual to trigger them to talk about what happened, and then they can treat it. He reports that VR brings remarkable results.

Photo credit: University of Southern California Institute for Creative Technologies (ICT). Dr. Skip Rizzo serves as the project leader for ICT's Bravemind project, which incorporates VR as a part of exposure therapy to provide relief for post-traumatic stress. A still from the VR experience is above.

Dr. Rizzo started the Medical Virtual Reality lab in 1995 after getting excited in the early 1990s about an experience that opened his eyes to the possibility of game-based rehabilitation. He found that a 22-year-old male who had injuries to his frontal lobe had improvements in cognition after playing video games.

VR could also help with autism. In autism, the cells that deal with the ability to correctly discern emotions, for instance, haven't formed properly. Spatial Computing can solve that problem with brute-force methods. Someone with autism might have glasses on in the future that simply say: "Your friend is sad." However, studies are underway to see if some of VR's brain tricks can be used to actually get patients to build new pathways to understand when their friend is sad even without being told via glasses. Dr. Rizzo is also working on reducing anxiety in people who have autism, with a combination of VR experiences and video. The primary scenario that is being used is that of a job interview, where patients can practice going through an interview over and over until they are confident and relaxed.

Another brain trick is feeding it some education or changing the perspective of a patient. This works particularly well on eating disorders. A little education in VR goes further than other approaches, several researchers told us. How can they get patients to eat less salt that would lower their blood pressure? It turns out that by showing patients how their body works in VR and what happens as salt begins to destroy their organs, it got a study group to eat less salt and saw an average seven-point drop in their blood pressure, Dr. Spiegel reported to us. Other studies are seeing an effect if food sizes are virtually made bigger or smaller. If your glasses make those Oreos look bigger, you will eat fewer of them.

Another method of dealing with eating disorders includes getting patients to see their bodies differently. Often when presented with someone who isn't eating enough, it is because the patient sees themselves as too fat, as what happens with someone who suffers from anorexia nervosa. Researchers are using one of the magic powers of VR: embodiment, which is where you can become something you aren't, either another person or, as Chris Milk, founder of Within, showed us, various animals. In the case of eating disorders, they take patients into the bodies of normal people so they can see that they are sensing their bodies improperly. This remaps their brains so that they no longer feel fat or skinny, and start eating better.

These brain tricks work with a whole range of conditions, Dr. Rizzo reports, from Parkinson's, where VR's brain-relaxing conditions help, to women trying to avoid birth pain. Dr. Melissa Wong, of Cedars-Sinai, uses them when performing difficult procedures on pregnant women. She showed attendees at the Virtual Medical conference how one procedure had women screaming in pain, but with VR on, the patient never winced and was more bothered by having the game she was playing being interrupted.

Additionally, researchers like Bailenson and doctors like Rizzo say they are seeing amazing results while the patient goes through rehabilitation too. VR gets these patients to not only move more, say stretching an arm that had been operated on, but do it more often. Many of these exercises are to be done at home and researchers say they are seeing better results because exercising in VR is more fun; something many VR users are discovering after they buy a VR headset and start playing games that require physical activity. Robert Long, a VR fan, reported on Reddit that he lost about 200 pounds mostly by playing Beat Saber.

Dr. Diane Gromala does chronic pain research at Simon Fraser University and found that adding in biofeedback loops improves the results even more. By changing the scene you are in, or the avatar you are interacting with in relation to, say, your heart rate, it lets you reduce pain levels to an even greater extent.

We tell you all these stories (which are only a small fraction of the good news being reported) to show that a new kind of healthcare is coming: one that uses Spatial Computing technology to solve a wide range of health issues. The techniques being discovered here could be delivered to patients as they sit at home, or even as they move around the world. How? We see a new kind of virtual coach that will help deliver this new "virtual" healthcare to you. Add that to the "Virtualist," who works inside the healthcare system, and you'll see healthcare administrators get excited as they see costs come down and fewer patients experiencing negative results due to addiction to opioids. This can only be a good thing.

We've seen how VR can be used to fix problems with the brain; now let's look at how it could be combined with other applications to give us a "check engine light" to further improve our health and healthcare.

Photo credit: Black Box VR. A new kind of gym you use while wearing a VR headset.

The Virtual Coach and the "Check Engine Light"

When Facebook founder and CEO Mark Zuckerberg goes running, he has a real coach. You see this coach running next to him in photos he uploads to Facebook. That coach pushes him and keeps him company on runs, and probably doubles as a security guard. Most of us can't afford such a coach, which is why so many of us join fitness classes at gyms. Now, however, a new virtual coach has emerged on our watches that cajoles us into doing more steps or performing mental health tasks like remembering to take a few deep breaths once in a while. You see that virtual coach in today's Apple Watch, for instance, as a circle of lines. It tells you that you haven't taken enough steps today, or it's time to take a break from sitting in front of a monitor.

When we've run with a real coach, that coach often rode a bike next to us, pushing us on runs, or getting us to do interval training up hills. A virtual coach could easily take over those roles and more.

This new virtual coach will appear in your Spatial Computing glasses. A virtual being, like Magic Leap's Mica, might run along with you, pushing you to keep ahead of last week's run pace. As of 2019, Mica only sits at a table and plays puzzle games with you―but you can expect virtual beings to do a lot more over the next decade. This coach will have access to all your data, or at least the data you want it to have access to. It will have access to your location, gait, and heart data at minimum, and will know whether it is pushing you hard enough to reach your fitness goals.

It will go a lot further, though, in motivating you to do better. On YouTube, you can see a video of 6,000 people running across Central Park in New York to catch a Pokémon. That wasn't a very compelling visual when users of Pokémon Go stared into their mobile phones, but it shows how computing visuals can be used to motivate people to exercise. With glasses that bring us AR, your healthcare provider will go a lot further we predict―helping you choose the right foods, smoke fewer cigarettes, exercise both body and mind, and more. This leads to lower healthcare costs, healthier lives, and more information for the coach itself to have a chat with your doctor about, if you allow such a thing (and you probably will, given the benefits). The games it will play will make that early Pokémon Go game seem quaint soon. Already we know of people who have lost many pounds playing Beat Saber in VR. Imagine games that can improve your eyes, like the one that Calder has built, or exercise your body, or get you to eat better. These are just a few examples of how healthcare is about to get more intimate, personalized, and in many ways "built in" to daily life.

Research already shows that if food portions are made bigger in either VR or AR, you will eat less of that item (the research study, funded by the University of Tokyo, used a variety of foods in its tests to show participants different serving sizes). Imagine a virtual coach who assisted you with your food goals this way with you all day long.

Dr. Ahmed explains that this kind of social reengineering is bound to happen and that patients will be persuaded to take the journey into a preventative world. "Your glasses will look at your food and the Augmented Reality will show you calories," he told us.

"Getting people to change is the Holy Grail," Dr. Spiegel says. "Human beings understand concepts better when presented with more direct evidence, such as that which is visual and made more accessible by being three-dimensional." Translation: a virtual coach could get you to learn and change faster, which will lead to massive changes ahead for everyone in the healthcare business, and more fun for patients as their virtual coaches entertain them while exercising or eating out.

These kinds of virtual coaches might not look like anything human, either. If you visit a new gym in San Francisco or Boise, Idaho, run by Black Box VR, they have weight-lifting machines that you use while you are participating in a VR experience. If you pull harder on the cable coming out of the machine, the visual in the headset plays a new kind of game with you. As you work out, the machine increases the resistance as you "level up." We tried it and it uses all the pain distraction techniques that the University of Washington used with burn victims. While exercising we didn't notice how hard we were working while playing a variety of games, for instance. Most of the games are things like punching out a monster. Once out, though, we were sore from the day's workout. On return visits, it remembers us and pushes us to go further in its game. They promise "every minute is optimized through Artificial Intelligence and adaptive workout prescription." They use visuals where your brain perceives an actual danger―virtually―and get you to work out harder to achieve your goals. The AI even watches your workout so it knows not to push you past injury levels.

The age of virtual coaching and preventative healthcare. To enable all this, we need a ton of data to drive the future of personalized, predictive healthcare.

Data, Data, Everywhere

In 2020, as we still write this book, there still are many hospitals around the world that haven't digitized their patients' health data yet, and even those that have are struggling to make these EMR (Electronic Medical Records) systems useful or the data in them clean and understandable.

Doctors aren't famous for using clear, consistent language, and they often use tons of shorthand and very specialized terms that can contain lots of typos upon entry. We heard tons of stories about problems when we attended Ray Wong's influential Healthcare Transformation Summit for CIOs of healthcare organizations about how medical systems are struggling to not just get the data in these systems into a useful form, but how to get doctors back to focusing on care rather than inputting a ton of data that doesn't lead to better healthcare for patients.

We see Spatial Computing glasses as a potential answer. It is in the supporting technologies, whether to listen to doctors' and nurses' intent and language, support new kinds of Computer Vision that can look at various things and come up with better data inputs, or connect all the systems in a hospital to talk to your health record. The pressure to improve the efficiency of the system so more people can afford good healthcare will push administrators to continue investing in new technologies like the ones we've laid out.

How many hospitals still have a ton of machines that don't talk to each other in a patient's care room? Most, if not all. All of these problems give skeptics the encouragement to say it's impossible to get to the world we are laying out, and other skeptics are worried the surveillance systems that are under development will lead to dystopian outcomes (your glasses could soon see that you eat too many donuts and could increase your insurance costs accordingly, they worry, and their fears get worse from there). We will take on these fears, and the balance between utility and dystopian control, elsewhere in the book.

Many parts of the healthcare system will be slow to upgrade, like changing a simple drug dispensing machine that might take years to transition to a new always-connected digital model. Connecting machines to a cloud is necessary for the complete overhaul of the medical system, but Douglas Purdy, who used to run Facebook's Platform, gave us another place to look at just how deeply data and Machine Learning could affect healthcare and all of our lives. He did this by showing us that data is already changing in how we can provide healthcare in ways that were hard to dream about only a few years ago.

Purdy built an AI that monitors heart rate variability (which is the signal in between the electrical pulses that pump your heart) and told us that, if you train a computer correctly, it can detect patterns in the information between your heartbeats that can tell whether or not you are getting enough sleep, or taking care of your mental health, among other things.

He explained that the heart doesn't actually pump all at once, which is why there's so much untapped data here. What actually happens is an electrical signal starts in one nerve pathway in your heart, causing the muscles to contract, and that signal passes literally cell by cell through your heart.

Humans simply can't watch this―the same way humans had a hard time telling whether a horse's feet totally left the ground until motion pictures came along. But now, our sensors are getting so good at recording the heart that new patterns are being found in between the heartbeats. AI can see those patterns where humans couldn't.

Purdy is using that AI inside an app called Zendō (http://zendo.tools/) to watch your health and tell you if you are meditating right, or enough, among other things that are on the way, including sleep, diet, and more. Dr. Spiegel and others validated that heart rate variability is something new, and an exciting place to use AI that has only recently became usable for consumers because of the widespread use of watches that have heart-beat sensors built into them, and said there's exciting new research that is underway now that we have powerful computing tools to analyze patient data like that in real time. Even Apple now has a "Research" app that you can download and use to add your own experience―data, essentially―to a variety of medical studies underway. This, along with other data, is doing something new, and could lead to the "check engine light" kind of functionality discussed earlier. Will it lead to your glasses telling you in advance that you will soon have a heart attack? That's a bit presumptuous for us to say today, but we can see how data will lead to that kind of system.

Dr. Brandon Birckhead put it clearest in our discussions: "A new era of precision VR therapy is here," he says. "We will be able to tailor things precisely down to the person."

That kind of hyper-personalized, intimate healthcare is possible thanks to wearable devices like Spatial Computing glasses along with watches and other devices, like a pacemaker that Boston Scientific developed that listens to your heart directly, leading to benefits. Said pacemaker can save your life if you aren't following doctor's orders to get enough exercise, or are eating unhealthily, for instance.

Dr. Rizzo told us he is seeing in his studies with PTSD sufferers that many actually share more when they think they are talking to a computer. "They self-disclose more information," he says. "Contrary to popular belief, people are less concerned about shame [when talking to a computer]," he told us. They worry less about consequences. Even spoken words contain data that can be quite revealing. At SRI International, researchers showed us an AI-driven system where you read a few hundred words from a card, and it identifies whether you are suicidal or not. Stress in your voice can't easily be detected by other people, but a computer can see patterns most will miss.

Dr. Daniel Kraft explained the role of data best, and it probably comes from his work in the military; he was a flight surgeon, but did a bunch of other work, including conducting research on aerospace medicine that was published by NASA. He says all this data will lead to a new kind of situational awareness machine that will watch all parts of the healthcare industry.

Today, he says, you are in the dark with a lot of healthcare. From staffing, to triage, to people pushing food trays around, inefficiencies abound within the hospital infrastructure. Worse, this could be compounded by poor communication, and in a critical environment such as healthcare, this can lead to tragic results in the worst-case scenario. He sees a lot of misaligned incentives, which lead to bad outcomes, but this "situational awareness machine" could exponentially learn as doctors, nurses, administrators, hospital staff, and patients themselves teach it new things and correct old, bad patterns of work.

Dr. Rizzo sees that all this will build a new virtual environment that will help us in every step of our healthcare journey. He says: "The future of this is a companion, buddy, or support agent. Medicine is always about dose/response, and if you can quantify those things better, without compromising patient privacy, we are going to find relationships, or causal things, that go beyond our brains' ability to read journal articles and learn with traditional techniques."

Precision and Personalization

Data collection will only increase as time goes on. Our modern-day society is obsessed with precision and personalization. All of this data collection combined with AI and Spatial Computing is bound to produce healthcare that saves many more lives than it does now.

Spatial Computing brings many changes to not just the healthcare system of labs and hospitals, but heralds a new move to very personalized medicine that can both help you avoid the healthcare system longer, but, once you are in it, can watch your care, and even deliver new care through AR or VR headsets that was impossible only a few years ago.

This brings us to the end of our journey through healthcare and the changes that are expected to come. In the next chapter, we'll take a look into a very different world―banking and trading―and how Spatial Computing is also making waves in this sector.