NewPathVR spends the major portion of its time conducting research and development. We read and follow as much research as we can about the psychological, neurological, psychoacoustic, and affective scientific behavioral changes that can be made using immersive technologies. Simon Chandler at Wired recently wrote in an article titled “Virtual Reality’s Latest Use? Diagnosing Mental Illness” that VR is a promising diagnostic tool, researchers say, because it generates scenarios and experiences that can’t easily be produced in a traditional clinical setting. The question the article asks “What is it about VR that makes it a promising technology for detecting neurological and psychiatric conditions?”
Early applications of the technology were in designing vehicle simulators. The technology was mainly used for training military personnel, pilots, and astronauts; but largely remained out of the public eye. The first applications of VR in medical teaching occurred in the 1990s with colonoscopy and upper gastrointestinal tract endoscopy simulation. Today, VR is being used to detect and treat psychological and neurological conditions and the questions are why and is it better than current methods. The answer is yes, and here’s why.
First, VR simulates real life very “convincingly”, so convincingly that your brain remembers VR as if it were real life. A notable 2009 Stanford study showed that preschool students surveyed 5 days after a VR experience of them swimming with whales believed that the experience was a real memory. They believed they actually swam with whales. Younger minds tend to be more impressionable but this begins to show you how powerful VR is.
Jeremy Bailenson’s Four Reasons to Test in VR
Jeremy Bailenson, founder of Stanford’s Virtual Human Interaction Lab and author of “Experience on Demand”, states four reasons for testing something in virtual reality. He says you should build something in VR if otherwise, it would be:
If something were dangerouslike spiders or the plank experience, VR would make sense. To address the fear of flying, or reproducing an auditorium of people to practice speaking in front of would be quite expensivecompared to a VR experience you could do again and again. To create a game where you shoot frogs in space would be impossible. Finally, it may be counterproductiveto recreate role-playing between domestic abuse partners.
However, there are many other reasons to build in the immersive environment and many reasons why you would diagnose psychological and neurological conditions using VR.
VR for Real-Time Observation and Modification
One of the benefits of virtual technologies is that of the therapist’s ability to monitor the patient’s activity. Some systems can be used to identify emotional facial expressions in patients with psychosis, or how to pick up other visual cues by watching the user or using heat mapping software. In this way, the therapist can be responsive to the patient and adjust the therapy in real-time in some cases.
VR for Testing Consistency
Virtual reality allows one to create a consistent, error-free environment or environments that can be customized for your assessment conditions. In the VR environment, you have complete control and can repeat simulations with finite accuracy and consistency. This combined with the real-life likeness of surroundings and objects provides control in order to make assessments.
VR for Privacy and Removing Fear of Personal Judgment
There is some evidence to show that people will divulge more information to an AI or scripted avatar than they will to one embodied by a person or to a person directly. They also prefer to interact with an avatar of their own sex. This has to do with the feeling of being judged, which patients feel less of when dealing with an AI avatar.
VR for Gamification During Research
VR can provide patients with added motivation by adding gaming factors. This motivational aspect can lead to more accurate results and deeper engagement by participants. Gamification in VR is especially compelling, again because of the realism. For generations, game theory has been developed and refined and this psychology can transfer to VR, particularly in the dynamics surrounding how to keep the user engaged through each step or stage and for gathering emotional feedback.
VR for Safe Psychological Activation
There are other reasons, too, ie many therapeutic methods in vivo encourage you to put yourself in a state of stress, anxiety or PTSD, such as exposure therapy for agoraphobia or fear of crowds which include actually exposing yourself to crowds of people, perhaps activating a panic or anxiety attack you can’t escape easily. If you were doing this therapy in VR, you could immediately remove the headset and address the emotions.
Creating Specialized Scenarios in VR to Detect Psychological or Neurological Conditions
To answer more on detection, Greek researchers in 2015 built a cognitive training game for patients with mild cognitive impairment, something that comes before Alzheimer’s. This can detect memory loss by presenting scenarios that are increasingly complex such as financial planning.
When compared with existing cognitive tests, VR is showing great promise in helping to diagnose and help with memory. Cambridge University’s Dr. Dennis Chan tested participants’ spatial navigation and memory by having them don an HTC Vive headset, follow an L-shaped path in a virtual environment (initially mapped out by cones), and then trace their footsteps back to their starting point without the help of any markers. Chan’s team reported that the VR-based navigation test was more accurate in diagnosing mild Alzheimer’s-related impairment than traditional “gold-standard” cognitive tests, such as figure recall and symbol tests.
Here at NewPath we are also inspired by Adam Gazzaley’s work at UCSF’s Sandler Neuroscience Center. Some of this research says that essentially as we get older, we lose the ability to ignore distractions. Adam’s research has shown that you can train the elderly brains to concentrate and focus, through gaming mechanics, improving memory. His research has also shown that video games can increase cognitive ability.
VR and Biometrics to Determine Psychological and Neurological Conditions
Both neurological and psychological conditions may present themselves physically, hence the warranted excitement about biometrics and certainly those are being utilized for detection, but there are many issues to consider in this area.
Cubicle Ninjas used the heart rate tracking in concert with their VR meditation application called Guided Meditation VR. Users would attempt to lower their heart rate while using their app and a biometric headband. Some of the issues with using these devices together become evident right away — privacy, user education, sizing, consistent results, and shared data across both apps. Well, it’s not a closed loop, right? The VR application is not integrated, not HIPAA-compliant. Companies need to think these things through. Cubicle Ninjas removed this feature in 2018 due to “increased regulation around biometric data.” they said. Another problem is the specificity to the hardware. An app that is specific to a headset, specific to a biometric device for the app? That’s like a spoon that only works for hot carrots.
Detection of Psychological Conditions Through AI
NewPathVR is working on an AI-enabled AR/VR product that assesses your depression level upon intake of your therapy session. Currently a cold, numeric assessment, we’ve taken an avatar, meant to be one of many you could choose from, who asks you questions using natural language and a more personable interaction that we believe will generate more accurate answers and engaged patients, helping patients and clinicians and decreasing healthcare costs.
We also just held our 2nd Annual VR Wellness Hackathon, and one team created a prototype for an app that allows you to externalize and identify visually your anxiety or depression or the emotion you are experiencing with colored and textured brushes in VR and then interact with it by lighting it on fire, pushing it far away, erasing it, or just giving it a hug. By seeing your emotions as external notions rather than as ourselves, as we sometimes say “I am anxious”, and that is an identifier, but by seeing the emotion outside of ourselves, we are better able to act upon feelings.
Many people are excited about facial recognition. The problem with facial recognition at this time is that the VR headsets inhibit recognition of the full face. Eye tracking is something that you might see come out now incorporated into some of the headsets, but again, we’re edging into the business of your bodily health information. Pupillary dilation, for example, says a lot about people and we are definitely headed into Minority Report territory — it’s the merging of these two that is a delicate weaving we’re seeing.
VR Compared to Other Methods of Research Recording Methods
VR is certainly more accurate than human recording methods, because of pilots errors and more. But beyond that, beyond say a computer program, why is a VR environment better at diagnosing than a mobile app, for instance? It’s because of the touch points. It has the potential to be a better diagnostic tool because of this, a better education tool, a better change tool.
Put it this way. If TV was a seal, and computers were maybe an octopus, VR is a jellyfish. It can touch you everywhere!
Because you are PART of the experience of VR, VR can connect to you in so many more ways. You are IN it. VR is experiential.
The reason we believe that VR is such a powerful tool for psychology is that memories and behaviors are formed from experiences. And VR is the first “experiential” medium. We can present experiential environments and present experiences for people and form memories, writing neural networks to change behaviors once people remove the headset. That’s a major driving principle and theory for us at NewPathVR.
Our team built an app here for role-playing in which you choose actors and scenes and role-play past memories to do just this. Users record dialog and then role-play conversations from those memories to reform memories and change narratives in their lives to which they may have formed maladapted behaviors and coping mechanisms that don’t suit their present-day relationships and healthy behaviors.
Dr. Brian Chau, a physician who writes about new medical technology, says, “The key here is data — we need validated measurements” to show VR is comparable to or better than traditional methods, he says. Continued partnerships between clinicians and VR developers are needed to move the technology” from the lab bench to the clinical bedside. NewPathVR agrees.
Why Psychological and Neurological Conditions Go Undiagnosed
Mental Health is a silent epidemic. For example,1 in 5 people have depression, 70% of these are between the ages 18–25 per the CDC in 2018. Eight (8) million people die each year due to mental illness. Mental health is not spoken about or given as much funding as physical illness, so it is harder to find research and support and find new more effective, ways to solve problems for this area.
Traditional techniques do not always work. It is difficult to diagnose mental illness due to issues of stigma, lack of mental healthcare workers, expense, and the nature of the illnesses themselves which pose inconsistencies and challenges. Individuals vary in responses to traditional treatments and for some people it is a lifelong commitment (e.g. there are many patients who cannot afford to keep up long term therapy, especially people who are more disadvantaged, most at risk and most in need, and government counseling services cannot possibly reach everyone) and although through community outreach programs it is possible to deliver these services to many more people, some people tend to be more closed off and find it difficult to connect with traditional processes or it can take them a while to do it.) In these situations, VR could offer a better solution as it addresses issues such as expense, accessibility, and stigma.
The Psychological Dangers of VR
Yes, there are dangers to consider. As one example, I mentioned the strategy of “activation” in order to practice skills during your stress state so you will readily be able to call up those skills during your next stress state. This is often done under the supervision of a therapist. Developers need to consider how to contain a user’s experience in this regard when a therapist is not present.
When subjecting individuals to tests intended to bring out symptoms associated with panic or psychosis, there is a danger that VR could provoke or cause these symptoms in people who would have otherwise never exhibited them. In other words, there is, in fact, the risk that VR could cause psychiatric symptoms, such as PTSD, in healthy individuals.
There are several other important things to consider when adding VR to a clinician’s toolkit. Practicality, privacy, safety and more. NewPathVR teaches courses on these topics precisely. We are the only company approved to teach accredited courses on VR Psychology for the practical and clinical use of VR with patients. We teach practitioners about the guidelines for the different headsets, the research behind conditions being treated with VR successfully and areas that are still gray. VR applications available for each headset, pricing and options and solutions that suit individual or organizational scenarios are also covered. We teach safety, what population will be using the technology, app insurability, sanitization, we cover all of this. Importantly, which applications are for use with a clinician and which apps people can use at home on your own.
We also believe that standards and guidelines need to be developed for XR across the board, usability, and ratings, for example, and we are familiar with some associations working on this.
I wrote an article about the responsibility of VR developers in 2016 to try to shed light on this topic. This indicates that the tool has to be used with care and caution, particularly with the entering and exiting of an application (the “handling”, I call it, or “containing” some people say) however, the fortitude of the platform also translates to its positive effects, and when using it to treat and enhance mental health, it can have enormous benefits for an individual.
In summary, VR provides an excellent opportunity for optimized testing and detection of psychological and neurological conditions. There are important things researchers and clinicians must keep in mind with this incredibly powerful technology when doing so, and working with experienced professionals is a good way to avoid some of the pitfalls inherent in this newly emerging, but very promising, technology.
Virtual Reality for Mammograms and Annual Exam Anxiety
Kaiser Permanente has retrieved its poor reputation from the clenches of the 80’s and now boasts many accolades and awards of recognition for its leadership in healthcare and programs, leaders, and in places to work. Patients no longer complain regularly about mishaps, insurance nightmares and long office waits. Kaiser ranks top in member satisfaction and has invested billions in electronic records and modernizing medical department and administrative equipment for the benefit of practitioners and patients alike. I, myself, am a Kaiser patient, and credit the payer-provider organization for saving the life of both myself and my husband when we were each faced with life-threatening diseases — him with a heart attack in 2009 and me with breast cancer, 5 years ago in 2013.
I found out about my breast cancer on my 43rd birthday at 8 am. The phone call woke me up. I thought it may have been my dad, calling to sing me happy birthday. “This is Kaiser. We’re calling to tell you that your test showed that you are positive for cancer in your left breast. We recommend a mastectomy We have an opening on Thursday. Would you like to schedule it?”
Um, can I get a second opinion?
“You can,” she responded, “but you don’t want to wait long, it’s very aggressive.”
After a second and third opinion, I took the Thursday appointment and had a mastectomy. I would love to tell you how brave I was, how quickly I recovered, and what a poster child I am for how to approach breast cancer as a patient. But the story is the exact opposite. I went in totally uninformed. I went in with cringed eyes and phenomenal stress. There was no time, and no one there to explain it adequately. It was terrifying. I had barely enough time to search Google to determine which kind of breast cancer it was and what my chances were for surviving. I didn’t know how to find information about what the process was going to be, what my options were, what recovery would be like, or how this would impact me or my family. I was completely in the dark.
The whole process was so quick, there was no time to think about the decisions I was making. We were given pamphlets and a page with checkboxes and if we needed more time to decide, we were left alone in a room to discuss it (and look things up on Google), but I mean, Jesus. Both breasts, just one? Reconstruction or not? Saline or silicone? Tummy tuck add-on (for real, yes, no charge)? Is 11 am good for you?
“You’re so brave,”
“You’re so brave,” people kept saying. That’s a nice thought. I want to be someone that others can look up to, someone who weathered the storm with a stiff upper lip, but I’m afraid I let everyone down on that count. I was in a panic most of the time, I cried after sometimes in confusion about what was happening to my body, sometimes just out of fear of what would happen next. I was a total nuisance to the nurses and doctors, calling on them constantly with questions and concerns and escalating my issues to their supervisors when I didn’t like the answers or treatment I received.
When I got home, I couldn’t stand to look at the scars so I showered in the dark for months. The PTSD from the surgery and procedures afterward did not fade really. I didn’t want to be touched. My self-confidence started to suffer. My very understanding husband was doing whatever he could to support me in my healing, but I kept falling into despair and self-loathing, I didn’t feel whole anymore. Year after year I would watch the breast cancer walk on TV and wonder why I hadn’t bounced back so effortlessly they way the spokeswomen had, in their bright pink t-shirts with ribbons pinned to them. I looked for breast cancer survivor support groups and found two within an hour’s drive. One was for young women under 40. I was 43. The other required that you leave a voice message so I did and no one called me back.
Statistically speaking, there is a 3% chance that I will have cancer in my right breast in the future. Doesn’t seem like a lot but….the chances of winning the National Lottery are 1 in 45,057,474. And I’ve played that, probably, 20 times in my lifetime. Every year since the surgery, when I receive a call from my Kaiser nurse telling me it is time for my annual mammogram and women’s exam, an anvil drops into my stomach. Time to play again.
I schedule a day that I have no meetings because I know that I will be so psychologically and physically triggered that I shouldn’t plan anything else. I usually plan to get cupcakes or ice cream afterward so I have something to look forward to and reward myself (something leftover that my mother used to do for me, I still do for myself).
When cupcakes aren’t enough
We promised you the technology is evolving and this year Oculus came out with a phone-free, tether-free headset with a nice resolution that runs apps that ran on the Samsung Gear VR, which required an expensive phone. I’ve been singing its praises for months, suggesting patients use it in hospital beds, waiting rooms, and doctor’s offices. We even built an Oculus Go application called the Prostate Procedure Guide for patient education and anxiety reduction. I imagine you’re beginning to see some of the origins of our motivations.
At previous exams, I premedicated with anti-anxiety medications and/or recreational drugs. Yes, I’m (not) the portrait of strength and fearlessness, as mentioned. Yet these solutions somehow made me more sensitive to the experience in many ways and that wasn’t productive. This year I am convinced by the extensive research and people I have witnessed and their healing experiences with VR. I have seen and reviewed hundreds of wellness applications on RenewVR and have read hundreds of articles and research papers on the mental health benefits of VR. So I decided to load up my Oculus Go and take it to my appointment.
First was my mammogram. I sat in the waiting room and started an app called Kaleidorium.
I paired the app with my Spotify playlist I use while I make art made up of some current favorite songs (Hearts and Bones, Paul Simon; Let You Love Me, Rita Ora; Wild Things, Alessia Cara; What About Us, Pink) and I watch the beautiful colors change and fly by.
I hear the nurse call my name and forget that I was supposed to be worrying about this procedure. I quickly disrobe and step into the room with the radiologist to perform the mammogram. She does her thing. I resist the urge to look at the screen but wait for her response. She sees these all day, she knows what’s up. She says it looks good but they’ll contact me in a few days to confirm. I notice that overall I am not so freaked out. Now it’s time for my pelvic exam.
Ladies. Let us speak frankly. One really can’t equally compare a prostate exam to a pelvic exam. I’ll take one of those any day of the week, in fact, well nevermind. Could we more fairly compare it to an alien abduction with a non-consensual probe? Yes, see, several hands just went up in agreement. In a prostate exam, the doctor uses a finger. In a pelvic exam, they start with this.
Wait, you are wondering, are we not past the 1700’s? Why yes, that’s true, but we still use steam-punk, friggin’ ice-cold, torture-looking devices and this is just so the doctor can get inside our lady bits, I’m not going to go into what they do once they get in there.
Think cupcakes. Think cupcakes. Think cupcakes.
So, you still get to lie in the same cold slab of weird, paper-covered, stirrup-equipped bench of the 1920’s, thank GOD, thank you SO much! I was left alone to undress and my anxiety started to rise. This procedure can be as brief as 5–15 minutes, but my knees are often times left shaking afterward for hours. This time, when the doctor came in with her assistant, I said, “Give me a moment,”
“I brought my VR headset to help me relax,”
This time I put on an app called Azul. It’s a meditation and mindfulness app that helps you focus.
Instead of looking at the foam panels in the office ceiling and fluorescent light bulbs, I selected a scene, a field with birds in the sky, and I selected the music and laid back. I watched the birds fly in circles.
Before I knew it, she was done. It was that fast. Time flies when you are in VR, people often recount that they thought less time had gone by while they were using it. But in this case, shorter perceived time, no pain, very little discomfort, and no PTSD as I usually have had after this procedure.
As the CEO of NewPathVR, I spend my days evangelizing the psychological benefits of virtual reality. Taking it into my own life is a natural step but one that surpassed my expectations still. I didn’t actually need those cupcakes or meds after all. Virtual reality is a powerful state change tool and remains the most evolutionary force to meet so many industries in years, particularly emotional wellness.
The great news is that I received a clean bill of health. 5 years cancer free!
I recommend the Oculus Go this winter for you, your kids, your friends, and family, believe me, you’ll find uses for it. This spring, you’ll want to grab the Oculus Quest too, for full room-scale VR and higher resolution apps. Here are links to the apps I used but there are hundreds more on renewvr.com.
When Lisa Padilla first approached me with her vision for NewPathVR, I was immediately inspired by what we could achieve. Imagine the possibilities to not only unlock the hidden power of the brain but to empower our users to look inside themselves and find a way to improve. To feel better. Gain the strength to refine, and then to share that with others. While VR may be many things, to the folks at NewPath, it is the best tool to reprogram the brain for success. We huddled around and researched hundreds of papers where studies demonstrated how positive reinforcement, perception, and sensory filters can influence our behavior, as well as our memories.
Research and Design Phase
We knew we needed a prototype to prove VR as the right medium for spiritual growth, but what platform could serve us best?. Was this going to be a seated experience? How much interaction should it have? And what to build? For instance, did you know that if you perceive yourself as taller in VR, it actually makes you feel more confident during and after you remove your head-mounted display (HMD)? As it turns out, this is very true. Our discussions turned towards identity, and how to connect our users with their VR self. In gaming, this is called your “player self”. You have three “selves” actually. The first self, the real you, is what you do outside of games or VR experiences – ie your life: work, job, family, etc. The second self is the you that plays the game using peripherals, and experiences the content through the point of view of one or several characters in the game. You become a “player self” and share characteristics with the “game self”, but you are not the character. The third self, the “game self”, is the content’s avatar that represents you, and has a role to play in the content’s story or scenarios that your avatar experiences. By witnessing the story, and in some cases, by making choices for your avatar “game self”, your “player self” gets to experience those same emotions, and thus, share those same experiences. The cool thing about VR is that the player and the game self boundaries become blurred, such that you feel as if you are literally inside the content, and you feel much closer emotionally to the experience then you would be if you were observing the content through a monitor or TV screen. Keep in mind that taking an experience designed for a flat screen does not merit porting it to VR. All content in VR should be unique and specifically designed to transform and empower the user.
When building a VR experience, game mechanics that are based on challenge-reward systems create much more value for the user if they incorporate your senses. Adding a 360 visual experience may not be enough to trigger personal, real-world change. Adding 360 audio to that experience brings us closer, but is still not enough. By adding the ability to use your body, such as walking around and touching virtual objects, to influence the content, our team realized that we could create a world where consequences could have just as much impact on the “player self” as incentives. What’s more, if we incorporate at least 3 senses, this combination activates the memory centers or the brain. With the right experience, a player may create an association between something he/she experiences in VR, and a similar experience in his/her real life.
So, we knew we had to make a room-scale experience, and we knew we wanted it to leverage game mechanics that could change a player’s mood. The obvious choice for platform was the HTC Vive. But what about the content? I volunteered that for a prototype, we should keep things simple and demonstrate that we could achieve a basic goal. We wanted to transform a user’s mood from a negative or an indifferent position into a positive one. However, going through a sequential experience in VR, will usually not improve your life on exposure alone. In the real-world (or what we perceive to be our reality), we can usually learn any skill and master it, by practicing it over and over again. In our VR prototype, we needed to do the same. We decided that if we could create content that would teach our players some moral or zen-like lesson. A takeaway. But then also provided an opportunity to apply it, then we could create real personal growth.
Our “Self-Compassion Buddy” vision was born. In our prototype, we essentially introduced our users to their avatar self by literally creating a virtual mirror. The system tracked each of the user’s arms and mimicked his/her movement through the avatar that was reflected in the mirror. Our research showed that in order to strengthen the bond between the player-self and the avatar game-self, our users would need to interact with their mirrored reflection for approximately 70 seconds. This seems like a long time for a prototype, but in a future, commercial version of our product, the interaction could involve a game mechanic with a reward incentive. Next, we focused on the story ingredient. We did not require an elaborate story to demonstrate our vision; only a simple scene based on some narrative context. I believe that in order to create positive change or at least to invoke positive feelings, you need to have contrast, and that means placing the user into a negative situation – for a very short time of course. Then, follow it up with a positive environment filled with good energy. By placing the user into a slightly distressed state, and then moving him/her into a comfort zone, you can generate a sensation of emotional gratification. Games also apply a similar approach when they create a difficult challenge, in which a player must learn a new skill to overcome it. Once the skill is learned, the obstacle is easily navigated, and the player moves on to claim his/her reward. But more importantly, in your own life, when you undergo challenging times, and not only survive them, but learn to be stronger as a result of them, you then undergo positive change within yourself. Through overcoming these challenges, you may either improve your level of independence and self-sufficiency, or you may grow more carefree by successfully navigating stress and becoming familiar with it.
The Prototype Phase
We applied this to our “self-compassion” prototype. Imagine being immersed in a dirty, poorly-lit, virtual environment that exhumed negativity. You find yourself staring at your reflection in a mirror. You move, it moves. After a lengthy exposure to your reflected avatar, your avatar aka “buddy”, starts moving independently of you. It steps out of the mirror. Charges at you, invading your personal space. The result. You start feeling threatened. Your “inner bully” points his red finger at you and verbally abuses you, calling you “a loser…and a failure in life”. After a few moments of this, your brain switches to panic mode – a sort of fight-or-flight response. We kept the user in this state for about 7-10 seconds before interrupting the experience with another friendlier avatar. Any longer than that and we would have risked spoiling the whole experience and alienating our user.
The friendlier avatar, a nurturing female guide appears and rushes to your rescue. Freezing the bully in action, she explains to you that the bully is you. Recap: by belittling yourself, you lose confidence in yourself. The female guide then offers words of encouragement to rebuild your confidence. Her words manifest in a new scene in which soap bubbles shower from the sky. Some of the bubbles contain cute gifts such as adorable stuffed animals and pets. We introduced a bit of the fun factor in this scene. When the user pops one of these soap bubbles containing a gift, the female avatar aka your “guide” offers words of encouragement with a positive message, ie: “Lots of people care about you.” Each time you pop a bubble, the gift item inside goes into your collection, and new positive words materialize. In the next scene, you have an opportunity to apply these gifts and redeem yourself. We call this the “pay-it-forward phase.” You observe three couples, standing at surrounding points around you. These couples each reveal a virtual buddy figure matched with his respective inner-bully avatar. Similar to your initial case, the bully verbally terrorizes his victim. This continues in an endless cycle, until you interrupt the buddy, and hand out gifts from your previous scene’s collection. Each time your gifts are received, the words that were initially associated with the gift, retrigger audio playback again. The buddy’s inner-bully vanishes and the buddy then begins playing the ukulele. When all three avatar buddies receive their gifts and play the ukulele, a well-known song begins playing, lifting the mood even more and embracing a sense of relief and closure. The three playing buddies merge into one buddy enclosed behind a giant mirror. And you are once again faced with your own buddy reflection. The simulation ends when the environment cross-fades into a sunny sky, and you find yourself standing on Cloud Nine. Literally.
Post-Analysis Phase We quickly discovered that when a user goes through this VR experience, they feel better coming out, then going in. Our research also suggests that if we had ended simulation after the initial soap bubbles were popped, gifts received, and words of encouragement heard, then the effect would have been short-lived. By adding the “pay it forward” scene, where the user returns the favor and gives a gift to his surrounding buddies, we essentially teach our users to apply their acquired skills and pay them forward. To share. Therefore, our users resolve to help themselves, feeling a sense of contribution and meaningful value. Generosity in VR impacts our users in the real world and has longer lasting effects on their mood. It builds confidence and self-love.
Our biggest challenge in building this first prototype was testing it. I relied on using Unity as our main game engine development tool. Unity not only has established partnerships all over the world, and is compatible with 27 target platforms, but it is also free to use for development. Due to our limited resources and limited access to the HTC Vive, I ended up building our prototype using game object placeholders to represent both the HMD and the two game controllers. I then parented OpenVR’s game controller objects under these placeholders. When I repositioned the placeholder game objects in Unity’s simulator, I was able to estimate fairly accurately how the HTC Vive’s game controllers’ movement would impact the VR environment around them. On a weekly basis, under limited time and limited test access to the shared hardware, I methodically validated and tested our experience on the Vive hardware, tweaking and improving our prototype step by step. Unity is not quite WYSIWYG (What You See Is What You Get) when it comes to the built-in development simulator. That’s why end-to-end testing is so important, especially in VR. I set up a mode that would allow me to switch between my mocked game objects and the hardware game objects in the scene project. Doing so, allowed me to execute a series of tests on our shared hardware device while continuing development in our mocked Unity environment.
Our second challenge was getting our 3D buddy avatar to move correctly in our initial mirror reflection scene, used to build an identity association with your inner buddy. I solved this by building a mimic-engine that tracked the delta positions of the mocked, game-controller-object placeholders. The engine then inverted these vectors and applied the new deltas to a basic, rigged stickman model. I added constraints on the limbs of the stickman and locked the lower limbs so that only the upper body would be affected to move freely. And it worked. Additionally, since the Vive is a room-scale experience, the position and orientation of the stickman (aka our buddy reflection) needed to map to my HMD game object’s position, such that when I moved left or right, my reflection (facing me) would move in the opposite direction. And because the mirror image itself has borders all around, our 3D stickman was piped through a render-texture camera, that projected the image onto a 3D mirror game object as a texture. The mirror game object itself had no reflection, but projecting the stickman as a texture on top of it, gave the illusion of a mirror reflection.
No matter how you choose to implement your own VR project, remember that VR is highly immersive. Due to VR’s transformative nature, the underlying purpose of your content should support a key responsibility for contributing to social goodness, and hopefully, empower our users to live more fulfilling lives.
Eiran Shalev is an experienced technical, hands-on leader with 18 years of professional expertise overseeing top teams on mobile, social, and web technologies for products ranging from multi-player mobile & social games, to streaming video ads to interactive television and more. He comes from Disney Interactive, where he established the technical vision, and helped to scale and deliver Disney’s mobile technology platform to all game studios. Before that, he spent time at Koolbit, Kabam, and RockYou! He has built more than 50 games. He joins NewPathVR as CTO.
There are two technologies that humanity has been looking forward to for decades. One of these technologies is time travel. The other is virtual reality. You may want to throw ‘flying cars’ up there somewhere as well, but some people haven’t seemed to realize that planes essentially are just that. Regardless, of all of these technologies that humanity has wished for, only one of them has come to pass so far: virtual reality. The thing is, with virtual reality, it may be possible to experience the other technologies through it.
Granted, virtual reality will never be able to truly, physically send you through time and space to exist in a different era. But, with the power of virtual reality, it may be possible to experience the past and future in every sense besides the legitimate physical.
Imagine, for instance, virtual reality that lets you experience the American Revolution firsthand. As a soldier perhaps, or even one of the commanders. Or perhaps you would just be present in an ethereal sense, watching everything from the safety of virtual reality. Or consider the opposite. What if you could go into virtual reality, and experience a programmer’s idea of the world in the year 3000? While you can’t claim any accuracy to what people think is the future (though Back to the Future did an uncannily good job with their Cubs World Series prediction), you could still experience what people believe the future will be like, from flying DeLorean’s to those hover boards we never got when we were supposed to.
Obviously, none of this will truly take you to the past or the future. You will still physically be present in the world of 2017, but, the real question is, does it make a difference? If you experience something that is so real to your senses that it may as well have been, does it really matter if it actually happened? To some people it might, but to many others it does not.
If you could use virtual reality to experience say, sky-diving, would it not be the same as actually sky diving so long as it was realistic? The same notion can be held to the idea of pseudo-time travel through virtual reality. Maybe you didn’t really travel in time back to the Crusades, the American Revolution, or one of the World Wars. Maybe you didn’t really travel to the year 3000 and witness the future. But if the experience is real enough, is it not the same as though you actually did?
It is something that is yet to be seen, but highly anticipated by all. Moreover, the potential is nearly limitless. With this kind of virtual reality, will the way history is taught change? Will students be able to experience renditions of the history they are learning first hand? And what of religion? Rather than read the Bible or the Quran, will you be able to experience every part of it from the perspective of someone who was actually there? You very well could, provided that a programmer desires to make it so.
In the end, virtual reality is one of the most exciting things to happen to humanity in decades. And though we may never truly travel through time, who knows what we could one day experience through the power of the technology we now have. I’ll meet you there or, rather, then.