Analysis of a Scene: Building Our First VR Prototype

by Guest Author Eiran Shalev, CTO, NewPathVR

When Lisa Padilla first approached me with her vision for NewPathVR, I was immediately inspired by what we could achieve. Imagine the possibilities to not only unlock the hidden power of the brain but to empower our users to look inside themselves and find a way to improve. To feel better. Gain the strength to refine, and then to share that with others. While VR may be many things, to the folks at NewPath, it is the best tool to reprogram the brain for success. We huddled around and researched hundreds of papers where studies demonstrated how positive reinforcement, perception, and sensory filters can influence our behavior, as well as our memories.

Research and Design Phase
We knew we needed a prototype to prove VR as the right medium for spiritual growth, but what platform could serve us best?. Was this going to be a seated experience? How much interaction should it have? And what to build? For instance, did you know that if you perceive yourself as taller in VR, it actually makes you feel more confident during and after you remove your head-mounted display (HMD)? As it turns out, this is very true. Our discussions turned towards identity, and how to connect our users with their VR self. In gaming, this is called your “player self”. You have three “selves” actually. The first self, the real you, is what you do outside of games or VR experiences – ie your life: work, job, family, etc. The second self is the you that plays the game using peripherals, and experiences the content through the point of view of one or several characters in the game. You become a “player self” and share characteristics with the “game self”, but you are not the character. The third self, the “game self”, is the content’s avatar that represents you, and has a role to play in the content’s story or scenarios that your avatar experiences. By witnessing the story, and in some cases, by making choices for your avatar “game self”, your “player self” gets to experience those same emotions, and thus, share those same experiences. The cool thing about VR is that the player and the game self boundaries become blurred, such that you feel as if you are literally inside the content, and you feel much closer emotionally to the experience then you would be if you were observing the content through a monitor or TV screen. Keep in mind that taking an experience designed for a flat screen does not merit porting it to VR. All content in VR should be unique and specifically designed to transform and empower the user.

When building a VR experience, game mechanics that are based on challenge-reward systems create much more value for the user if they incorporate your senses. Adding a 360 visual experience may not be enough to trigger personal, real-world change. Adding 360 audio to that experience brings us closer, but is still not enough. By adding the ability to use your body, such as walking around and touching virtual objects, to influence the content, our team realized that we could create a world where consequences could have just as much impact on the “player self” as incentives. What’s more, if we incorporate at least 3 senses, this combination activates the memory centers or the brain. With the right experience, a player may create an association between something he/she experiences in VR, and a similar experience in his/her real life.

So, we knew we had to make a room-scale experience, and we knew we wanted it to leverage game mechanics that could change a player’s mood. The obvious choice for platform was the HTC Vive. But what about the content? I volunteered that for a prototype, we should keep things simple and demonstrate that we could achieve a basic goal. We wanted to transform a user’s mood from a negative or an indifferent position into a positive one. However, going through a sequential experience in VR, will usually not improve your life on exposure alone. In the real-world (or what we perceive to be our reality), we can usually learn any skill and master it, by practicing it over and over again. In our VR prototype, we needed to do the same. We decided that if we could create content that would teach our players some moral or zen-like lesson. A takeaway. But then also provided an opportunity to apply it, then we could create real personal growth.

Our “Self-Compassion Buddy” vision was born. In our prototype, we essentially introduced our users to their avatar self by literally creating a virtual mirror. The system tracked each of the user’s arms and mimicked his/her movement through the avatar that was reflected in the mirror. Our research showed that in order to strengthen the bond between the player-self and the avatar game-self, our users would need to interact with their mirrored reflection for approximately 70 seconds. This seems like a long time for a prototype, but in a future, commercial version of our product, the interaction could involve a game mechanic with a reward incentive. Next, we focused on the story ingredient. We did not require an elaborate story to demonstrate our vision; only a simple scene based on some narrative context. I believe that in order to create positive change or at least to invoke positive feelings, you need to have contrast, and that means placing the user into a negative situation – for a very short time of course. Then, follow it up with a positive environment filled with good energy. By placing the user into a slightly distressed state, and then moving him/her into a comfort zone, you can generate a sensation of emotional gratification. Games also apply a similar approach when they create a difficult challenge, in which a player must learn a new skill to overcome it. Once the skill is learned, the obstacle is easily navigated, and the player moves on to claim his/her reward. But more importantly, in your own life, when you undergo challenging times, and not only survive them, but learn to be stronger as a result of them, you then undergo positive change within yourself. Through overcoming these challenges, you may either improve your level of independence and self-sufficiency, or you may grow more carefree by successfully navigating stress and becoming familiar with it.

The Prototype Phase
We applied this to our “self-compassion” prototype. Imagine being immersed in a dirty, poorly-lit, virtual environment that exhumed negativity. You find yourself staring at your reflection in a mirror. You move, it moves. After a lengthy exposure to your reflected avatar, your avatar aka “buddy”, starts moving independently of you. It steps out of the mirror. Charges at you, invading your personal space. The result. You start feeling threatened. Your “inner bully” points his red finger at you and verbally abuses you, calling you “a loser…and a failure in life”. After a few moments of this, your brain switches to panic mode – a sort of fight-or-flight response. We kept the user in this state for about 7-10 seconds before interrupting the experience with another friendlier avatar. Any longer than that and we would have risked spoiling the whole experience and alienating our user.

buddy1

The friendlier avatar, a nurturing female guide appears and rushes to your rescue. Freezing the bully in action, she explains to you that the bully is you. Recap: by belittling yourself, you lose confidence in yourself. The female guide then offers words of encouragement to rebuild your confidence. Her words manifest in a new scene in which soap bubbles shower from the sky. Some of the bubbles contain cute gifts such as adorable stuffed animals and pets. We introduced a bit of the fun factor in this scene. When the user pops one of these soap bubbles containing a gift, the female avatar aka your “guide” offers words of encouragement with a positive message, ie: “Lots of people care about you.” Each time you pop a bubble, the gift item inside goes into your collection, and new positive words materialize. In the next scene, you have an opportunity to apply these gifts and redeem yourself. We call this the “pay-it-forward phase.” You observe three couples, standing at surrounding points around you. These couples each reveal a virtual buddy figure matched with his respective inner-bully avatar. Similar to your initial case, the bully verbally terrorizes his victim. This continues in an endless cycle, until you interrupt the buddy, and hand out gifts from your previous scene’s collection. Each time your gifts are received, the words that were initially associated with the gift, retrigger audio playback again. The buddy’s inner-bully vanishes and the buddy then begins playing the ukulele. When all three avatar buddies receive their gifts and play the ukulele, a well-known song begins playing, lifting the mood even more and embracing a sense of relief and closure. The three playing buddies merge into one buddy enclosed behind a giant mirror. And you are once again faced with your own buddy reflection. The simulation ends when the environment cross-fades into a sunny sky, and you find yourself standing on Cloud Nine. Literally.

Post-Analysis Phase
We quickly discovered that when a user goes through this VR experience, they feel better coming out, then going in. Our research also suggests that if we had ended simulation after the initial soap bubbles were popped, gifts received, and words of encouragement heard, then the effect would have been short-lived. By adding the “pay it forward” scene, where the user returns the favor and gives a gift to his surrounding buddies, we essentially teach our users to apply their acquired skills and pay them forward. To share. Therefore, our users resolve to help themselves, feeling a sense of contribution and meaningful value. Generosity in VR impacts our users in the real world and has longer lasting effects on their mood. It builds confidence and self-love.

buddy3

Our biggest challenge in building this first prototype was testing it. I relied on using Unity as our main game engine development tool. Unity not only has established partnerships all over the world, and is compatible with 27 target platforms, but it is also free to use for development. Due to our limited resources and limited access to the HTC Vive, I ended up building our prototype using game object placeholders to represent both the HMD and the two game controllers. I then parented OpenVR’s game controller objects under these placeholders. When I repositioned the placeholder game objects in Unity’s simulator, I was able to estimate fairly accurately how the HTC Vive’s game controllers’ movement would impact the VR environment around them. On a weekly basis, under limited time and limited test access to the shared hardware, I methodically validated and tested our experience on the Vive hardware, tweaking and improving our prototype step by step. Unity is not quite WYSIWYG (What You See Is What You Get) when it comes to the built-in development simulator. That’s why end-to-end testing is so important, especially in VR. I set up a mode that would allow me to switch between my mocked game objects and the hardware game objects in the scene project. Doing so, allowed me to execute a series of tests on our shared hardware device while continuing development in our mocked Unity environment.

Our second challenge was getting our 3D buddy avatar to move correctly in our initial mirror reflection scene, used to build an identity association with your inner buddy. I solved this by building a mimic-engine that tracked the delta positions of the mocked, game-controller-object placeholders. The engine then inverted these vectors and applied the new deltas to a basic, rigged stickman model. I added constraints on the limbs of the stickman and locked the lower limbs so that only the upper body would be affected to move freely. And it worked. Additionally, since the Vive is a room-scale experience, the position and orientation of the stickman (aka our buddy reflection) needed to map to my HMD game object’s position, such that when I moved left or right, my reflection (facing me) would move in the opposite direction. And because the mirror image itself has borders all around, our 3D stickman was piped through a render-texture camera, that projected the image onto a 3D mirror game object as a texture. The mirror game object itself had no reflection, but projecting the stickman as a texture on top of it, gave the illusion of a mirror reflection.

No matter how you choose to implement your own VR project, remember that VR is highly immersive. Due to VR’s transformative nature, the underlying purpose of your content should support a key responsibility for contributing to social goodness, and hopefully, empower our users to live more fulfilling lives.

Eiran Shalev is an experienced technical, hands-on leader with 18 years of professional expertise overseeing top teams on mobile, social, and web technologies for products ranging from multi-player mobile & social games, to streaming video ads to interactive television and more. He comes from Disney Interactive, where he established the technical vision, and helped to scale and deliver Disney’s mobile technology platform to all game studios. Before that, he spent time at Koolbit, Kabam, and RockYou! He has built more than 50 games. He joins NewPathVR as CTO.

Advertisements

Can Virtual Reality Sidestep The Time Travel Paradox?

tree-bubble
There are two technologies that humanity has been looking forward to for decades. One of these technologies is time travel. The other is virtual reality. You may want to throw ‘flying cars’ up there somewhere as well, but some people haven’t seemed to realize that planes essentially are just that. Regardless, of all of these technologies that humanity has wished for, only one of them has come to pass so far: virtual reality. The thing is, with virtual reality, it may be possible to experience the other technologies through it.
Granted, virtual reality will never be able to truly, physically send you through time and space to exist in a different era. But, with the power of virtual reality, it may be possible to experience the past and future in every sense besides the legitimate physical.

Imagine, for instance, virtual reality that lets you experience the American Revolution firsthand. As a soldier perhaps, or even one of the commanders. Or perhaps you would just be present in an ethereal sense, watching everything from the safety of virtual reality. Or consider the opposite. What if you could go into virtual reality, and experience a programmer’s idea of the world in the year 3000? While you can’t claim any accuracy to what people think is the future (though Back to the Future did an uncannily good job with their Cubs World Series prediction), you could still experience what people believe the future will be like, from flying DeLorean’s to those hover boards we never got when we were supposed to.

Obviously, none of this will truly take you to the past or the future. You will still physically be present in the world of 2017, but, the real question is, does it make a difference? If you experience something that is so real to your senses that it may as well have been, does it really matter if it actually happened? To some people it might, but to many others it does not.

If you could use virtual reality to experience say, sky-diving, would it not be the same as actually sky diving so long as it was realistic? The same notion can be held to the idea of pseudo-time travel through virtual reality. Maybe you didn’t really travel in time back to the Crusades, the American Revolution, or one of the World Wars. Maybe you didn’t really travel to the year 3000 and witness the future. But if the experience is real enough, is it not the same as though you actually did?

It is something that is yet to be seen, but highly anticipated by all. Moreover, the potential is nearly limitless. With this kind of virtual reality, will the way history is taught change? Will students be able to experience renditions of the history they are learning first hand? And what of religion? Rather than read the Bible or the Quran, will you be able to experience every part of it from the perspective of someone who was actually there? You very well could, provided that a programmer desires to make it so.

In the end, virtual reality is one of the most exciting things to happen to humanity in decades. And though we may never truly travel through time, who knows what we could one day experience through the power of the technology we now have. I’ll meet you there or, rather, then.

Original post on SpiritualVR.com

Mindfulness and Technology

girlsky1

Originating in Buddhist philosophy, mindfulness is becoming increasingly popular in the West as the incidence of anxiety, depression, and stress-related disorders plague the undercurrent of our fast-paced industrialized way of life. Recent scientific research on mindfulness has demonstrated beneficial effects on several holistic aspects of personal health, including the mind, the body, and behavior.

Mindfulness meditation has been proven medically effective to decrease stress and improve well-being when practiced consistently. Yet many people still struggle with the concept or application of mindfulness-based therapy. A new wave of delivery is emerging which is combining this ancient practice with modern technology to bridge the gap and appeal to a modern generation of meditators. Studies show not only relaxation, but important shifts in cognition, emotion, biology, and behavior that may work synergistically to improve health. There is also emerging evidence that mindfulness training is associated with greater meaning and peace in one’s life (spirituality), as well as enhanced relationships with others (Carmody et al., 2008Carson et al., 2004)

Imagine you are sitting peacefully on a beautiful beach. You can hear seagulls against a backdrop of pebbles clinking together with each breaking wave. You take deep belly breaths and listen to your meditation teacher as she sits beside you and guides you through the film roll of anxiety and consciousness unfolding behind your eyes. Now imagine that you take off your virtual reality headset to discover you in fact never left your own living room (and saved hundreds of dollars on a flight to a meditation retreat in India.) This is an example of one scenario that modern entrepreneurs are envisioning the marriage of mindfulness and technology to enhance the effectiveness of well-being and relaxation intervention. Virtual reality devices can be combined with health tracking technology such as Provada Health‘s iOS app; “…incorporated into (the) app (is) the ability to link health-tracking wearables, such as the Apple Watch, to quantify the effects of a meditation session on, for example, your resting heart rate. Or look at how your sleep is being affected by taking time out to meditate.”

Modern gaming technology is another avenue where it seems there is potential for mindfulness to be cultivated. Take for example one gaming app available via Play Store called ‘Pause,’ which was created through the principles of mindfulness meditation and Tai Chi. The creator Peng Cheng explains, “It started with my own severe experience of stress and depression. I gave myself 6 months, I practically didn’t do anything but I meditated and practiced Tai Chi with the goal to do nothing but staying in the here and now as much as possible.” The simple game involves a little blob which follows your finger across the screen and facilitates focused awareness by growing in size as you maintain a slow concentrated speed. “Most of our stress only exists in our head and absorbs all our attention. To break this pattern, I need to focus on what is physical and tangible and actively put my attention in the moment.”

Cultivating focused attention in the present moment is the core foundation of mindfulness practice preached hundreds of years ago, in ancient India, and today via a squiggly blob on a hand-held screen or through a high-tech headset. Proper use of technology has the capacity to transform the quality of our lives and the delivery of ancient therapies such as mindfulness which are being lost on a section of the modern generation unaccustomed or afraid of ‘spiritual mumbo jumbo.’ Many trials of research have found that people with higher levels of mindfulness – even without “formal” meditation training – report feeling less stressed, anxious and depressed, and more joyful, inspired, grateful, hopeful, content, vital, and satisfied with life (Baer et al., 2006; Brown & Ryan, 2003; Cardaciotto et al., 2008; Feldman et al., 2007; Walach et al., 2006).

Another benefit of mindfulness is the ability to recognize and accurately label emotions (Analayo, 2003). More mindful people appear to have a greater ability to control emotional reactions in the middle part of the brain (the amygdala and the dorsal anterior cingulate cortex [ACC]) by engaging the front part of the brain (the prefrontal cortex [PFC]), which is associated with attention, concentration, and emotion regulation. This means when you’re practicing mindfulness you’ll better be able to control your emotions and correct unpleasant mood states.

Believe it or not, there is increasing scientific evidence to support the therapeutic effect of mindfulness meditation training on stress-related medical conditions, including psoriasis, type 2 diabetes, fibromyalgia, rheumatoid arthritis, chronic low back pain, and attention-deficit hyperactivity disorder.

Several new technologies, including brain imaging, wearable tech. and virtual reality, are being used to look at and extend the potential health benefits of mindfulness. Finally, research is beginning to prove what mindfulness practitioners have known for centuries…that greater focus, awareness, acceptance, and empathy can make for more flexible, adaptive responses to stress, which, in turn, can help free us from suffering and realize greater well-being & happiness.

Original article