Analysis of a Scene: Building Our First VR Prototype

by Guest Author Eiran Shalev, CTO, NewPathVR

When Lisa Padilla first approached me with her vision for NewPathVR, I was immediately inspired by what we could achieve. Imagine the possibilities to not only unlock the hidden power of the brain but to empower our users to look inside themselves and find a way to improve. To feel better. Gain the strength to refine, and then to share that with others. While VR may be many things, to the folks at NewPath, it is the best tool to reprogram the brain for success. We huddled around and researched hundreds of papers where studies demonstrated how positive reinforcement, perception, and sensory filters can influence our behavior, as well as our memories.

Research and Design Phase
We knew we needed a prototype to prove VR as the right medium for spiritual growth, but what platform could serve us best?. Was this going to be a seated experience? How much interaction should it have? And what to build? For instance, did you know that if you perceive yourself as taller in VR, it actually makes you feel more confident during and after you remove your head-mounted display (HMD)? As it turns out, this is very true. Our discussions turned towards identity, and how to connect our users with their VR self. In gaming, this is called your “player self”. You have three “selves” actually. The first self, the real you, is what you do outside of games or VR experiences – ie your life: work, job, family, etc. The second self is the you that plays the game using peripherals, and experiences the content through the point of view of one or several characters in the game. You become a “player self” and share characteristics with the “game self”, but you are not the character. The third self, the “game self”, is the content’s avatar that represents you, and has a role to play in the content’s story or scenarios that your avatar experiences. By witnessing the story, and in some cases, by making choices for your avatar “game self”, your “player self” gets to experience those same emotions, and thus, share those same experiences. The cool thing about VR is that the player and the game self boundaries become blurred, such that you feel as if you are literally inside the content, and you feel much closer emotionally to the experience then you would be if you were observing the content through a monitor or TV screen. Keep in mind that taking an experience designed for a flat screen does not merit porting it to VR. All content in VR should be unique and specifically designed to transform and empower the user.

When building a VR experience, game mechanics that are based on challenge-reward systems create much more value for the user if they incorporate your senses. Adding a 360 visual experience may not be enough to trigger personal, real-world change. Adding 360 audio to that experience brings us closer, but is still not enough. By adding the ability to use your body, such as walking around and touching virtual objects, to influence the content, our team realized that we could create a world where consequences could have just as much impact on the “player self” as incentives. What’s more, if we incorporate at least 3 senses, this combination activates the memory centers or the brain. With the right experience, a player may create an association between something he/she experiences in VR, and a similar experience in his/her real life.

So, we knew we had to make a room-scale experience, and we knew we wanted it to leverage game mechanics that could change a player’s mood. The obvious choice for platform was the HTC Vive. But what about the content? I volunteered that for a prototype, we should keep things simple and demonstrate that we could achieve a basic goal. We wanted to transform a user’s mood from a negative or an indifferent position into a positive one. However, going through a sequential experience in VR, will usually not improve your life on exposure alone. In the real-world (or what we perceive to be our reality), we can usually learn any skill and master it, by practicing it over and over again. In our VR prototype, we needed to do the same. We decided that if we could create content that would teach our players some moral or zen-like lesson. A takeaway. But then also provided an opportunity to apply it, then we could create real personal growth.

Our “Self-Compassion Buddy” vision was born. In our prototype, we essentially introduced our users to their avatar self by literally creating a virtual mirror. The system tracked each of the user’s arms and mimicked his/her movement through the avatar that was reflected in the mirror. Our research showed that in order to strengthen the bond between the player-self and the avatar game-self, our users would need to interact with their mirrored reflection for approximately 70 seconds. This seems like a long time for a prototype, but in a future, commercial version of our product, the interaction could involve a game mechanic with a reward incentive. Next, we focused on the story ingredient. We did not require an elaborate story to demonstrate our vision; only a simple scene based on some narrative context. I believe that in order to create positive change or at least to invoke positive feelings, you need to have contrast, and that means placing the user into a negative situation – for a very short time of course. Then, follow it up with a positive environment filled with good energy. By placing the user into a slightly distressed state, and then moving him/her into a comfort zone, you can generate a sensation of emotional gratification. Games also apply a similar approach when they create a difficult challenge, in which a player must learn a new skill to overcome it. Once the skill is learned, the obstacle is easily navigated, and the player moves on to claim his/her reward. But more importantly, in your own life, when you undergo challenging times, and not only survive them, but learn to be stronger as a result of them, you then undergo positive change within yourself. Through overcoming these challenges, you may either improve your level of independence and self-sufficiency, or you may grow more carefree by successfully navigating stress and becoming familiar with it.

The Prototype Phase
We applied this to our “self-compassion” prototype. Imagine being immersed in a dirty, poorly-lit, virtual environment that exhumed negativity. You find yourself staring at your reflection in a mirror. You move, it moves. After a lengthy exposure to your reflected avatar, your avatar aka “buddy”, starts moving independently of you. It steps out of the mirror. Charges at you, invading your personal space. The result. You start feeling threatened. Your “inner bully” points his red finger at you and verbally abuses you, calling you “a loser…and a failure in life”. After a few moments of this, your brain switches to panic mode – a sort of fight-or-flight response. We kept the user in this state for about 7-10 seconds before interrupting the experience with another friendlier avatar. Any longer than that and we would have risked spoiling the whole experience and alienating our user.

buddy1

The friendlier avatar, a nurturing female guide appears and rushes to your rescue. Freezing the bully in action, she explains to you that the bully is you. Recap: by belittling yourself, you lose confidence in yourself. The female guide then offers words of encouragement to rebuild your confidence. Her words manifest in a new scene in which soap bubbles shower from the sky. Some of the bubbles contain cute gifts such as adorable stuffed animals and pets. We introduced a bit of the fun factor in this scene. When the user pops one of these soap bubbles containing a gift, the female avatar aka your “guide” offers words of encouragement with a positive message, ie: “Lots of people care about you.” Each time you pop a bubble, the gift item inside goes into your collection, and new positive words materialize. In the next scene, you have an opportunity to apply these gifts and redeem yourself. We call this the “pay-it-forward phase.” You observe three couples, standing at surrounding points around you. These couples each reveal a virtual buddy figure matched with his respective inner-bully avatar. Similar to your initial case, the bully verbally terrorizes his victim. This continues in an endless cycle, until you interrupt the buddy, and hand out gifts from your previous scene’s collection. Each time your gifts are received, the words that were initially associated with the gift, retrigger audio playback again. The buddy’s inner-bully vanishes and the buddy then begins playing the ukulele. When all three avatar buddies receive their gifts and play the ukulele, a well-known song begins playing, lifting the mood even more and embracing a sense of relief and closure. The three playing buddies merge into one buddy enclosed behind a giant mirror. And you are once again faced with your own buddy reflection. The simulation ends when the environment cross-fades into a sunny sky, and you find yourself standing on Cloud Nine. Literally.

Post-Analysis Phase
We quickly discovered that when a user goes through this VR experience, they feel better coming out, then going in. Our research also suggests that if we had ended simulation after the initial soap bubbles were popped, gifts received, and words of encouragement heard, then the effect would have been short-lived. By adding the “pay it forward” scene, where the user returns the favor and gives a gift to his surrounding buddies, we essentially teach our users to apply their acquired skills and pay them forward. To share. Therefore, our users resolve to help themselves, feeling a sense of contribution and meaningful value. Generosity in VR impacts our users in the real world and has longer lasting effects on their mood. It builds confidence and self-love.

buddy3

Our biggest challenge in building this first prototype was testing it. I relied on using Unity as our main game engine development tool. Unity not only has established partnerships all over the world, and is compatible with 27 target platforms, but it is also free to use for development. Due to our limited resources and limited access to the HTC Vive, I ended up building our prototype using game object placeholders to represent both the HMD and the two game controllers. I then parented OpenVR’s game controller objects under these placeholders. When I repositioned the placeholder game objects in Unity’s simulator, I was able to estimate fairly accurately how the HTC Vive’s game controllers’ movement would impact the VR environment around them. On a weekly basis, under limited time and limited test access to the shared hardware, I methodically validated and tested our experience on the Vive hardware, tweaking and improving our prototype step by step. Unity is not quite WYSIWYG (What You See Is What You Get) when it comes to the built-in development simulator. That’s why end-to-end testing is so important, especially in VR. I set up a mode that would allow me to switch between my mocked game objects and the hardware game objects in the scene project. Doing so, allowed me to execute a series of tests on our shared hardware device while continuing development in our mocked Unity environment.

Our second challenge was getting our 3D buddy avatar to move correctly in our initial mirror reflection scene, used to build an identity association with your inner buddy. I solved this by building a mimic-engine that tracked the delta positions of the mocked, game-controller-object placeholders. The engine then inverted these vectors and applied the new deltas to a basic, rigged stickman model. I added constraints on the limbs of the stickman and locked the lower limbs so that only the upper body would be affected to move freely. And it worked. Additionally, since the Vive is a room-scale experience, the position and orientation of the stickman (aka our buddy reflection) needed to map to my HMD game object’s position, such that when I moved left or right, my reflection (facing me) would move in the opposite direction. And because the mirror image itself has borders all around, our 3D stickman was piped through a render-texture camera, that projected the image onto a 3D mirror game object as a texture. The mirror game object itself had no reflection, but projecting the stickman as a texture on top of it, gave the illusion of a mirror reflection.

No matter how you choose to implement your own VR project, remember that VR is highly immersive. Due to VR’s transformative nature, the underlying purpose of your content should support a key responsibility for contributing to social goodness, and hopefully, empower our users to live more fulfilling lives.

Eiran Shalev is an experienced technical, hands-on leader with 18 years of professional expertise overseeing top teams on mobile, social, and web technologies for products ranging from multi-player mobile & social games, to streaming video ads to interactive television and more. He comes from Disney Interactive, where he established the technical vision, and helped to scale and deliver Disney’s mobile technology platform to all game studios. Before that, he spent time at Koolbit, Kabam, and RockYou! He has built more than 50 games. He joins NewPathVR as CTO.

Advertisements

Mindfulness and Technology

girlsky1

Originating in Buddhist philosophy, mindfulness is becoming increasingly popular in the West as the incidence of anxiety, depression, and stress-related disorders plague the undercurrent of our fast-paced industrialized way of life. Recent scientific research on mindfulness has demonstrated beneficial effects on several holistic aspects of personal health, including the mind, the body, and behavior.

Mindfulness meditation has been proven medically effective to decrease stress and improve well-being when practiced consistently. Yet many people still struggle with the concept or application of mindfulness-based therapy. A new wave of delivery is emerging which is combining this ancient practice with modern technology to bridge the gap and appeal to a modern generation of meditators. Studies show not only relaxation, but important shifts in cognition, emotion, biology, and behavior that may work synergistically to improve health. There is also emerging evidence that mindfulness training is associated with greater meaning and peace in one’s life (spirituality), as well as enhanced relationships with others (Carmody et al., 2008Carson et al., 2004)

Imagine you are sitting peacefully on a beautiful beach. You can hear seagulls against a backdrop of pebbles clinking together with each breaking wave. You take deep belly breaths and listen to your meditation teacher as she sits beside you and guides you through the film roll of anxiety and consciousness unfolding behind your eyes. Now imagine that you take off your virtual reality headset to discover you in fact never left your own living room (and saved hundreds of dollars on a flight to a meditation retreat in India.) This is an example of one scenario that modern entrepreneurs are envisioning the marriage of mindfulness and technology to enhance the effectiveness of well-being and relaxation intervention. Virtual reality devices can be combined with health tracking technology such as Provada Health‘s iOS app; “…incorporated into (the) app (is) the ability to link health-tracking wearables, such as the Apple Watch, to quantify the effects of a meditation session on, for example, your resting heart rate. Or look at how your sleep is being affected by taking time out to meditate.”

Modern gaming technology is another avenue where it seems there is potential for mindfulness to be cultivated. Take for example one gaming app available via Play Store called ‘Pause,’ which was created through the principles of mindfulness meditation and Tai Chi. The creator Peng Cheng explains, “It started with my own severe experience of stress and depression. I gave myself 6 months, I practically didn’t do anything but I meditated and practiced Tai Chi with the goal to do nothing but staying in the here and now as much as possible.” The simple game involves a little blob which follows your finger across the screen and facilitates focused awareness by growing in size as you maintain a slow concentrated speed. “Most of our stress only exists in our head and absorbs all our attention. To break this pattern, I need to focus on what is physical and tangible and actively put my attention in the moment.”

Cultivating focused attention in the present moment is the core foundation of mindfulness practice preached hundreds of years ago, in ancient India, and today via a squiggly blob on a hand-held screen or through a high-tech headset. Proper use of technology has the capacity to transform the quality of our lives and the delivery of ancient therapies such as mindfulness which are being lost on a section of the modern generation unaccustomed or afraid of ‘spiritual mumbo jumbo.’ Many trials of research have found that people with higher levels of mindfulness – even without “formal” meditation training – report feeling less stressed, anxious and depressed, and more joyful, inspired, grateful, hopeful, content, vital, and satisfied with life (Baer et al., 2006; Brown & Ryan, 2003; Cardaciotto et al., 2008; Feldman et al., 2007; Walach et al., 2006).

Another benefit of mindfulness is the ability to recognize and accurately label emotions (Analayo, 2003). More mindful people appear to have a greater ability to control emotional reactions in the middle part of the brain (the amygdala and the dorsal anterior cingulate cortex [ACC]) by engaging the front part of the brain (the prefrontal cortex [PFC]), which is associated with attention, concentration, and emotion regulation. This means when you’re practicing mindfulness you’ll better be able to control your emotions and correct unpleasant mood states.

Believe it or not, there is increasing scientific evidence to support the therapeutic effect of mindfulness meditation training on stress-related medical conditions, including psoriasis, type 2 diabetes, fibromyalgia, rheumatoid arthritis, chronic low back pain, and attention-deficit hyperactivity disorder.

Several new technologies, including brain imaging, wearable tech. and virtual reality, are being used to look at and extend the potential health benefits of mindfulness. Finally, research is beginning to prove what mindfulness practitioners have known for centuries…that greater focus, awareness, acceptance, and empathy can make for more flexible, adaptive responses to stress, which, in turn, can help free us from suffering and realize greater well-being & happiness.

Original article

Managing Television Choices (or “The 17 Things I Watch on TV”)

The Future of TV

choices.jpg

While it’s great to have many choices, we also need to be able to quickly hone in on those to exactly what you want, when you want it, and how.

Last I checked, the U.S. TV home received, on average, 189 TV channels. Despite this, the number of channels we actually watch each day is just about 17.

changing-channels-americans-view-just-17-channels-despite-record-number-to-choose-from

So we’ve narrowed it down to 17 channels, that’s only 408 hours a day of programming to weed through. Some of that you have to pay for, some is free — a differentiation I think is pretty important. When you already pay a hefty fee for your cable bill, and you probably do, the incidentals of renting movies each month, and buying series and seasons and trilogy collections you must have are not insignificant. Some is on Apple TV, some in Amazon, some on your cable provider, don’t forget Google Play, Hulu, Netflix and so on…just to complicate things, searching those services is different depending on which one you use:

tv-search-comparison

Different search results for “Amy Schumer” depending on service

Is the future consolidation or more fragmentation? While there will continue to be buyouts there are still so many opportunities for content producers so I see more fragmentation. But the providers, the TV manufacturers and Netflix’s need to iron out the discovery process to optimize for customers.

Form Factor

In the not-too-distant future, stand-alone TVs will be a thing of the past and wall screens will be commonplace. I’m looking forward to this as my husband and I cannot agree on what color to paint the walls, so someday we’ll just have floor to ceiling screens that we switch as we like…blue….yellow….blue….yellow….blue. I’m starting to see a flaw in my plan, but anyway.

interactive-wallpaper-screens

(Photo credit: Imagination Technologies Blog)

Here is a flexible TV screen from LG, pretty crazy, huh?

lg-display

LG Display

Suddenly this doesn’t look so far off, does it?…

harry-potter

 

TV All Around Us

What else happens with TV in the future? Netflix aired a series called Black Mirror and specifically Episode 2 called “Fifteen Million Merits” about a future world where media and attention are currency and TV is all around us.

Black-Mirror-15-million-merits

Black Mirror is a strange but thought-provoking show and Episode 2 shows us a scenario where media has taken hold like cement, binding us superficially and forcing us into ways from which we can’t effectively evolve, let alone remain sane within it all. What else do I watch?

“The 17 Things I Watch on TV”

Don’t give up your TV just yet. Some programming is pretty great. Plus there are some things I’ll always go back and watch again, and again. Here are 17 things (in no particular order) I suggest you watch on TV or streaming wherever you want.BaBlack Mirror

  1. Saturday Night Live
  2. House of Cards
  3. Game of Thrones
  4. Family Guy
  5. Boardwalk Empire
  6. Person of Interest
  7. Breaking Bad
  8. Mr. Robot
  9. Orange is the New Black
  10. Mike & Molly
  11. The Colbert Report
  12. Silicon Valley
  13. The Big Bang Theory
  14. Battlestar Gallatica
  15. Louie
  16. I Love Lucy

anigif_enhanced-767-1414179465-13

For more about the future TV, here’s a landscape snapshot from Venture Scanner and their recent post:

future-of-tv-visual-map5

Other big news from Nielsen was that music streaming was up 93% over last year. 93%!

Five Reasons Out-of-home Advertising is Gaining Momentum

Out-of-home advertising is projected to grow in 2014 and in the years beyond, thanks to advances in flat screen technology and digital displays. New devices are spurring the creation of eye-catching ads in public areas, causing marketers to adjust their ad campaigns and marketing strategies. In particular, there are five reasons why out-of-home advertising is gaining momentum.

High-quality Video Screens

Video screens that are durable, thin and display high-quality images are changing the way consumers view information. Digital devices can replace banners, posters and other print media that once dominated out-of-home advertising. Many of these digital devices have audio features, adding another dimension to ad campaigns that can capture consumers’ attention. Also, high-quality video screens can be used to feature multiple ads, making them more versatile than print ads.

Interactive Advertising Features

Print ads rarely have any type of interactive features, but digital devices can have touchscreen options to gain consumers’ attention and generate leads. For instance, a drawing to win a prize can be added to a digital device, allowing consumers to use touchscreen features to enter their information. Other interactive features such as games, which can be projected to wide audiences, also help in grabbing consumers’ attention.

Lower Advertising Costs

Since digital devices can display any number of ads, they cut down on the cost of advertising. Print ads are typically good for one campaign only. Posters and billboard signs have to be taken down and re-printed if there are changes to ads or prices. Digital devices don’t have to be taken down, remounted or re-designed. Instead, their programing has to be changed slightly in order to display new images, ads or promotions. This can save advertisers a lot of money over the long run.

Higher Market Penetration

Given the population density of most major U.S. cities and other major metropolitan areas around the world, out-of-home advertising has the power to reach large numbers of people. This helps advertisers penetrate their target markets, by displaying ads in the high-traffic areas that their customers frequent. For instance, ads for luggage or travel-related products displayed in busy airports have the power to capture the attention of passersby interested in new luggage or travel gear.

Captive Audiences

Even though digital devices are revolutionizing out-of-home advertising, traditional print ads and banners are still proving effective in areas with captive audiences. For instance, fans at ballparks are likely to see banners on outfield walls and in stadium hallways. Captive audiences are a prime target for out-of-home adverting, because marketers have a wide audience to promote their products, services and brand image to.

As the out-of-home advertising industry continues to evolve, consumers will see new types of ads in waiting rooms, train stations, airports and other public areas.  Given the foot traffic in public areas, ads in these places have the potential to capture customers’ attention and generate leads for future sales.

Resources:

(1)    The Economist: Out-of-home advertising — Billboard boom
(2)    Forbes: Out Of Home Ads Still Growing
(3)    The Wall Street Journal: Clear Channel Outdoor Showcases Power of Integrated Out-Of-Home and Mobile Advertising at Cannes Lions 2014
(4)    The Irish Times: Boom in out-of-home advertising as banks increase their spend by 200%

Who Listens to Lisacast

Lisacast listeners

You’re in great company. These are current demographic statistics about Lisacast listeners.

Married 96%
Single 4%

Male 67%
Female 33%

Median Income $50,000 – $74,999 40%

Age 35 to 49 63%

Caucasian 80%
African American 15%
Hispanic 4.5%

Sales and Marketing 19%
Entrepreneurs 15%
Senior managers 12%
Consultants 9%
Journalists 8%
Authors/writers 7.50%
Web developers 7.30%

A few of my most influential followers: Om Malik, Kara Swisher, Veronica Belmont, Google, iJustine, Michael Arrington, David Pogue, Whole Foods, Tony Robbins, Dooce, Arianna Huffington, Rachel Maddow, Starbucks Coffee, Anderson Cooper, George Stephanopoulos, Steven Johnson, Cory Doctorow, Facebook, Nicholas Kristof, Ana Marie Cox, Jenna Wortham, Ann Curry, Adventure Girl, Amy Jo Martin, Southwest, David Allen, Larry King, Dr. Drew, Brandon Mendelson, American Express, Yahoo, Amazon.com, Ford, Intel, Dell, Outlet, Microsoft, Cisco Systems, Home Depot, American Airlines, PepsiCo, Apple, Coca-Cola, Smithsonian, Nordstrom, Amnesty International, AT&T, McDonald’s, Nike, CBOE, Room to Read, World Wildlife Fund, Refugee Agency, Lowe’s, Greenpeace, Staples, MoveOn.org, The Vatican

Would you like to recommend a guest for Lisacast?

Would you like to listen to previous shows?