Analysis of a Scene: Building Our First VR Prototype

by Guest Author Eiran Shalev, CTO, NewPathVR

When Lisa Padilla first approached me with her vision for NewPathVR, I was immediately inspired by what we could achieve. Imagine the possibilities to not only unlock the hidden power of the brain but to empower our users to look inside themselves and find a way to improve. To feel better. Gain the strength to refine, and then to share that with others. While VR may be many things, to the folks at NewPath, it is the best tool to reprogram the brain for success. We huddled around and researched hundreds of papers where studies demonstrated how positive reinforcement, perception, and sensory filters can influence our behavior, as well as our memories.

Research and Design Phase
We knew we needed a prototype to prove VR as the right medium for spiritual growth, but what platform could serve us best?. Was this going to be a seated experience? How much interaction should it have? And what to build? For instance, did you know that if you perceive yourself as taller in VR, it actually makes you feel more confident during and after you remove your head-mounted display (HMD)? As it turns out, this is very true. Our discussions turned towards identity, and how to connect our users with their VR self. In gaming, this is called your “player self”. You have three “selves” actually. The first self, the real you, is what you do outside of games or VR experiences – ie your life: work, job, family, etc. The second self is the you that plays the game using peripherals, and experiences the content through the point of view of one or several characters in the game. You become a “player self” and share characteristics with the “game self”, but you are not the character. The third self, the “game self”, is the content’s avatar that represents you, and has a role to play in the content’s story or scenarios that your avatar experiences. By witnessing the story, and in some cases, by making choices for your avatar “game self”, your “player self” gets to experience those same emotions, and thus, share those same experiences. The cool thing about VR is that the player and the game self boundaries become blurred, such that you feel as if you are literally inside the content, and you feel much closer emotionally to the experience then you would be if you were observing the content through a monitor or TV screen. Keep in mind that taking an experience designed for a flat screen does not merit porting it to VR. All content in VR should be unique and specifically designed to transform and empower the user.

When building a VR experience, game mechanics that are based on challenge-reward systems create much more value for the user if they incorporate your senses. Adding a 360 visual experience may not be enough to trigger personal, real-world change. Adding 360 audio to that experience brings us closer, but is still not enough. By adding the ability to use your body, such as walking around and touching virtual objects, to influence the content, our team realized that we could create a world where consequences could have just as much impact on the “player self” as incentives. What’s more, if we incorporate at least 3 senses, this combination activates the memory centers or the brain. With the right experience, a player may create an association between something he/she experiences in VR, and a similar experience in his/her real life.

So, we knew we had to make a room-scale experience, and we knew we wanted it to leverage game mechanics that could change a player’s mood. The obvious choice for platform was the HTC Vive. But what about the content? I volunteered that for a prototype, we should keep things simple and demonstrate that we could achieve a basic goal. We wanted to transform a user’s mood from a negative or an indifferent position into a positive one. However, going through a sequential experience in VR, will usually not improve your life on exposure alone. In the real-world (or what we perceive to be our reality), we can usually learn any skill and master it, by practicing it over and over again. In our VR prototype, we needed to do the same. We decided that if we could create content that would teach our players some moral or zen-like lesson. A takeaway. But then also provided an opportunity to apply it, then we could create real personal growth.

Our “Self-Compassion Buddy” vision was born. In our prototype, we essentially introduced our users to their avatar self by literally creating a virtual mirror. The system tracked each of the user’s arms and mimicked his/her movement through the avatar that was reflected in the mirror. Our research showed that in order to strengthen the bond between the player-self and the avatar game-self, our users would need to interact with their mirrored reflection for approximately 70 seconds. This seems like a long time for a prototype, but in a future, commercial version of our product, the interaction could involve a game mechanic with a reward incentive. Next, we focused on the story ingredient. We did not require an elaborate story to demonstrate our vision; only a simple scene based on some narrative context. I believe that in order to create positive change or at least to invoke positive feelings, you need to have contrast, and that means placing the user into a negative situation – for a very short time of course. Then, follow it up with a positive environment filled with good energy. By placing the user into a slightly distressed state, and then moving him/her into a comfort zone, you can generate a sensation of emotional gratification. Games also apply a similar approach when they create a difficult challenge, in which a player must learn a new skill to overcome it. Once the skill is learned, the obstacle is easily navigated, and the player moves on to claim his/her reward. But more importantly, in your own life, when you undergo challenging times, and not only survive them, but learn to be stronger as a result of them, you then undergo positive change within yourself. Through overcoming these challenges, you may either improve your level of independence and self-sufficiency, or you may grow more carefree by successfully navigating stress and becoming familiar with it.

The Prototype Phase
We applied this to our “self-compassion” prototype. Imagine being immersed in a dirty, poorly-lit, virtual environment that exhumed negativity. You find yourself staring at your reflection in a mirror. You move, it moves. After a lengthy exposure to your reflected avatar, your avatar aka “buddy”, starts moving independently of you. It steps out of the mirror. Charges at you, invading your personal space. The result. You start feeling threatened. Your “inner bully” points his red finger at you and verbally abuses you, calling you “a loser…and a failure in life”. After a few moments of this, your brain switches to panic mode – a sort of fight-or-flight response. We kept the user in this state for about 7-10 seconds before interrupting the experience with another friendlier avatar. Any longer than that and we would have risked spoiling the whole experience and alienating our user.

buddy1

The friendlier avatar, a nurturing female guide appears and rushes to your rescue. Freezing the bully in action, she explains to you that the bully is you. Recap: by belittling yourself, you lose confidence in yourself. The female guide then offers words of encouragement to rebuild your confidence. Her words manifest in a new scene in which soap bubbles shower from the sky. Some of the bubbles contain cute gifts such as adorable stuffed animals and pets. We introduced a bit of the fun factor in this scene. When the user pops one of these soap bubbles containing a gift, the female avatar aka your “guide” offers words of encouragement with a positive message, ie: “Lots of people care about you.” Each time you pop a bubble, the gift item inside goes into your collection, and new positive words materialize. In the next scene, you have an opportunity to apply these gifts and redeem yourself. We call this the “pay-it-forward phase.” You observe three couples, standing at surrounding points around you. These couples each reveal a virtual buddy figure matched with his respective inner-bully avatar. Similar to your initial case, the bully verbally terrorizes his victim. This continues in an endless cycle, until you interrupt the buddy, and hand out gifts from your previous scene’s collection. Each time your gifts are received, the words that were initially associated with the gift, retrigger audio playback again. The buddy’s inner-bully vanishes and the buddy then begins playing the ukulele. When all three avatar buddies receive their gifts and play the ukulele, a well-known song begins playing, lifting the mood even more and embracing a sense of relief and closure. The three playing buddies merge into one buddy enclosed behind a giant mirror. And you are once again faced with your own buddy reflection. The simulation ends when the environment cross-fades into a sunny sky, and you find yourself standing on Cloud Nine. Literally.

Post-Analysis Phase
We quickly discovered that when a user goes through this VR experience, they feel better coming out, then going in. Our research also suggests that if we had ended simulation after the initial soap bubbles were popped, gifts received, and words of encouragement heard, then the effect would have been short-lived. By adding the “pay it forward” scene, where the user returns the favor and gives a gift to his surrounding buddies, we essentially teach our users to apply their acquired skills and pay them forward. To share. Therefore, our users resolve to help themselves, feeling a sense of contribution and meaningful value. Generosity in VR impacts our users in the real world and has longer lasting effects on their mood. It builds confidence and self-love.

buddy3

Our biggest challenge in building this first prototype was testing it. I relied on using Unity as our main game engine development tool. Unity not only has established partnerships all over the world, and is compatible with 27 target platforms, but it is also free to use for development. Due to our limited resources and limited access to the HTC Vive, I ended up building our prototype using game object placeholders to represent both the HMD and the two game controllers. I then parented OpenVR’s game controller objects under these placeholders. When I repositioned the placeholder game objects in Unity’s simulator, I was able to estimate fairly accurately how the HTC Vive’s game controllers’ movement would impact the VR environment around them. On a weekly basis, under limited time and limited test access to the shared hardware, I methodically validated and tested our experience on the Vive hardware, tweaking and improving our prototype step by step. Unity is not quite WYSIWYG (What You See Is What You Get) when it comes to the built-in development simulator. That’s why end-to-end testing is so important, especially in VR. I set up a mode that would allow me to switch between my mocked game objects and the hardware game objects in the scene project. Doing so, allowed me to execute a series of tests on our shared hardware device while continuing development in our mocked Unity environment.

Our second challenge was getting our 3D buddy avatar to move correctly in our initial mirror reflection scene, used to build an identity association with your inner buddy. I solved this by building a mimic-engine that tracked the delta positions of the mocked, game-controller-object placeholders. The engine then inverted these vectors and applied the new deltas to a basic, rigged stickman model. I added constraints on the limbs of the stickman and locked the lower limbs so that only the upper body would be affected to move freely. And it worked. Additionally, since the Vive is a room-scale experience, the position and orientation of the stickman (aka our buddy reflection) needed to map to my HMD game object’s position, such that when I moved left or right, my reflection (facing me) would move in the opposite direction. And because the mirror image itself has borders all around, our 3D stickman was piped through a render-texture camera, that projected the image onto a 3D mirror game object as a texture. The mirror game object itself had no reflection, but projecting the stickman as a texture on top of it, gave the illusion of a mirror reflection.

No matter how you choose to implement your own VR project, remember that VR is highly immersive. Due to VR’s transformative nature, the underlying purpose of your content should support a key responsibility for contributing to social goodness, and hopefully, empower our users to live more fulfilling lives.

Eiran Shalev is an experienced technical, hands-on leader with 18 years of professional expertise overseeing top teams on mobile, social, and web technologies for products ranging from multi-player mobile & social games, to streaming video ads to interactive television and more. He comes from Disney Interactive, where he established the technical vision, and helped to scale and deliver Disney’s mobile technology platform to all game studios. Before that, he spent time at Koolbit, Kabam, and RockYou! He has built more than 50 games. He joins NewPathVR as CTO.

Five Reasons Out-of-home Advertising is Gaining Momentum

Out-of-home advertising is projected to grow in 2014 and in the years beyond, thanks to advances in flat screen technology and digital displays. New devices are spurring the creation of eye-catching ads in public areas, causing marketers to adjust their ad campaigns and marketing strategies. In particular, there are five reasons why out-of-home advertising is gaining momentum.

High-quality Video Screens

Video screens that are durable, thin and display high-quality images are changing the way consumers view information. Digital devices can replace banners, posters and other print media that once dominated out-of-home advertising. Many of these digital devices have audio features, adding another dimension to ad campaigns that can capture consumers’ attention. Also, high-quality video screens can be used to feature multiple ads, making them more versatile than print ads.

Interactive Advertising Features

Print ads rarely have any type of interactive features, but digital devices can have touchscreen options to gain consumers’ attention and generate leads. For instance, a drawing to win a prize can be added to a digital device, allowing consumers to use touchscreen features to enter their information. Other interactive features such as games, which can be projected to wide audiences, also help in grabbing consumers’ attention.

Lower Advertising Costs

Since digital devices can display any number of ads, they cut down on the cost of advertising. Print ads are typically good for one campaign only. Posters and billboard signs have to be taken down and re-printed if there are changes to ads or prices. Digital devices don’t have to be taken down, remounted or re-designed. Instead, their programing has to be changed slightly in order to display new images, ads or promotions. This can save advertisers a lot of money over the long run.

Higher Market Penetration

Given the population density of most major U.S. cities and other major metropolitan areas around the world, out-of-home advertising has the power to reach large numbers of people. This helps advertisers penetrate their target markets, by displaying ads in the high-traffic areas that their customers frequent. For instance, ads for luggage or travel-related products displayed in busy airports have the power to capture the attention of passersby interested in new luggage or travel gear.

Captive Audiences

Even though digital devices are revolutionizing out-of-home advertising, traditional print ads and banners are still proving effective in areas with captive audiences. For instance, fans at ballparks are likely to see banners on outfield walls and in stadium hallways. Captive audiences are a prime target for out-of-home adverting, because marketers have a wide audience to promote their products, services and brand image to.

As the out-of-home advertising industry continues to evolve, consumers will see new types of ads in waiting rooms, train stations, airports and other public areas.  Given the foot traffic in public areas, ads in these places have the potential to capture customers’ attention and generate leads for future sales.

Resources:

(1)    The Economist: Out-of-home advertising — Billboard boom
(2)    Forbes: Out Of Home Ads Still Growing
(3)    The Wall Street Journal: Clear Channel Outdoor Showcases Power of Integrated Out-Of-Home and Mobile Advertising at Cannes Lions 2014
(4)    The Irish Times: Boom in out-of-home advertising as banks increase their spend by 200%

Being Technology Forward (aka a Glasshole)

San Francisco #throughglass

I’m a little surprised that people seem to have widely varying opinions about Google Glass. While I’m wearing it, people blurt out “Glass!”, “Google Glasses!”, “Cyclops!”, “What is that?”, “Terminator!” and “Glasshole!”. People stop me everywhere — grocery stores, bars, the street, my doctor’s office. Wearing them is an invitation to be asked about them and I don’t mind. I’d like people to understand them better. In fact, I like when people try it on, with a couple of commands, they get the Glass experience and their eyes light up like children. Even my teenager, who, despite being so dependent on her iPhone, rejects technology…even she couldn’t hold back saying “That’s actually pretty sick, mom” and lets me wear it in public.

The press is all over Sarah Slocum’s use of Glass and her run in with some people who didn’t want to be taped in San Francisco’s Lower Haight district. I’ve been to that area, there are friendlier neighborhoods. However, just like the poster child for wearing Glass while driving, Sarah has been experiencing some early-stage device use hatred. We can get philosophical as to why: they have exclusive distribution and an unwieldy price, a clear and noble use for them hasn’t been communicated, Google has been characteristically quiet about its controversial product.

Nearly everybody has a photo and video capable phone, and nearly all of them are connected to Instagram, Facebook, Twitter, YouTube and other networks. Unlike Glass, there is no visible indication that they are being used for photos or videos. But in this town, we don’t care what you do. The things I’ve seen. Buy me a drink sometime and I’ll tell you stories. I will capture the world with Glass, with as much permission as I’ve asked for in the past with a camera, audio recorder or similar and that’s where I stand. At a time when we distrust the government because they’ve been tapping into our lives without our permission, Glass is facing some unwarranted displaced fear. It’s a smartphone you wear like glasses, not a futuristic tracking device.

It's not science fiction

And although Robert Scoble recently said he is “skeptical” about Google pushing forward with the device, I think Glass has a functional future, and I love it, however I could see another device outsmarting, out-designing, and out-penetrating Glass.

So, what’s using Google Glass like and what does it do? I’ve had Glass for 6 months, so I’ll tell you what I do with it. Some of what I do on my phone I can also do with Glass. I send and receive text messages (like sending a grocery list to my husband he can pull up on Glass, himself, and not have to take his phone out of his pocket at the grocery store), take and share photos and videos (there are multi-shots, short/long videos, and a community of Glass photographers taking interesting pictures), look up the weather (by voice) and anything else you want on the Internet. Sure, like with any new technical device, I can take photos or videos without people knowing, but let’s be real, wearing Glass is NOT discreet. And I’m no jerk, if you’re interesting enough to tape up close, trust me I’ll ask permission.

Will you get Google Glass when the price comes down and it’s made available to everyone?

Selected Lisacast Shows

I’m rebroadcasting a few shows, in case you missed some of the better ones:

Lisacast with Guest Vipul Vyas of Skewz.com
Lisacast with Guest Jeff Robbins of Lullabot
Lisacast with Guest Juan Carlos Soto
Lisacast with Guest Marla the FlyLady
Lisacast with Guest Marcien Jenckes, Voxant
Lisacast with Guest Michael Leach
Lisacast with Guest Jon Hammond
Lisacast with Guest Alan Levy, BlogTalkRadio
Lisacast with Guest Steve Gal of ProQuo.com
Lisacast with Guest Daphne Kwon
Lisacast with Guest Liad Agmon of Delver
Lisacast with Guest Gina Bianchini, CEO of Ning.com
Lisacast with Guest John Battelle
Lisacast with Guest Don Pierce of Micoy
Lisacast with Guest Elad Yoran of KoolSpan
Lisacast with Guest Jeff Crigler, Voxant CEO
Lisacast with Guests from NY to South Korea
Lisacast with Guest K. Daniel Glover of Aircongress.com
Lisacast with Guest Noam Lemelshtrich-Latar
Lisacast with Guest Thomas Frostberg, Editor-in-Chief and Co-Founder of Rapidus
Lisacast with Guest Sean Wise of WiseMentorCaptial
Lisacast with Guest Rafe Needleman of Webware
Lisacast with Guest Rafael Martinez Alequin
Lisacast with Guest Lee Dryburgh, eComm
Lisacast with Guest Thomas J. Buckholtz, PhD
Lisacast with Guests Claire Ulrich (Le Monde) and Thierry Bezier
Lisacast Interview: Dr. Wong
Lisacast Interview: Dr. Wang, Georgia Tech
Lisacast Interview: David Fox
Lisacast Interview: Dr. Julian Vincent, University of Bath

Lisacast interviews

Do you have a suggestion for a guest on Lisacast? Email me.

SQL Saturday Silicon Valley Event

Here’s a great upcoming event. Lisacast is going. Hope to see you there!

SQL Saturday Silicon Valley

SQL Saturday Silicon Valley with Ross Mistry (Microsoft) and Mark Ginnebaugh (DesignMind)

February 23rd, 8:30 am to 6:00 pm
Microsoft Technology Center, 1065 La Avenida, Building 1, Mountain View, CA, 94043
The largest Bay Area SQL Server event is here again! SQL Saturday sessions include SQL Server 2012, DBA, Development, Business Intelligence, and Big Data. A great post-event networking reception includes raffle prizes! Even better, the event only charges a $10 lunch fee.

Speakers:

  • Angel Abundez, DesignMind
  • Kevin Boles
  • Carlos Bossy, Quanta Intelligence
  • Mitchell Bottel, Innovative IT Consulting
  • William Brown, Microsoft
  • Dan Bulos, Symmetry Corporation
  • Denny Cherry
  • Rob Farley, LobsterPot Solutions
  • Argenis Fernandez, Microsoft
  • Grant Fritchey, Red Gate
  • Mark Ginnebaugh, DesignMind
  • Janis Griffin, Confio
  • Mark Gschwind, DesignMind
  • Allan Hirt, SQLHA
  • Jason Horner
  • Phil Hummel, EMC
  • Rob Kerr, Blue Granite
  • Scott Klein, Microsoft
  • Kevin Kline, SQLSentry
  • Randy Knight, SQL Solutions Group
  • Julie Koesmarno, LobsterPot Solutions
  • Ami Levin, SolidQ
  • Denise McInerney, Intuit
  • Joy Mundy, Kimball Group
  • Dilip Nayak, CUDirect Corporation
  • Aaron Nelson
  • Joe Sack, SQLskills
  • Chris Shaw, Xtivia
  • Pat Sinthusan, NetApp
  • Mickey Steuwe
  • Mark Tabladillo, Mark Tab Consulting
  • Joseph Vertido, Melissa Data
  • Eddie Wuerch, ExactTarget
  • Wenming Ye, Microsoft

Also, join one of two $149 all-day, pre-conference trainings February 22nd Database Configuration and Tuning with Kevin Kline or Introduction to SQL Server Data Mining with Mark Tabladollo

Real World Database Configuration and Tuning
Speaker: Kevin Kline, SQL Server MVP and Director of Engineering at SQL Sentry

Kevin Klien

This multi-module full day seminar covers best practices for database and application design and configuration, implementation, maintenance and performance tuning.

This session has a special focus on IT organizations with large SQL Server deployments.

– Bare metal tuning of server and disk configuration
– Benchmarking performance
– Physical machine versus virtual machine deployment
– Database conceptual design and normalization through to physical deployment of databases, indexing and partitioning
– Understanding SQL Server query performance
– Identifying performance bottlenecks and resolving performance issues
– Automating as much as possible so that life gets easier!

Prerequisites: Basic understanding of SQL Server architecture and administration.

Kevin Kline has been a Microsoft SQL Server MVP since 2004 and is a noted leader in the SQL Server industry. His product designs have won multiple “Best of TechEd” and “Readers’ Choice” awards. He is a founding board member and former president of PASS – The Professional Association for SQL Server. He has written or co-written eleven books including the best-selling SQL in a Nutshell. Kevin contributes monthly columns to SQL Server Pro and DBTA magazines. Kevin is also a top-rated speaker at conferences worldwide such as Microsoft TechEd, the PASS Summit, DevTeach, Oracle OpenWorld, and SQL Connections. He tweets via @kekline and blogs at KevinEKline.com.

Introduction to SQL Server Data Mining
Speaker: Mark Tabladillo, Ph.D and SQL Server MVP

Mark Tabladillo

QL Server 2012 provides enterprise-level data mining technology, ranging from supporting individual analysts through scalable server solutions.  You will see some of the elements of SQL Server Data Mining from the Excel add-in to full-scale production examples implemented by Microsoft Business Intelligence technology.

Mark will explain data mining in a scientifically accurate way while making the technology practically accessible and even fun. This course is appropriate for people new to data mining technology.  However, feel free to bring your more advanced challenges and questions too.

Objectives:

– Provide a fundamental scientific framework for data mining
– Demonstrate the technology with the fundamental Microsoft data mining interfaces
– Offer practical advice in moving from desktop analysis to enterprise-level deployment

Technology:

– SQL Server 2012 Analysis Services (Multidimensional)
– Office 2013 x64 (Excel and Visio) with Data Mining Add-In
– SQL Server 2012 Integration Services
– SQL Server Management Studio
– SQL Server Data Tools

Mark Tabladillo is a SAS expert, Ph.D, and Microsoft MVP.   He helps teams become more confident in making actionable business decisions with the use of data mining and analytics.  Mark provides training and consulting for companies in United States and around the world. He also teaches part-time with the University of Phoenix, and supervised a dissertation with a graduate student living in Canada.

Contact URL: http://marktab.net/datamining