Unseen Media

Overview

My team and I at Unseen Media engaged in a lengthy research project in an attempt to meaningfully bring augmented reality technology into gaming. We researched, designed, and built a mystery-themed board game that incorporates augmented reality technology into the game experience.

For our first game, we designed a mail-to-home box that players open to reveal “clues” that help them solve the mystery of “what killed social media influencer, Charlotte Depsi?”. Combining the physical “clue” pieces in front of them with our web-based AR tools that players can access through their online “agent profile,” players slowly unravel the story as they move through the game.

This project evolved out of an idea to build a platform that incorporated AR technology into escape room games in mid 2017. Through over 100 interviews with potential customers, we eventually landed on a mail-to-home game that does not limit players by location or by time. The mail-to-home concept opened our idea to a larger gaming market.

To date, we have participated in NYU’s Fall 2017 Prototyping Fund, NYC Media Lab’s inaugural XR Startup Bootcamp, the XR Beta Residency Program at the RLab in Brooklyn Navy Yard, and presented our prototype at the Augmented Reality Expo 2019 in Santa Clara.

As a culmination of everything we accomplished during the XR Bootcamp, teammate Brian Hui pitched our product to an audience of over 300 at the Exploring Future Reality conference at NYU Kimmel Center. Video of our pitch can be viewed below.

My Role

For this project, I served as a product designer and researcher.

Customer Discovery

Before building or designing anything, we set out to gather as much customer discovery research as possible. We did this by conducting over 100 interviews with potential customers and taking extensive notes.

Overall, we received a lot of positive feedback about our idea, but it became clear that our original proposed game concept did not have the level of scalability and market reach that we desired to achieve. After analyzing a couple of responses we got from interviews suggesting we look into games that could be played at home and reach a wider, we began testing our original concept vs. a tabletop mystery game concept that also heavily utilized augmented reality to create an alternate dimension for the game.

It quickly became clear that an at-home mystery game provided the scalability we desired and leveraged the power of augmented reality while still achieving the same level of engagement from customers. Overall, we found that customers valued:

• Convenience

• Cost

• Story

We pivoted, and went full steam ahead to build a prototype of an augmented tabletop mystery game.

Early Prototypes

Early prototypes of the game were a means of observing how users interacted with a semi-linear storyline communicated through a series of puzzles. They also helped us to understand how users would interact with augmented reality elements that were woven into gameplay. Overall, we saw that:

• Augmented reality elements required some hand-holding

• Careful consideration in puzzle design was needed to keep users from jumping ahead in the story

• Puzzle elements that became overly frustrating caused people to lose interest

Competitive Analysis

In order to better understand ship-to-home game experiences, we conducted a competitive analysis by playing similar products such as the ones available from Hunt A Killer, The Mysterious Package Company, and Breakout. We found that:

• Many games take advantage of web and social media platforms to bring the story into the real world

• Physical components always begin the experience, but clues can jump across many mediums

• Players can discover narrative information in a non-linear fashion, but password walls and code unlocks keep players from skipping too far ahead and missing pieces of information

• Tangible game objects are satisfying to interact with

Designing for Augmented Reality

Augmented reality is still an emerging medium, and has its limitations. A lot of people have never interacted with an augmented reality interface before. This presented a unique challenge for us in that we needed not only to build intuitive game concepts in AR, but we also needed to introduce our audiences to augmented reality interaction for the first time. In designing AR interfaces, we stuck to the following principles

• Simplicity is key

• When in doubt, offer text instructions on screen

• Consistently remind players that movement through space is possible

Prototype: The Dyatlov Files

It became clear during our customer discover phase that customers valued a story-driven experience above all else. Since our team was made up of designers and developers, we played to our strengths and selected a true crime story to lean on for our first iteration of this product.

The Dyatlov Pass Incident was an even in which nine hikers mysteriously and inexplicably died during a hiking expedition in Russia. We took elements of this story and built out an extensive mind map that allowed us to create puzzles that were based on, and rewarded, story elements.

Testing

The first prototype was tested through short, unmoderated gameplay sessions at various events hosted at the RLab. Users included AR/VR enthusiasts, college students, casual gamers, and NYC-based entrepreneurs. Through these user testing sessions, we learned: - Users were highly engaged by story elements - Tactile puzzle elements were appealing to users - The Unity App we were using for AR limited our customer base: users on older phone models were not able to access the content - Web elements were easily navigated by users

• Users were highly engaged by story elements

• Tactile puzzle elements were appealing to users

• The Unity App we were using for AR limited our customer base: users on older phone models were not able to access the content

• Web elements were easily navigated by users

Iteration

Through many hours of playtesting and discovery, we fleshed out the details of the game and learned what did work and what did not work. Some key takeaways were:

• Envelope and paper feels cheap

• Augmented reality did not fit into this story (1950s Russia)

• Players wanted more hands-on elements and less reading

Interface Mockups for AR

A unique challenge of designing for AR is properly communicating spacial requirements and interactability through traditional wireframing techniques. I had to learn how to communicate AR interactions when designing many of the interfaces used in this game, and employed a lot of visual methods to do so. Of course, there is nothing better than keeping a constant and clear channel of communication with the development team every step of the way.

Much like the early days of computers and smartphones, AR is still a new technology and not all interactions are intuitive for users. I learned to draw from real-world names, symbols, and interactions in order to slowly introduce AR elements. As time goes on and the customer base becomes familiar with the expected interactions required by AR, the interface, too, can be simplified. But for now, things need to be explained a little bit more, and that’s okay.

Prototype Two: Charlotte Depsi

Once we learned everything we could from the Dyatlov Files prototype, it was time to move on to a higher fidelity “boxed” prototype. For this, we hired a writer to provide a mystery story that better fit with the game elements we were hoping to use.

The Charlotte Depsi game saw iteration not only in the physical game pieces and the gameplay, but also in the augmented reality interfaces. We partnered with 8th Wall to develop purely web-based AR experiences that could be accessed by players anywhere on any device.

We also incorporated multiple web elements and web puzzles into this version of the game, which gave players the feeling of knowing there’s more out there than just the contents of the box they were sent. Players were also given their own Agent Portal where they could track their progress throughout the game.

Feedback from Demos and Conferences

My team and I were lucky enough to heavily involved with the AR/VR community in NYC through our connections with RLab. Thanks to this, were were able to bring our prototype to many different game conferences, media expos, game jams, and playtesting sessions and gather feedback from users in real time.

Since augmented reality technology is new to many audiences, it is important to note that some of the excited responses we got were a reflection of the technology itself, and not the product we designed.

My preferred method of testing this experience was unmoderated, closely observed testing sessions that were followed up by user interviews. Since the product itself was meant to be played through unmoderated, I found that the best feedback came from watching users interact with the product on their own. If a player got hung up at a certain point in the experience, I knew we had a problem that needed to be addressed in the game. Overall, I learned:

• Users will automatically mimic “pinch to zoom” iPhone gestures instead of moving closer to an AR object

• AR framed as CSI-style hacking tools made the experience really enjoyable for players

• There was opportunity for potential partnerships with licensed content or educational environments

• We needed to add a “?” modal for additional help with AR elements for those who needed it

• It was always safer to add a clear, written prompt instructing a user to move to the next step and take a step out of the immersive environment rather than make them guess about what to do next

Final Product

Our final product was a ship-to-home box that provided everything required for a small group of friends to learn the story of who killed Charlotte Depsi. The overwhelming majority of players (91%) were able to get through the experience beginning to end, and we received very positive feedback from those who played the game.