(Short version) Building a Mobile, Social, Location-Based Game in 5 Weeks

Editor's Notes

• #2: Introduce self and the experiment. Encourage people to find another session if they thought this would all be about coding something up as fast as humanly possible. This is a session about pretotyping - you can’t build it right before you find the right “it” to build.\n
• #3: The flipside of failing fast is knowing when to pivot - change direction - and following through on it. There’s an economic principle called the sunk cost fallacy - people are psychologically less likely to change direction if they have invested time (and money!) into something, even if that investment is now worthless. \n\nSo your aim is to do enough research to show what’s going to work and what isn’t early on, keep pivoting - drastically or gradually - until you find the sweet spot, where you have both data and instinct pointing to success. Even then, keep testing!\n
• #4: I’m going to walk you through a five week experiment a team of us conducted (two developers, one UX designer and myself, a product manager). Our aim was to build a smartphone game within the 5 week timeframe, but not just to build it - to prove it would work. We wanted to fail fast, and to pivot.\n
• #5: We started off with a load of assumptions, many of which we didn’t realise -were- assumptions.\n\nThe key here, and this is something I will mention again and again, is to figure out what you are assuming to be true in order for your game to succeed. You’re assuming it’ll be a great game, it’ll be fun, everyone will want to play it, the reviews will be awesome, and so on - but what else are you assuming about the people who will be playing it? About the way it will fit into their lives? About the way they play games right now?\n\nSmartphone games are an interesting case to dive into, because they really do fit into people’s lives in a way that a console or PC doesn’t. We started out by thinking about people’s needs when they are out and about with their phones, and problems we had personally run into.\n
• #6: This was our starting point. We love location and mobile, so we combined the two into a walking tour idea. It’s not actually a game yet, although we envisaged gameplay elements as being key to the interactivity. The basic idea was about allowing people to walk around their city recording an audio tour through our app, while travellers in the area could pull up the app to see the tours locals had created and follow them.\n
• #7: Talked to 10 people along Fisherman’s Wharf as well as phone interviews with 25 people who travelled a lot. Quickly learnt that we had made two very wrong assumptions. Nobody liked walking tours, and nobody cared about using their phone when they were travelling. In a very specific tech-savvy data-plan-free world, we might have been on to something, but the feedback we got was universally pointing us in one direction: away from this idea! And notice - we didn’t even start to test the gameplay elements yet.\n
• #8: How can you make the most of time in front of real people?\nTarget: Is your target player “everyone in the world”? Really? Whether it’s “people that buy games on smartphones”, “people who own a smartphone” or even “people who play Angry Birds addictively”, there’s some factor you can use to hone in on finding people who would actually, under the right circumstances, install your game. Even if you have nothing but an idea yet - hey, I’m going to do a game about fruit - you can find out more from real people that is super valuable in giving you context during the development process. And you can find out their likes and problems, which might give you a flash of insight into something new you could do.\n
• #9: \n
• #10: Step 2 - learning from feedback on the assumptions\n
• #11: We thought there was room for a location based mobile app that would pull together fellow users and arrange a ‘flashmob’ style event for strangers in a city, whether it was a get together or group tickets to a local attraction. Think of it as Groupon meets friend dating meets location.\n
• #12: To test this new style of assumption, we felt we needed mass data as well as more personalised responses. We continued to interview travellers, but also sent out a survey to 300 people to learn more about their solo travel experience and to ask directly if they would use this app. Surveys are one way of testing a product that doesn’t exist yet, although there are other means that I will get into a little later, such as setting up fake landing pages and driving traffic via AdWords.\n\nOne interesting fact about our survey: from people who saw the form, 98% filled it in. People love to talk about travel!\n
• #13: Let’s rewind. We’d talked to people directly and found out that loneliness was a big problem. Yet when we asked people en masse, anonymously, where there’s not even a hint of embarrassment, we found that half of them don’t have the problem at all. How could we have been so wrong?\n\nActually, as the statisticians here already know, there are various biases and issues at hand here. It’s perfectly possible that we got the two conflicting data points - and also possible that the two can coexist. In fact, 50 people said they did feel lonely. The question is whether that was good enough, combined with the 27% who said they would try the app.\n
• #14: One of the most useful things that came out of the work we had done so far was a building gut instinct among the entire team. We had become much closer to our target market and we had learnt a lot about what works and what doesn’t work for them. We’d all spoken to people directly and we’d all helped get our survey responses, which I think is important: especially in an engineer focused culture, it’s good for everyone to have the data, not just the project manager or CEO.\n\nAs a team, we realised our gut was telling us one thing: we were in the wrong place altogether. We were passionate about location and mobile, but not travel, and the mixed messages and lack of a clear “Yes!” moment in all our travel interviews were telling us not to continue down this path. Fail fast!\n\nSo we took a step back and looked at some of the other learnings and hypotheses we had about location based apps.\n
• #15: Step 3 - refocus. Trust instinct. Reframe world.\n
• #16: Aha! Now the good stuff, you say!\n\nFocusing on a location-based smartphone game let us widen our target audience and pull in some of the things we had learnt so far about people’s behaviour with phones while travelling. We wanted to bring in some elements of the real world to the game, so our game design was to have a set of minigames that could be played quickly while waiting in line, with a coupon as the reward if you got a particular score.\n
• #17: How many of you play games when you’re in line somewhere? I know I do, in fact our entire team did, which was part of the genesis of this idea. We wanted to take that game experience and make it special based on the location. \n\nHowever, there was a key assumption around distribution and uptake that this coupon idea centred around: that businesses would partner to give us coupons, leading to all kinds of scaling problems, or that we could get the coupons somewhere else. \n\nWe spoke to a couple of businesses and other companies which worked with coupons and discounts, and realised a crucial factor: businesses buy users, not ideas. “Come back with a million users”. This reframed our problem into one of figuring out the most compelling way to get people playing our game, and for them to keep playing it - and tell their friends!\n
• #18: ---\n
• #19: From storyboarding several ideas, and despite our personal assumptions and experiences with mobile games, we discovered an important thing: the compelling hook was a really big factor. When we tested out a paper prototype with no real reason to play, people were lukewarm, saying they would probably play it but weren’t sure why they would bother in the first place. This is a vicious circle a lot of mobile games can get into, where you build something that is in and of itself fun, but without the big press coverage and massive buzz, nobody will find that out in the first place.\n\nThere are many answers to this. Sometimes the compelling part of a game is a really core part of the game itself. Hitting stuff with birds that make cute noises is fun. Sometimes it’s great marketing. Sometimes it’s finding a way to tap into an audience that already exists. And sometimes it’s a case of finding a compelling part of the game that is enough to make people curious and want to try it. We found, by storyboarding different scenarios, that adding in the promise of monetary gain (or saving) was enough to make people want to try it out at least once. From then on, it’s our job to make the game fun enough that the once becomes ‘every time I’m in line’.\n
• #20: While we were testing out assumptions still, we did start building an actual game in code, as we thought this idea was going to go somewhere interesting. However, this was nowhere near ready to test. So how did we playtest a game we hadn’t built?\n\nAs our concept centred around minigames, we identified a few different types of games that we wanted to focus on initially, and backed this up with a survey to find out 50 people’s favourite mobile game types, knowing that we could easily expand our repertoire if needed. Games of skill, competitive games, and trivia games were three popular responses that we found it very easy to test - because similar games were already downloadable and playable on smartphones.\n\nThe first step of this test was to find a place where our target players would play our game: waiting at a coffee shop. We could actually test out two different hypotheses around boredom - would people play while in line, or after they had ordered their drink, in the few minutes it took to arrive?\n\nThe second step was to mock up the game experience using existing apps that we paid for and downloaded, and faked screens that showed a win or loss depending on the score the player got. \n\nHere’s what we learnt:\nGames of chance work well, as long as it’s balanced: the player has some chance of winning eventually, but doesn’t win immediately.\nEven people who were waiting together were not interested in competitive games.\nA game we had found a lot of fun in isolation was not so much fun in context; have you played that game where you drag out toilet paper as far as possible in 30 seconds? People were a little embarrassed to be playing it ‘in public’, though with friends it’s fine.\n\nWays of tying the game into the venue itself were generally well received (but we couldn’t test these at this stage). The two ideas we explained to users here were a trivia game based on the venue, or minigames that were specific, e.g. a Diner Dash style game if you were in a fast food joint.\n\nThe social aspects of friends’ high scores were important, but not as important as the coupon. So this wasn’t the viral hook we had hoped for.\n
• #21: How will we get our first users?\n\nWe wanted to try this “startup lean marketing” approach with a game, as we thought the results would be interesting. A lot of the normal ways of promoting a game won’t work if you haven’t finished building it yet, but this technique does. It’s used a lot by people starting up web products to see if there are even enough people interested in it in the first place.\n\nYou create a website for your game, explaining how awesome it is, even showing screenshots and listing all the benefits. For us, we focused on the concept of ‘play this game while you’re in line, and save money’. We then set up a Play Now link that led users to a form saying “Our game is being developed, but leave your email and we’ll give you exclusive access to our beta”.\n\nThen we took out online ads against the site. You can do this on any platform, from AdWords to Facebook ads to, I guess, mobile advertising or even a billboard. What you need to do is track EVERYTHING. The final email signups are going to be the really dedicated people - if you get a significant proportion of those, drop everything and build it already! But it’s not too likely that everyone will give you their email address. You can look at how many people visit the site in the first place - what’s your click through rate? Are you advertising to the right people? Then look at how long they spend on the site, and where they look, and whether they click through to try it or not. There’s some really interesting data in there, even for a game. \n\nAnother thing you can do here is obsessively A/B test your site. Change the ads, the wording, the text on the button (“Try Now”, “Play Now”, “Try for Free”, “Install”, “Beta Signup”...). Use stock photos, screenshots, drawings. You can test everything. Don’t get too carried away with it, but don’t overlook this as a good source of data too.\n\n
• #22: And it tells you a lot about whether it will sell or not when you do have it...\n\nBut be careful and don’t be evil!\n
• #23: Step 5 - prototyping\n
• #24: Back to minimum viable product. The concept we needed to test was the viral mechanics, and we focused on trivia as something we could trivially (sorry) implement.\n\nWas this a huge assumption? Yes!\n\nShould we have tested different types of game with the social mechanics? Yes!\n
• #25: Eh?\n\nHow does Twitter fit into all this?\n\nThe important part for us was to test whether we could get any kind of viral traction with a simple game. We realised that in picking trivia for our first game type, we had picked something that would work outside the mobile context, and decided to take advantage of this.\n
• #26: Concierge testing is another kind-of-lean concept. It’s that you don’t NEED to spend a long time creating fancy algorithms if you can do the job in five minutes with a person. If you can find people willing to interact with the person (even if they think it’s a computer), you’re in a good place to take it further.\n\nWe had a “Wizard of Oz” setup with our twitter prototype. Instead of spending an hour or so coding up a simple question/answer counter script, we started immediately with a human operator who took on a Frankenstein character (it was close to hallowe’en) and typed out questions and timers as if they were automated. The initial volume was low enough that we could count up the correct answers without needing any code, while in parallel we could code up a script to help when needed.\n
• #27: We found that opinionated questions were far more popular and engaging than simple trivia. What’s the best game of all time? had more responses than Who’s the President of the USA? or How many times a day do a clock’s hands overlap?\n\nbandwagon jumping - trying popular topics such as justin bieber and directly addressing people didn’t work either.\n
• #28: What would you have done?\ninteractive part. Ask people\n - What might you have tested\n - What else might you have done at this stage\n - would you just release and see?\n
• #29: \n