Making The Leap From Web To Mobile

Making the Leap from Web to Mobile Best Practices in Mobile User Experience ResearchAmy Buckner, AnswerLabKris Mihalic, Yahoo!UPA 2009, June 12, Portland

There are mobile phone subscribers worldwide4.1 BillionOnly of US mobile phones are iPhones (~5 Million)2%Social networking on mobile phones is growing Y/Y196%There are more mobile IM users than mobile business email users in the US1.4x

Mobile Market SummaryGlobally, there are 800 million cars, 850 million PCs, 1.3 billion fixed landline phones, 1.4 billion credit cards, 1.5 billion TV sets – and 2.7 billion mobile phones – in use. (Experian 2008) eMarketer projects that worldwide spending on mobile advertising will reach a total of $19.1 billion in 2012 – up from 4.6 billion in 2008.Source: Experian 2008, eMarketer 2008

U.S. Mobile Web Users Growing RapidlySource: m:metrics/comScore, 2009

Mobile Audience Size (# of users)In the U.S., the size of the Mobile Audience is 250 MillionSource: m:metrics/comScore, 2009

U.S. Mobile Internet UsageMobile Internet users are heavy users60% access the Internet from their phone at least once/day48% access the Internet more than once per daySource: TSM|TargetProfile 2007

Device TrafficMobile phones dominate mobile Web traffic with 66%market share (US).BlackBerry and Smartphones (Windows Mobile, Palm, etc.) are heavy users of mobile Internet – combined 26% of trafficiPhone/iPod Touch has shown tremendous growth, capturing 8% of the mobile Internet market in the USSource: m:metrics/comScore, 2009

Widgets Improve Mobile Internet ExperienceSource: ‘Widgets Improve the Convenience of the Mobile Internet,’ Forrester, 2008

Take out your cell phone and search for times that ‘Star Trek’ is playing in your neighborhood. Raise your hand when you have finished the task.

Discussion Points How long did it take?How did you do it? (web, app, SMS?) What problems, if any, did you have?

Overview of Research MethodsQuick and Dirty feedbackLab StudiesField StudiesOnline SurveysBasic SurveySurvey with Behavioral Tracking (Keynote on iPhone only)Quantitative Behavioral Analysis

Quick and Dirty – What are obvious problems?AdvantagesInformal setting with Lo-Fi prototypesFast iteration cyclesBrain-damage checkFinds critical usability issues in shortest timeChallengesExamplesInsider/stakeholder viewAnecdotal, unstructured resultsSmall sample sizeDifficult to convey real usage scenariosCross-functional team feedbackInternal alpha/beta

Lab Studies – Can they use it?AdvantagesAllows for probing by moderatorCan record interactionsBest for prototype testingAllows for real-time viewing of interaction – team engagementChallengesExamplesArtificial environmentModerator bias; participant ‘pleasing’Small sample sizeDifficult to test all hardware / network scenariosUsability studyParticipatory design sessions

Field Studies – What is the context of usage?AdvantagesAssess usage under real conditionsCandid user feedbackDelivers unknown use-casesChallengesExamplesSmall sample sizeDifficult to observe and probeParticipant engagement difficultRequires robust productDiary studyCommunity study

Online Surveys – What do they think of it?AdvantagesCan deliver statistically valid resultsProvides qualitative and/or quantitative dataGeographical diversity (remote)Broad representation of devicesChallengesExamplesDifficult to observe and probeOut-of-context interaction, esp. with desktop surveyTechnical constraintsDesktop surveysOn-device surveySurvey with behavioral tracking

Behavioral Analysis – How do they use it?AdvantagesReflects what people do, not sayStatistically valid resultsShows all behaviors, rather than those confined to a single taskChallengesExamplesProbing / deep-dive difficultContext and intent unknownAttitudes and perceptions unknownLog data mining

Types of PrototypesPaper prototypesMocksPaper and pencilInteractive prototypesAppsFlashNative, e.g. iPhoneBrowser-basedSimple HTML prototypeHigh-fidelity prototype (with session variables)

Yahoo! GoMobile ApplicationSearchNewsSportsWeatherOther Topics

Y! oneSearch integrated into Y! Go

Business IssueDrive mobile search uptake – Improved user experience can help accelerate growth of the serviceImprove relevancy – Understanding search intent can contribute to delivering more relevant resultsProduct differentiation – New approach to search results page can deliver higher value to customers

Research ObjectivesUnderstand mobile search behavior in users’ daily livesIdentify content users seekwhen conducting mobile searchesAssess the context surrounding mobile searchesEvaluate effectiveness of new productIdentify opportunities to improve the user experience

Form into teams of 4.Develop a research plan for Yahoo! Go.You have 10 minutes.

Research ObjectivesTeam ActivityUnderstand mobile search behavior in users’ daily livesIdentify content users seekwhen conducting mobile searchesAssess the context surrounding mobile searchesEvaluate effectiveness of new productIdentify opportunities to improve the user experienceDevelop a research plan for Yahoo! GoInclude: Recommended method(s)

Profile of participantsDiscussion Points What methods do you recommend & why? What are your anticipated concerns? Any potential limitations?

Our Solution: One-Month Field Study (1)Daily Mobile SurveyDigital PhotosVoicemails & Pocket CardDaily SMS mini-survey Linked to phone number for voicemail Users sent photos of themselves or surroundings in the context of using oneSearch9 users Daily voicemailsLaminated pocket card with key questions

‘Pocket Card’ Questionnaire

Sample Voicemail Diary Messages -- WinchelleFriday 7:39pmI used oneSearch today, was looking for a hospital – directions and address to it. I was getting off work and sitting in my car, trying to find directions. It was about 4:30. I liked that it came up with the website and with information about that. And the directions on how to go.I didn’t like that it could not find my work address – which is saved on my Yahoo! directions (recently gone) saved places. I think using the actual internet through the PC will help me better because it knows where the address is. I’m not sure why the address is not coming up in my oneSearch and that sucked.Saturday 8:06pmI didn’t use oneSearch today. I looked for info, looked for a store and I was at home. I wanted to know if the store was open STORE HOURS (9:45am). I used the internet with my phone and used Google to search. It was not very helpful because I wanted to see what time they opened and I couldn’t find it. I’ve used oneSearch in the past and I guess the results were the same but I guess it’d be nice to go to the site for that particular store.Monday 10:48pmI used oneSearch to find Expedia, was at work on my lunch break. It was about 12:20. I liked that I can find it quickly. I guess the website didn’t support my mobile phone though, so I didn’t like that. Would rather book a flight/hotel at home because its supported.

Our Solution: One-Month Field Study (2)Pre-InterviewMid-Check InterviewWrap-up InterviewAn initial 45-minute interview Verified technical capabilities of phoneExplained research program20-minute check-in phone interviewInitial feedbackQ&AFinal in-person interview of 75 minutes Clarified voicemail reportsOverall impressions of the product from the month-long usage

Screening Criteria: 9 Experienced Mobile Users Have an unlimited data plan for WAP services through their mobile carrierUse their mobile for text messages (SMS) daily and use mobile more than three times per week for activities that require the transmission of online dataHave Cingular, Sprint, T-Mobile, or Verizon as their mobile carrierHave a primary mobile device capable of downloading and using Y! GoLive or work within San FranciscoMix of gender, age (23-46+),and typical method of commuting to work (4 users primarily drive, 5 primarily take mass transit)

Sliding Scale for Participant Incentives$100 for the initial in-person interview$50 for the mid-way phone interview$150 for the final in-person interview$3 each day a report is given

Natural environment key to context All research activities on the mobile device Mix of methods Engage users in program Breadth of insights – context, usability, impressionsWhy did we choose this method?

What did we learn? (1)Perceived value increased with usageUsage typically driven by lack of computer availabilityHowever, certain scenarios drove phone choice over computer: Social setting Privacy Convenience

What did we learn? (2)Mental model based on computer usageSpeed and relevance of search results highest area of frustrationLack of understanding that results were customized by widget

Daily SMS Survey Results*Website not adapted for mobile device

Summary of Use Case FrequencyNote: The 166 searches were conducted by 9 users

ShoppingBusiness’ number, location, directions, and hours of operation “While driving home from work, I did a search for Fry’s electronics on the local search and it came up with all the different Fry’s in the area, their numbers, as well as directions and all that. The results were all helpful and fast in helping me find the one closest.”“Since we were already out shopping, I wanted to find where the closest Citibank was to where we were.”Pricing/Product comparisons“I was at Costco and comparing TV prices. I wanted to know how much these same TV’s would be at the Circuit City down the street.”

DiningA restaurant’s number, location, directions, and hours of operation “I just woke up and didn’t want to boot my computer up but I wanted to find out what time this breakfast place opened so I could know how soon I could get there.”“I was in my car and wanted their number so I could call and place an order to pick it up on my way home from work.”Ideas of places to eat“I was with my wife in the Haight and we wanted to find nearby restaurants so we could decide what we wanted to eat that was within walking distance.”“I was in the South Bay with some friends and we wanted to find a good sushi place nearby.”Reservations & Reviews“I was hoping to make reservations straight from the phone, or at least call them to place them.”“I was out with some friends and curious about whether the place nearby was any good.”

TravelingTransit schedules and numbers “I wanted to find out when the next MUNI but the MUNI’s site wouldn’t work on my phone.”“I wanted the number to call a taxi. I was outside and tired of waiting for the bus.”Tourist activities“I wanted to see what kind of ferry tours we could do before looking for another museum that would be fun for my daughter.”“I was looking for fun things to do while at Hermosa beach.”Maps and directions“I was trying to get a map of the Mt. Shasta area and I wanted to plan which route to take. It would be nice if the driving directions had live updates on traffic and road conditions.”Flight status / check-in“I was in the taxi to the airport and wanted to check my flight’s status.”

Business ImplicationsDrive mobile search uptake – Distribution through partnerships, marketing campaigns, opening to third party services and developersImprove relevancy – Focused on improving results for specific use cases (e.g. local businesses, flights, etc.)Product differentiation – Federated search results, integrated user experience across multiple services (e.g. search results + maps)

Mobile Usability Research Challenges & Tips

Variety of Mobile DevicesChallengesTipsDifficult to know and understand user experience across all devicesDevices vary by model, browser type, carrier, and input typeModerator may be unable to help users through usability tasks – resetting prototypes, navigating – if interface is not well-knownDesign team should have a variety of devices simulators / devices on hand for testingPrototype designs should be tested across as many devices as possibleLimit recruiting to only devices that have been testedHave a back-up plan with either a basic device or paper prototypes

ChallengesTipsMobile Devices are Small and . . . Well, Very Mobile Create a hot zone on the table, encouraging users to keep the device within a narrow frame Consider light source and potential reflection on the phone; continually adjust mid testing; turn-off lightsUse remote-controlled video camera (with technician in back room) or have a second technician available in interview room Project the mobile screen on a larger monitor in interview room and back roomCreatively screen out participants whose finger size may distract from findings (if non-touch screen)Users move the device around while interacting and explaining, often moving video display out of focusReflectors on screens and smudges can make video all reflectionSmall device screens make it difficult to see what users are doingLarge fingers and long fingernails can cause unintended device responses

ChallengesTipsVaried Locations and Use Cases Mobile device usage occurs in a variety of places59% of Americans check email while in the bathroom*Difficult to recreate the true experience in a lab setting Create opportunities for feedback in context of usageConduct field studies on beta productsMOBILE DEVICE ACCESS Source: Nielsen Mobile 2008

ChallengesTipsVaried Carrier Network CoverageNetwork coverage is inconsistent; difficult to predict accessibility during lab testingDifficult to validate that out-of-town lab truly has adequate coverageVisit facility with colleagues who have various carriers to confirm network availability in labFrom out-of-town labs, request:Reference clients from last mobile studiesList of carriers confirmed to have consistent coverageAt a minimum, check carrier coverage in various cities online

Screen during recruiting for:Unlimited data plansWillingness to send / receive text messages during studyRescreen again upon arrival for studyChallengesTipsCalling / Data Plans Vary WidelyParticipants may be charged for mobile web access or sample texts sent during testingParticipants may change data plan between recruiting interview and date of study

ChallengesTipsVaried User ExperienceMost mobile device owners only utilize a small percentage of total device capabilities43% of mobile subscribers do not use text messaging on a regular basis*Users may have no experience typing in a web address to access a site or with sending SMS messages (which may be critical to your study)If using a prototype phone for downloadable app, users may not be comfortable with itPre-test participants to ensure appropriate device experienceMobile address typing Send SMS to recruiterSpend 2-5 minutes explaining how prototype phone works* Source: Nielsen 2008 ‘The Short Code Marketing Opportunity’

Other Helpful TipsTell participants to bring rechargerSimplify language (e.g., SMS vs. text message)Have a prototype strategyBack-end SMS text simulationEasy URL for accessBack-up paper prototypesIndex page for easy access to alternative flowsHave plan for interruption from phone call or text (resetting session variables)Plan for getting screenshots for your reportiPhone: Press hold button and home button at same time (saves into photo gallery)Use documented comps

Making The Leap From Web To Mobile

More Related Content

What's hot (20)

Viewers also liked (18)

Similar to Making The Leap From Web To Mobile (20)

Recently uploaded (20)

Making The Leap From Web To Mobile