SlideShare a Scribd company logo

The Lean Data Field Guide by Acumen

M
Mobile Surveys Inc.

This document provides guidance on using Lean Data, which is an approach to impact measurement developed by Acumen that emphasizes efficient data collection to create value for companies and customers. It discusses choosing questions and methodology, available technologies for data collection like SMS, phone calls and in-person interviews, example question sets, and taking action based on the data collected. The overall goal is to help organizations use customer feedback to build more impactful businesses through improvements and better targeting of customers.

Data & Analytics
1 of 24
Downloaded 10 times
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
1The Lean Data Field Guide Introduction Tips For Collecting Customer Data to Build More Impactful Businesses November 2015 THE LEAN DATA FIELD GUIDE
2The Lean Data Field Guide Introduction Lean Data is an approach to impact measurement developed by Acumen. The approach involves two main elements: + A shift in mindset away from reporting and compliance and toward creating value for a company and its customers + The use of methods and technologies for data collection that emphasize efficiency and rapid response while still achieving a sufficient degree of rigor. WHAT IS LEAN DATA? This document is designed to serve as a field guide to help you conduct Lean Data projects. It is meant to be practical and action- oriented. For a more in depth overview of the theory and practice behind Lean Data, please see: http://www.ssir.org/articles/entry/the_ power_of_lean_data http://acumen.org/wp-content/ uploads/2015/11/Innovations-in-Impact- Measurement-Report.pdf
3The Lean Data Field Guide Introduction HOW LEAN DATA WORKS Choose Your Questions Method Choose your question sets and devise an execution plan based on our “how to” guides. Get Started Establish what you hope to discover through your Lean Data project. 1 2 3 4 Choose Your Technology Determine the technology/ method to communicate with customers quickly and efficiently. Take Action Based On Your Data Use the data to decide on concrete actions and next steps.
4The Lean Data Field Guide Get Started Step 2: Identify the data you need to understand if you’re delivering on this promise Step 1: Define your customer promise and the impact you are trying to achieve Example Do my customers see my product as affordable and high-quality? Am I reaching the poor? Are customers actually reducing their kerosene usage? Provide high-quality, affordable solar home solutions that eradicate the need for kerosene for low- income households Customer promise: Customer question areas: What do you want to find out? 1. GET STARTED How are my customers experiencing my product/service? How do I understand and refine my value proposition? Who are my customers? How can I hear from them to improve my business? Is my product/service having social impact?
5The Lean Data Field Guide Get Started As in the example above, force yourself to summarize your customer promise in a single sentence and think about what will make a customer choose your product or service over the competition. Breaking your customer promise down into different parts helps to identify the key areas and assumptions you want to test. Your customer promise can also be thought of as your Social value proposition: What benefits do you intend to deliver for whom? What does success look like?
6The Lean Data Field Guide Choose Your Technology 2. CHOOSE YOUR TECHNOLOGY Start Here Do you need to ask over 10 questions? Sensitive questions? Budget Go with SMS Go with Phone Go with IVR Do you need qualitative, detailed responses? Less than $5k More than $5k Are literacy rates very low? No No Yes Yes Yes No No Yes Note If you can’t get a representative set of phone numbers, go with in-person interviews
7The Lean Data Field Guide Choose Your Technology Pros ConsTechnology SMS + Ability to incentivize customer with airtime bonus + Customer can complete at his/her convenience + Good quality open-ended questions + Honest responses to sensitive questions + Ability to compare responses to general population panel (requires existing panel – e.g. Kenya) − Need high mobile penetration decent literacy rates IVR + Works in low-literacy areas (e.g. parts of rural India) + Honest responses to sensitive questions − Multiple choice only, not able to analyze qualitative feedback easily − Less customer-control over timing of survey PHONE CALLS + Works great in low literacy contexts + Good for more complicated questions that require explanation / probing (“tell me more about that”) + Higher confidence in the quality of data − Still need phone numbers − 3-5x more expensive than IVR SMS IN-PERSON + Overcomes low mobile phone penetration + Allows observation (e.g. see household environment to witness how a product is being used) + High quality, trusted data − Time consuming, expensive (cost depends on quality of enumerators). − Experience can be intrusive to customer SENSORS + Avoids bias inherent in self-reporting + Good for measuring frequency of predictable events e.g. usage patterns. Example sensors: heat sensor (cookstove), light sensor (solar), sanitation sensor (toilet usage) − Expensive; ethical concerns 2. TECHNOLOGY PROS CONS
8The Lean Data Field Guide This is a list of selected survey partners. This is not an exhaustive list as the number of providers in this field is constantly growing. Choose Your Technology Name WebsiteTechnology Country http://msurvey.co.ke/ https://www.echomobile.org/public/main https://www.votomobile.org/ https://www.awaaz.de/ http://research.geopoll.com/ http://goodworldsolutions.org/#labor-link https://www.fourthlion.in/ http://home.magpi.com/mobile-data-collection https://www.ipsos-mori.com/ https://www.findyr.com/ https://telerivet.com/ http://www.ureport.ug/ SMS SMS, Tablet SMS, IVR SMS, IVR SMS IVR IVR, SMS SMS, IVR Phone, In-person In-person + photos SMS SMS Global (mainly E.Africa) Global Global Global (mainly India) Global India, LatAm India Global Global Global Global Global mSurvey Echo mobile Voto Awaaz.De GeoPoll Labourlink Fourth Lion Magpi Ipsos Mori Findyr Telerivet U-report 2. CHOOSE YOUR SURVEY PROVIDER
9The Lean Data Field Guide Question Set + In the next section, we’ve grouped tried and tested questions into themes to make it easy for you to mix and match questions based on your needs and specific customer promise. + There are 2-5 questions in each question set, and each question has been tested with low-income respondents (mainly in East Africa) and over SMS. What Are You Trying To Find Out Value proposition Why are customers buying your product/service? What do they see as valuable? 1 Customer satisfaction How happy are your customers with your product/ service? What would they improve? 2 Meaningfulness Is your product/service actually having a meaningful impact on your customers’ lives? 3 Outcome measurement Are social outcomes improving as a result of your product/service? To what degree? What outcomes do customers value most? 4 Poverty profile What is the income profile of your customers? What % of them live below the poverty line? 5 Customer Archetypes What attitudinal / behavioral characteristics are common in your customers? Are they more risky? More aspirational? 6 Choose Your Questions Method 3. CHOOSE YOUR QUESTION SET
10The Lean Data Field Guide Enjoyable ethical Put the respondent first: include an introduction with a statement of consent, summary of the survey purpose, and a close out text with personalized content. Who is responding? For phone surveys, keep in mind that shared cell phones are common. If you need respondent-specific information, check to see if you are speaking to the right person. Decision-focused Make sure your questions are relevant and actionable: what will you do with the information you expect to collect? Scales If you are using a scale in your question, make sure it’s appropriate for and understood by your audience (e.g. be creative: use smiley faces, thumbs up/thumbs down, 1-5 is better than 0-9). Test test test Test the questions on yourself and then test questions with a small sample of locals: do they understand the questions right away? Don’t assume Don’t assume knowledge on spending habits as respondent might not be the primary buyer (see Question Set 4 for an example). Short Try to keep surveys between 5-10 questions. Statistical significance The size of sample required to achieve statistical significance can vary depending on the complexity of your study and population size - seek expert advice if needed. A general rule for most simple surveys is a sample size of 200. Mix it up Achieve a balance between open-ended and multiple choice questions (and mix up the order). Non-customers! Don’t forget about speaking to non-customers: have they heard of the product / service or thought about purchasing it? If not, what’s stopping them? 3. SURVEY RULES OF THUMB Choose Your Questions Method
11The Lean Data Field Guide Result: More impactful businesses + higher sales Data needed to make informed decisions Desiredaction Question Set: Amend design of product / service to better meet customer needs Align marketing messages to value prop that resonates most with customers Make quick-win changes to improve customer satisfaction Provide donors/investors with real impact data; access additional grant funding Identify ways to reach more low-income customers Target customers more effectively along attitudinal dimensions Value Proposition Customer Satisfaction Meaning- fulness Outcome Measure- ment Poverty Profile Customer Archetypes BASED ON YOUR DATA + The question sets in this guide are tightly connected to a desired action and enable companies to make better, more informed decisions. This ultimately drives higher sales and more impactful businesses. 4. GET READY TO TAKE ACTION BASED ON YOUR DATA Take Action Based On Your Data
QUESTION SETS
13The Lean Data Field Guide QUESTION SET 1: VALUE PROPOSITION Use this question set to understand + Reason for purchasing + Perceived value for money + Quality How would you rate the value for money of (product/service)? A. It is expensive B. It is fair C. It is cheap Please explain your answer Q Why did you purchase (product/service)? Q How is the quality of (product/service) compared to the (product/service) you were using before? A. It is better B. It is the same C. It is worse Please explain your answer Q What feature do you like best? Please explain your answer Q Question Sets
14The Lean Data Field Guide Q Use this question set to understand: + How likely your customers are to recommend your product/service + Suggestions for improvement + Customer experience and satisfaction Note On Our Amended NPS: After testing the traditional NPS developed by Bain, we modified the phrasing to be more applicable across different regions. We found that our customers have trouble understanding the “Would you recommend” language and the 0-10 answer scale. Our revised version aims to get similar data as the NPS, with language that we’ve found is clear to the customers we are serving. QUESTION SET 2:CUSTOMER SATISFACTION Have you ever recommended (product/service) to your friends? A. Yes, I have told many friends What have you told them? B. Yes, I have told some friends What have you told them? C. No, I have not Why not? How can we improve (product/service)? Q Is there anything else you’d like to share with us? TIP: This question works well at the end of a survey Q Amended Net Promoter Score (NPS) Question Sets
15The Lean Data Field Guide + This toolkit is a combination of our best questions on usage, value proposition, customer satisfaction, and meaningfulness. + When asked in this sequence, this question set is our shortest and simplest way to understand depth of impact. QUESTION SET 3: MEANINGFULNESS TOOLKIT Q When do you use your [product/service]? A. Everyday B. 3-4 times a week C. 1-2 times a week D. Less than once a week E. It’s not working Q There have been changes in my home because of [product/service] A. Strongly agree B. Agree C. Neutral D. Disagree E. Strongly disagree Q Have you ever recommended [product/service] to a friend? A. Yes, I have told many friends. Please explain what you said B. Yes, I have told some friends. Please explain what you said C. No, I have not. Why not? Q Were these changes positive or negative? A. Positive i. Please explain the positive changes in your life because of (product/ service) ii. Out of those changes, which one is most important to you? B. Negative. Please explain your answer Question Sets
16The Lean Data Field Guide + To understand if social outcomes are improving as a result of a product/service, recall-based questions are useful. + However, when relying on customer memory, keep in mind that the accuracy of responses diminish after roughly one month of purchasing a new product/service. + To avoid recall error, try to survey new customers. Alternatively, you can do a baseline / endline, and survey the same customers, asking the same questions, at different points in time Example How to ask about the effect of solar lighting on kerosene expenditure and quality of available light Q Do you know how much your family spends on kerosene? A. Yes B. No Q (If yes) Now that you have a solar light, what did you spend on kerosene last week? Q Before you had a solar light, what did you spend on kerosene each week? Q How is the quality of your light compared to what you had before? A. Much brighter B. Somewhat brighter C. The same D. It is less bright QUESTION SET 4: SOCIAL OUTCOMES Question Sets
17The Lean Data Field Guide We use the Progress out of Poverty Index (PPI) developed by Grameen Foundation to understand: + Percentage of customers living below poverty line + Customer segmentation, e.g. different household profiles of customers QUESTION SET 5: POVERTY PROFILE How to conduct a PPI Go to: http://www.progressoutofpoverty.org/ Create a free account Click on “Find your PPI” and select your country of interest Download the country specific “PPI Scorecard and Lookup Table” Follow the instructions to conduct the PPI survey questionnaire. Aim for a minimum sample of 200 respondents Once you have the data, budget 30-60 minutes to complete the analysis using the scorecard and lookup tables to work out what percentage of customers live below the poverty line 1 3 5 2 4 6 Question Sets
18The Lean Data Field Guide Q QUESTION SET 6: CUSTOMER ARCHETYPE + When marketing a product/service, understanding the attitudes, behaviors, and preferences of your customers (and non-customers) is key. + This question set highlights our recent thinking about how to build and understand your customers along attitudinal lines vs. simple demographics Tip Don’t forget about surveying non-customers! You can glean important insights on how to better market your product to meet their needs. What did you buy in the last year that has had a big impact on your life? What did you buy in the last year that has had a big impact on your life? Question Sets Q What is something you hope to own in the next 5 years? Q How likely are you to purchase a new product before any of your friends own it? A. Very likely B. Likely C. Neutral D. Unlikely E. Very unlikely
19The Lean Data Field Guide Introduction “HOW TO” GUIDES
20The Lean Data Field Guide HOW TO: CHOOSE A SAMPLE SIZE “How To” Guides Golden rule Sample of size of 250 is sufficient in most cases! Useful definitions + Confidence level: is the probability that your sample accurately reflects the attitude of your population +Margin of error: the range (measured as a %) that your population’s responses may deviate from your sample Visit this link for additional details: https://www.surveymonkey.com/mp/sample-size-calculator/ 10K 50K 1M+ 5% 5% 5% Number of customers Margin of error Sample size 162 164 370 382 204 207 Confidence Level 262 268 164 385208 269 80% 95%85% 90%
21The Lean Data Field Guide Clever cost-effective ways to start gathering mobile contact information: Customer Registration Make Phone Number a “must-have” field in the customer registration process. Can use mobile-apps or tablets to make data capture more efficient and error-free (vs. paper entry) Radio + SMS/IVR For potential customers, hold a radio campaign to encourage SMS/ IVR inquiries: e.g. “call/SMS this number free to find out more and be entered into a lottery” Join Codes Place a number to register your product on your packaging This word is called a JoinCode and can be the name of your product or company or any other word This number is set by your JoinCode provider Choose a credible incentive for your customer to text the joincode “How To” Guides HOW TO: GET CUSTOMER PHONE NUMBERS 1 2 3 80249 for FREE Text [HELLO] To register your warranty / sign up for offers / enter a lottery etc
22The Lean Data Field Guide When to use focus groups + To dig deeper into perceptions and attitudes and get feedback on product ideas and marketing campaigns + Particularly useful for understanding non-customers + Advantages: allows you to study people in a more natural conversation pattern than typically happens in one-to-one interviews. Efficient: collect views from multiple people at once “How To” Guides HOW TO: RUN A FOCUS GROUP Who: 6-12 people, don’t know each other but feel comfortable together (think about gender, age, income level when selecting participants). 1 trained moderator, 1 assistant moderator to take notes Set-up: chairs in a circle. Example locations: quiet hotel, café, restaurant. Outdoors in shade can also work well Feel: Warm, relaxed environment. Offer drinks and small monetary incentive to say thank you for participation. It’s important to make everybody feels comfortable to share views. Length: 45-90 mins. 8-9 question areas What makes a good focus group 1. Welcome: Thank participants. Introduce moderators and purpose of focus groups “We have been asked by _____ to find out about ______. We need your input and want you to share your honest and open thoughts with us” 2. Ground Rules: “(1) We want you do the talking. We would like everyone to participate. I may call on you if I haven’t heard from you in a while. (2) No right or wrong answers: every person’s experiences and opinions are important. Speak up whether you agree or disagree. We want to hear a wide range of opinions. (3) What is said in this room stays here. Your answers are completely anonymous. (4) Any questions? Is the process clear?” 3. Focus Group Questions: See do’s and don’ts on the next page 4. Close out: Ask “Is there anything we missed?” and thank for participation How to run a focus group
23The Lean Data Field Guide “How To” Guides It’s easier than you think! All you need to do is: + Find a moderator who understands the local context (Impact team can help) + Secure a suitable venue for 2 hours + Recruit participants: go to local market ask “are you free [insert time] for 2 hours for a focus group? You will be compensated for your time” + Prepare a thoughtful moderator guide: what questions do you want to ask? Setting up a focus group http://www.focusgrouptips.com/focus-group-questions.html For helpful tips on question ordering and phrasing http://www.eiu.edu/~ihec/Krueger-FocusGroupInterviews.pdf For a full guide on how to design and conduct a focus group http://www.theexperiencebusiness.co.uk/downloads/content-docs/top_ten_tips.pdf Top 10 tips for great focus groups Additional guides Ask open-ended Qs, e.g.: “What do you like best about… what are the biggest problems with… what comes to mind when xyz” etc Follow up if unclear—”will you tell me more about that?” Ask for specifics if you get a very general response to a question Ask others in the group their views are when you only have heard one person’s opinion Seek out input from all participants Ask many “yes” or “no” questions. Ask more than one question at a time. Ask leading questions (“Isn’t this true that Y is important?”) “correct people” or tell them they are wrong about something Be afraid of asking for clarification Pressure someone to speak Question Do’s Don’ts Do: Don’t:
24The Lean Data Field Guide Introduction Website www.acumen.org Twitter @Acumen

More Related Content

PDF
Customer FAQs
Mobile Surveys Inc.
 
PDF
Stanford social innovation review The Power of Lean Data
Mobile Surveys Inc.
 
PDF
Top tips
Evelyn Mwangi
 
PDF
Improving Internal Communications
Granicus
 
PPTX
TVC Eye Tracking - Example Report
EyeSee Research
 
PPT
Asmi San Diego
donhendricks
 
PDF
The Definitive Guide to Qualitative Analytics
Bar Clara Mendez
 
PDF
Using Employee Communication as a Business Process to Improve Financial Results
ISM University of Management and Economics
 
Customer FAQs
Mobile Surveys Inc.
 
Stanford social innovation review The Power of Lean Data
Mobile Surveys Inc.
 
Top tips
Evelyn Mwangi
 
Improving Internal Communications
Granicus
 
TVC Eye Tracking - Example Report
EyeSee Research
 
Asmi San Diego
donhendricks
 
The Definitive Guide to Qualitative Analytics
Bar Clara Mendez
 
Using Employee Communication as a Business Process to Improve Financial Results
ISM University of Management and Economics
 

What's hot (20)

PDF
The Hole, Not The Drill. Why reporting should focus on the solution (and not ...
Ray Poynter
 
PPSX
MMRSS Perception Analyzer
Majestic MRSS
 
PDF
MMRSS Perception analyzer
MajesticMRSS
 
DOC
Online surveys
ResearchShare
 
PDF
Eye see overview general capabilities
EyeSee Research
 
PPTX
Adobe Education Leadership Forum 2011 in Hong Kong
Tim Elleston
 
PDF
Effective E Detailing
ayyosama
 
DOCX
A3321146
sal_s
 
PDF
Zendesk benchmark research
Jason Maynard
 
PDF
From Hype to Reality: AI in Market Research
Tom De Ruyck
 
PDF
Week 5 - Media Briefing
Ben Shepherd
 
PDF
Liberating Research: A Manifesto for Change
Kantar
 
PDF
Engaging users in digital strategy development
Endeavor Management
 
PPT
"Building a Well-oiled Sales Machine" Kathy Ruggiero: Senior Director Corpora...
LeadJen
 
PDF
Peanut Labs - Responses To Esomar Questions 2015
Peanut Labs
 
PDF
1120 track2 bennett
Rising Media, Inc.
 
PPTX
#Seven Basics Tools of Quality# By SN Panigrahi
SN Panigrahi, PMP
 
PPTX
All global brand essence and elements
eSeidr
 
PPTX
[Webinar] Elevating Internal Communications: From Tactical Function to Strate...
Poppulo
 
PDF
How to Use the Barcelona Principles to Save Time & Money, Make Better Decisio...
Paine Publishing
 
The Hole, Not The Drill. Why reporting should focus on the solution (and not ...
Ray Poynter
 
MMRSS Perception Analyzer
Majestic MRSS
 
MMRSS Perception analyzer
MajesticMRSS
 
Online surveys
ResearchShare
 
Eye see overview general capabilities
EyeSee Research
 
Adobe Education Leadership Forum 2011 in Hong Kong
Tim Elleston
 
Effective E Detailing
ayyosama
 
A3321146
sal_s
 
Zendesk benchmark research
Jason Maynard
 
From Hype to Reality: AI in Market Research
Tom De Ruyck
 
Week 5 - Media Briefing
Ben Shepherd
 
Liberating Research: A Manifesto for Change
Kantar
 
Engaging users in digital strategy development
Endeavor Management
 
"Building a Well-oiled Sales Machine" Kathy Ruggiero: Senior Director Corpora...
LeadJen
 
Peanut Labs - Responses To Esomar Questions 2015
Peanut Labs
 
1120 track2 bennett
Rising Media, Inc.
 
#Seven Basics Tools of Quality# By SN Panigrahi
SN Panigrahi, PMP
 
All global brand essence and elements
eSeidr
 
[Webinar] Elevating Internal Communications: From Tactical Function to Strate...
Poppulo
 
How to Use the Barcelona Principles to Save Time & Money, Make Better Decisio...
Paine Publishing
 
Ad

Similar to The Lean Data Field Guide by Acumen (20)

PPT
Voice of the Customer (VOC) Research
QuestionPro
 
PDF
ikano_whitepaper_ceo
Barry Smith
 
PDF
Business success through data driven insights
Jeffrey Evans
 
PDF
Convincing your CEO to invest in a loyalty programme
colinjones001
 
PDF
CS 2014-15-2
Jim Head
 
PDF
5 things you should be doing with your content strategy but aren't
Gina Om
 
PDF
BRIDGEi2i Whitepaper - The Science of Customer Experience Management
BRIDGEi2i Analytics Solutions
 
PDF
2016 Lead Nurturing Report In Partnership With Salesforce Pardot
Veena Glover
 
PDF
Maximizing the Value of Customer Feedback Surveys Design and Benefits.pdf
CViewSurvey
 
PDF
Digitizing the Customer Experience within a Utility
Robert Simon
 
DOCX
Ganesh introduction
Sudarshan Bhosale
 
PPTX
Social Fresh Tampa 2011: The Social Engagement Journey w/ Kristy Bolsinger
Social Fresh Conference
 
PDF
2015_Using-NPS-Successfully
Brett Sharp, MMR
 
PDF
Voice pick Research - Product Information
VoicePick1
 
PDF
The Transformative Role of Data Analysis in Enhancing Customer Experience.pdf
Soumodeep Nanee Kundu
 
PPTX
What is InfoQuest?
Tom McCabe
 
PDF
Liferay overview of predicitve analytics
Joe Brandenburg
 
PDF
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
Vanessa Stirling
 
PPTX
1 market research
Apsara Gunarathne
 
PDF
Market research brochure : Transvisionarysolutions
Transvisionary Solutions
 
Voice of the Customer (VOC) Research
QuestionPro
 
ikano_whitepaper_ceo
Barry Smith
 
Business success through data driven insights
Jeffrey Evans
 
Convincing your CEO to invest in a loyalty programme
colinjones001
 
CS 2014-15-2
Jim Head
 
5 things you should be doing with your content strategy but aren't
Gina Om
 
BRIDGEi2i Whitepaper - The Science of Customer Experience Management
BRIDGEi2i Analytics Solutions
 
2016 Lead Nurturing Report In Partnership With Salesforce Pardot
Veena Glover
 
Maximizing the Value of Customer Feedback Surveys Design and Benefits.pdf
CViewSurvey
 
Digitizing the Customer Experience within a Utility
Robert Simon
 
Ganesh introduction
Sudarshan Bhosale
 
Social Fresh Tampa 2011: The Social Engagement Journey w/ Kristy Bolsinger
Social Fresh Conference
 
2015_Using-NPS-Successfully
Brett Sharp, MMR
 
Voice pick Research - Product Information
VoicePick1
 
The Transformative Role of Data Analysis in Enhancing Customer Experience.pdf
Soumodeep Nanee Kundu
 
What is InfoQuest?
Tom McCabe
 
Liferay overview of predicitve analytics
Joe Brandenburg
 
InfinityQS_7 Habits of Quality Obsessed Manufacturers_Print final
Vanessa Stirling
 
1 market research
Apsara Gunarathne
 
Market research brochure : Transvisionarysolutions
Transvisionary Solutions
 
Ad

More from Mobile Surveys Inc. (9)

PDF
mSurvey Owen Masterclass
Mobile Surveys Inc.
 
PDF
Customer FAQs
Mobile Surveys Inc.
 
PDF
Top tips
Mobile Surveys Inc.
 
PPTX
Advertisement conversion
Mobile Surveys Inc.
 
PPTX
Money matters
Mobile Surveys Inc.
 
PPTX
Polygamy survey in Kenya
Mobile Surveys Inc.
 
PDF
Hpv awareness kenya
Mobile Surveys Inc.
 
PPTX
Msuu
Mobile Surveys Inc.
 
PPTX
Taxi operators in Nairobi Survey findings
Mobile Surveys Inc.
 
mSurvey Owen Masterclass
Mobile Surveys Inc.
 
Customer FAQs
Mobile Surveys Inc.
 
Top tips
Mobile Surveys Inc.
 
Advertisement conversion
Mobile Surveys Inc.
 
Money matters
Mobile Surveys Inc.
 
Polygamy survey in Kenya
Mobile Surveys Inc.
 
Hpv awareness kenya
Mobile Surveys Inc.
 
Msuu
Mobile Surveys Inc.
 
Taxi operators in Nairobi Survey findings
Mobile Surveys Inc.
 

Recently uploaded (20)

PDF
Galatica Smart Energy Infrastructure Startup Pitch Deck
Shahzaib Ajmal
 
PPT
Reliability_Chapter_ presentation 1221.5784
abobaker13
 
PDF
22.Patil - Early prediction of Alzheimer’s disease using convolutional neural...
ngaviet5
 
PDF
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
JamesWilliams752225
 
PPTX
Business Acumen Training GuidePresentation.pptx
stephvshelton19
 
PDF
Mega Projects Data Mega Projects Data
saidsalah8181
 
PPTX
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
chepkoitcheruiyot
 
PDF
“Getting Started with Data Analytics Using R – Concepts, Tools & Case Studies”
Nusrat Gulbarga
 
PPTX
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
rohullahansari5
 
PPTX
Major-Components-ofNKJNNKNKNKNKronment.pptx
dushyantsharma1221
 
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
hshakobe3
 
PPTX
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
AceAquino4
 
PPT
Chapter 2 METAL FORMINGhhhhhhhjjjjmmmmmmmmm
JanakiRaman206018
 
PPTX
CEE 2 REPORT G7.pptxbdbshjdgsgjgsjfiuhsd
hildogavino28
 
PDF
Clinical guidelines as a resource for EBP(1).pdf
MashalKhan626345
 
PPTX
IBA_Chapter_11_Slides_Final_Accessible.pptx
SrikantKapoor1
 
PPT
Chapter 3 METAL JOINING.pptnnnnnnnnnnnnn
JanakiRaman206018
 
PPT
Quality review (1)_presentation of this 21
abobaker13
 
PDF
"Python Programming for Geospatial Data Science." ...
institute of Geoinformatics and Earth Observation at PMAS ARID Agriculture University of Rawalpindi
 
PPTX
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
chepkoitcheruiyot
 
Galatica Smart Energy Infrastructure Startup Pitch Deck
Shahzaib Ajmal
 
Reliability_Chapter_ presentation 1221.5784
abobaker13
 
22.Patil - Early prediction of Alzheimer’s disease using convolutional neural...
ngaviet5
 
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
JamesWilliams752225
 
Business Acumen Training GuidePresentation.pptx
stephvshelton19
 
Mega Projects Data Mega Projects Data
saidsalah8181
 
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
chepkoitcheruiyot
 
“Getting Started with Data Analytics Using R – Concepts, Tools & Case Studies”
Nusrat Gulbarga
 
advance b rammar.pptxfdgdfgdfsgdfgsdgfdfgdfgsdfgdfgdfg
rohullahansari5
 
Major-Components-ofNKJNNKNKNKNKronment.pptx
dushyantsharma1221
 
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
hshakobe3
 
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
AceAquino4
 
Chapter 2 METAL FORMINGhhhhhhhjjjjmmmmmmmmm
JanakiRaman206018
 
CEE 2 REPORT G7.pptxbdbshjdgsgjgsjfiuhsd
hildogavino28
 
Clinical guidelines as a resource for EBP(1).pdf
MashalKhan626345
 
IBA_Chapter_11_Slides_Final_Accessible.pptx
SrikantKapoor1
 
Chapter 3 METAL JOINING.pptnnnnnnnnnnnnn
JanakiRaman206018
 
Quality review (1)_presentation of this 21
abobaker13
 
"Python Programming for Geospatial Data Science." ...
institute of Geoinformatics and Earth Observation at PMAS ARID Agriculture University of Rawalpindi
 
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
chepkoitcheruiyot
 

The Lean Data Field Guide by Acumen

  • 1. 1The Lean Data Field Guide Introduction Tips For Collecting Customer Data to Build More Impactful Businesses November 2015 THE LEAN DATA FIELD GUIDE
  • 2. 2The Lean Data Field Guide Introduction Lean Data is an approach to impact measurement developed by Acumen. The approach involves two main elements: + A shift in mindset away from reporting and compliance and toward creating value for a company and its customers + The use of methods and technologies for data collection that emphasize efficiency and rapid response while still achieving a sufficient degree of rigor. WHAT IS LEAN DATA? This document is designed to serve as a field guide to help you conduct Lean Data projects. It is meant to be practical and action- oriented. For a more in depth overview of the theory and practice behind Lean Data, please see: http://www.ssir.org/articles/entry/the_ power_of_lean_data http://acumen.org/wp-content/ uploads/2015/11/Innovations-in-Impact- Measurement-Report.pdf
  • 3. 3The Lean Data Field Guide Introduction HOW LEAN DATA WORKS Choose Your Questions Method Choose your question sets and devise an execution plan based on our “how to” guides. Get Started Establish what you hope to discover through your Lean Data project. 1 2 3 4 Choose Your Technology Determine the technology/ method to communicate with customers quickly and efficiently. Take Action Based On Your Data Use the data to decide on concrete actions and next steps.
  • 4. 4The Lean Data Field Guide Get Started Step 2: Identify the data you need to understand if you’re delivering on this promise Step 1: Define your customer promise and the impact you are trying to achieve Example Do my customers see my product as affordable and high-quality? Am I reaching the poor? Are customers actually reducing their kerosene usage? Provide high-quality, affordable solar home solutions that eradicate the need for kerosene for low- income households Customer promise: Customer question areas: What do you want to find out? 1. GET STARTED How are my customers experiencing my product/service? How do I understand and refine my value proposition? Who are my customers? How can I hear from them to improve my business? Is my product/service having social impact?
  • 5. 5The Lean Data Field Guide Get Started As in the example above, force yourself to summarize your customer promise in a single sentence and think about what will make a customer choose your product or service over the competition. Breaking your customer promise down into different parts helps to identify the key areas and assumptions you want to test. Your customer promise can also be thought of as your Social value proposition: What benefits do you intend to deliver for whom? What does success look like?
  • 6. 6The Lean Data Field Guide Choose Your Technology 2. CHOOSE YOUR TECHNOLOGY Start Here Do you need to ask over 10 questions? Sensitive questions? Budget Go with SMS Go with Phone Go with IVR Do you need qualitative, detailed responses? Less than $5k More than $5k Are literacy rates very low? No No Yes Yes Yes No No Yes Note If you can’t get a representative set of phone numbers, go with in-person interviews
  • 7. 7The Lean Data Field Guide Choose Your Technology Pros ConsTechnology SMS + Ability to incentivize customer with airtime bonus + Customer can complete at his/her convenience + Good quality open-ended questions + Honest responses to sensitive questions + Ability to compare responses to general population panel (requires existing panel – e.g. Kenya) − Need high mobile penetration decent literacy rates IVR + Works in low-literacy areas (e.g. parts of rural India) + Honest responses to sensitive questions − Multiple choice only, not able to analyze qualitative feedback easily − Less customer-control over timing of survey PHONE CALLS + Works great in low literacy contexts + Good for more complicated questions that require explanation / probing (“tell me more about that”) + Higher confidence in the quality of data − Still need phone numbers − 3-5x more expensive than IVR SMS IN-PERSON + Overcomes low mobile phone penetration + Allows observation (e.g. see household environment to witness how a product is being used) + High quality, trusted data − Time consuming, expensive (cost depends on quality of enumerators). − Experience can be intrusive to customer SENSORS + Avoids bias inherent in self-reporting + Good for measuring frequency of predictable events e.g. usage patterns. Example sensors: heat sensor (cookstove), light sensor (solar), sanitation sensor (toilet usage) − Expensive; ethical concerns 2. TECHNOLOGY PROS CONS
  • 8. 8The Lean Data Field Guide This is a list of selected survey partners. This is not an exhaustive list as the number of providers in this field is constantly growing. Choose Your Technology Name WebsiteTechnology Country http://msurvey.co.ke/ https://www.echomobile.org/public/main https://www.votomobile.org/ https://www.awaaz.de/ http://research.geopoll.com/ http://goodworldsolutions.org/#labor-link https://www.fourthlion.in/ http://home.magpi.com/mobile-data-collection https://www.ipsos-mori.com/ https://www.findyr.com/ https://telerivet.com/ http://www.ureport.ug/ SMS SMS, Tablet SMS, IVR SMS, IVR SMS IVR IVR, SMS SMS, IVR Phone, In-person In-person + photos SMS SMS Global (mainly E.Africa) Global Global Global (mainly India) Global India, LatAm India Global Global Global Global Global mSurvey Echo mobile Voto Awaaz.De GeoPoll Labourlink Fourth Lion Magpi Ipsos Mori Findyr Telerivet U-report 2. CHOOSE YOUR SURVEY PROVIDER
  • 9. 9The Lean Data Field Guide Question Set + In the next section, we’ve grouped tried and tested questions into themes to make it easy for you to mix and match questions based on your needs and specific customer promise. + There are 2-5 questions in each question set, and each question has been tested with low-income respondents (mainly in East Africa) and over SMS. What Are You Trying To Find Out Value proposition Why are customers buying your product/service? What do they see as valuable? 1 Customer satisfaction How happy are your customers with your product/ service? What would they improve? 2 Meaningfulness Is your product/service actually having a meaningful impact on your customers’ lives? 3 Outcome measurement Are social outcomes improving as a result of your product/service? To what degree? What outcomes do customers value most? 4 Poverty profile What is the income profile of your customers? What % of them live below the poverty line? 5 Customer Archetypes What attitudinal / behavioral characteristics are common in your customers? Are they more risky? More aspirational? 6 Choose Your Questions Method 3. CHOOSE YOUR QUESTION SET
  • 10. 10The Lean Data Field Guide Enjoyable ethical Put the respondent first: include an introduction with a statement of consent, summary of the survey purpose, and a close out text with personalized content. Who is responding? For phone surveys, keep in mind that shared cell phones are common. If you need respondent-specific information, check to see if you are speaking to the right person. Decision-focused Make sure your questions are relevant and actionable: what will you do with the information you expect to collect? Scales If you are using a scale in your question, make sure it’s appropriate for and understood by your audience (e.g. be creative: use smiley faces, thumbs up/thumbs down, 1-5 is better than 0-9). Test test test Test the questions on yourself and then test questions with a small sample of locals: do they understand the questions right away? Don’t assume Don’t assume knowledge on spending habits as respondent might not be the primary buyer (see Question Set 4 for an example). Short Try to keep surveys between 5-10 questions. Statistical significance The size of sample required to achieve statistical significance can vary depending on the complexity of your study and population size - seek expert advice if needed. A general rule for most simple surveys is a sample size of 200. Mix it up Achieve a balance between open-ended and multiple choice questions (and mix up the order). Non-customers! Don’t forget about speaking to non-customers: have they heard of the product / service or thought about purchasing it? If not, what’s stopping them? 3. SURVEY RULES OF THUMB Choose Your Questions Method
  • 11. 11The Lean Data Field Guide Result: More impactful businesses + higher sales Data needed to make informed decisions Desiredaction Question Set: Amend design of product / service to better meet customer needs Align marketing messages to value prop that resonates most with customers Make quick-win changes to improve customer satisfaction Provide donors/investors with real impact data; access additional grant funding Identify ways to reach more low-income customers Target customers more effectively along attitudinal dimensions Value Proposition Customer Satisfaction Meaning- fulness Outcome Measure- ment Poverty Profile Customer Archetypes BASED ON YOUR DATA + The question sets in this guide are tightly connected to a desired action and enable companies to make better, more informed decisions. This ultimately drives higher sales and more impactful businesses. 4. GET READY TO TAKE ACTION BASED ON YOUR DATA Take Action Based On Your Data
  • 13. 13The Lean Data Field Guide QUESTION SET 1: VALUE PROPOSITION Use this question set to understand + Reason for purchasing + Perceived value for money + Quality How would you rate the value for money of (product/service)? A. It is expensive B. It is fair C. It is cheap Please explain your answer Q Why did you purchase (product/service)? Q How is the quality of (product/service) compared to the (product/service) you were using before? A. It is better B. It is the same C. It is worse Please explain your answer Q What feature do you like best? Please explain your answer Q Question Sets
  • 14. 14The Lean Data Field Guide Q Use this question set to understand: + How likely your customers are to recommend your product/service + Suggestions for improvement + Customer experience and satisfaction Note On Our Amended NPS: After testing the traditional NPS developed by Bain, we modified the phrasing to be more applicable across different regions. We found that our customers have trouble understanding the “Would you recommend” language and the 0-10 answer scale. Our revised version aims to get similar data as the NPS, with language that we’ve found is clear to the customers we are serving. QUESTION SET 2:CUSTOMER SATISFACTION Have you ever recommended (product/service) to your friends? A. Yes, I have told many friends What have you told them? B. Yes, I have told some friends What have you told them? C. No, I have not Why not? How can we improve (product/service)? Q Is there anything else you’d like to share with us? TIP: This question works well at the end of a survey Q Amended Net Promoter Score (NPS) Question Sets
  • 15. 15The Lean Data Field Guide + This toolkit is a combination of our best questions on usage, value proposition, customer satisfaction, and meaningfulness. + When asked in this sequence, this question set is our shortest and simplest way to understand depth of impact. QUESTION SET 3: MEANINGFULNESS TOOLKIT Q When do you use your [product/service]? A. Everyday B. 3-4 times a week C. 1-2 times a week D. Less than once a week E. It’s not working Q There have been changes in my home because of [product/service] A. Strongly agree B. Agree C. Neutral D. Disagree E. Strongly disagree Q Have you ever recommended [product/service] to a friend? A. Yes, I have told many friends. Please explain what you said B. Yes, I have told some friends. Please explain what you said C. No, I have not. Why not? Q Were these changes positive or negative? A. Positive i. Please explain the positive changes in your life because of (product/ service) ii. Out of those changes, which one is most important to you? B. Negative. Please explain your answer Question Sets
  • 16. 16The Lean Data Field Guide + To understand if social outcomes are improving as a result of a product/service, recall-based questions are useful. + However, when relying on customer memory, keep in mind that the accuracy of responses diminish after roughly one month of purchasing a new product/service. + To avoid recall error, try to survey new customers. Alternatively, you can do a baseline / endline, and survey the same customers, asking the same questions, at different points in time Example How to ask about the effect of solar lighting on kerosene expenditure and quality of available light Q Do you know how much your family spends on kerosene? A. Yes B. No Q (If yes) Now that you have a solar light, what did you spend on kerosene last week? Q Before you had a solar light, what did you spend on kerosene each week? Q How is the quality of your light compared to what you had before? A. Much brighter B. Somewhat brighter C. The same D. It is less bright QUESTION SET 4: SOCIAL OUTCOMES Question Sets
  • 17. 17The Lean Data Field Guide We use the Progress out of Poverty Index (PPI) developed by Grameen Foundation to understand: + Percentage of customers living below poverty line + Customer segmentation, e.g. different household profiles of customers QUESTION SET 5: POVERTY PROFILE How to conduct a PPI Go to: http://www.progressoutofpoverty.org/ Create a free account Click on “Find your PPI” and select your country of interest Download the country specific “PPI Scorecard and Lookup Table” Follow the instructions to conduct the PPI survey questionnaire. Aim for a minimum sample of 200 respondents Once you have the data, budget 30-60 minutes to complete the analysis using the scorecard and lookup tables to work out what percentage of customers live below the poverty line 1 3 5 2 4 6 Question Sets
  • 18. 18The Lean Data Field Guide Q QUESTION SET 6: CUSTOMER ARCHETYPE + When marketing a product/service, understanding the attitudes, behaviors, and preferences of your customers (and non-customers) is key. + This question set highlights our recent thinking about how to build and understand your customers along attitudinal lines vs. simple demographics Tip Don’t forget about surveying non-customers! You can glean important insights on how to better market your product to meet their needs. What did you buy in the last year that has had a big impact on your life? What did you buy in the last year that has had a big impact on your life? Question Sets Q What is something you hope to own in the next 5 years? Q How likely are you to purchase a new product before any of your friends own it? A. Very likely B. Likely C. Neutral D. Unlikely E. Very unlikely
  • 19. 19The Lean Data Field Guide Introduction “HOW TO” GUIDES
  • 20. 20The Lean Data Field Guide HOW TO: CHOOSE A SAMPLE SIZE “How To” Guides Golden rule Sample of size of 250 is sufficient in most cases! Useful definitions + Confidence level: is the probability that your sample accurately reflects the attitude of your population +Margin of error: the range (measured as a %) that your population’s responses may deviate from your sample Visit this link for additional details: https://www.surveymonkey.com/mp/sample-size-calculator/ 10K 50K 1M+ 5% 5% 5% Number of customers Margin of error Sample size 162 164 370 382 204 207 Confidence Level 262 268 164 385208 269 80% 95%85% 90%
  • 21. 21The Lean Data Field Guide Clever cost-effective ways to start gathering mobile contact information: Customer Registration Make Phone Number a “must-have” field in the customer registration process. Can use mobile-apps or tablets to make data capture more efficient and error-free (vs. paper entry) Radio + SMS/IVR For potential customers, hold a radio campaign to encourage SMS/ IVR inquiries: e.g. “call/SMS this number free to find out more and be entered into a lottery” Join Codes Place a number to register your product on your packaging This word is called a JoinCode and can be the name of your product or company or any other word This number is set by your JoinCode provider Choose a credible incentive for your customer to text the joincode “How To” Guides HOW TO: GET CUSTOMER PHONE NUMBERS 1 2 3 80249 for FREE Text [HELLO] To register your warranty / sign up for offers / enter a lottery etc
  • 22. 22The Lean Data Field Guide When to use focus groups + To dig deeper into perceptions and attitudes and get feedback on product ideas and marketing campaigns + Particularly useful for understanding non-customers + Advantages: allows you to study people in a more natural conversation pattern than typically happens in one-to-one interviews. Efficient: collect views from multiple people at once “How To” Guides HOW TO: RUN A FOCUS GROUP Who: 6-12 people, don’t know each other but feel comfortable together (think about gender, age, income level when selecting participants). 1 trained moderator, 1 assistant moderator to take notes Set-up: chairs in a circle. Example locations: quiet hotel, café, restaurant. Outdoors in shade can also work well Feel: Warm, relaxed environment. Offer drinks and small monetary incentive to say thank you for participation. It’s important to make everybody feels comfortable to share views. Length: 45-90 mins. 8-9 question areas What makes a good focus group 1. Welcome: Thank participants. Introduce moderators and purpose of focus groups “We have been asked by _____ to find out about ______. We need your input and want you to share your honest and open thoughts with us” 2. Ground Rules: “(1) We want you do the talking. We would like everyone to participate. I may call on you if I haven’t heard from you in a while. (2) No right or wrong answers: every person’s experiences and opinions are important. Speak up whether you agree or disagree. We want to hear a wide range of opinions. (3) What is said in this room stays here. Your answers are completely anonymous. (4) Any questions? Is the process clear?” 3. Focus Group Questions: See do’s and don’ts on the next page 4. Close out: Ask “Is there anything we missed?” and thank for participation How to run a focus group
  • 23. 23The Lean Data Field Guide “How To” Guides It’s easier than you think! All you need to do is: + Find a moderator who understands the local context (Impact team can help) + Secure a suitable venue for 2 hours + Recruit participants: go to local market ask “are you free [insert time] for 2 hours for a focus group? You will be compensated for your time” + Prepare a thoughtful moderator guide: what questions do you want to ask? Setting up a focus group http://www.focusgrouptips.com/focus-group-questions.html For helpful tips on question ordering and phrasing http://www.eiu.edu/~ihec/Krueger-FocusGroupInterviews.pdf For a full guide on how to design and conduct a focus group http://www.theexperiencebusiness.co.uk/downloads/content-docs/top_ten_tips.pdf Top 10 tips for great focus groups Additional guides Ask open-ended Qs, e.g.: “What do you like best about… what are the biggest problems with… what comes to mind when xyz” etc Follow up if unclear—”will you tell me more about that?” Ask for specifics if you get a very general response to a question Ask others in the group their views are when you only have heard one person’s opinion Seek out input from all participants Ask many “yes” or “no” questions. Ask more than one question at a time. Ask leading questions (“Isn’t this true that Y is important?”) “correct people” or tell them they are wrong about something Be afraid of asking for clarification Pressure someone to speak Question Do’s Don’ts Do: Don’t:
  • 24. 24The Lean Data Field Guide Introduction Website www.acumen.org Twitter @Acumen