Designing with Data

Editor's Notes

• #2: Hey and thanks for joining this MasterClass. I’m really excited to talk today about this exciting future Quick poll: how many designers / product people / founders / marketers / podcasters?
• #3: There are many types and definitions for design. Design = a tool for asking questions, Designing with Data = building things that are uniquely personalized to the user (experiences, products, campaigns)
• #6: “For designers, it means being able to understand users more holistically through their interactions with the interfaces and experiences we build. This understanding will allow us to better anticipate and meet users’ needs, elevate their capabilities and extend their reach.” - Machine Learning for Designers Designing a model for enabling a machine to successfully deliver user experiences (“Machines will increasingly be making decisions within user experiences, and co-designing with them is an essential partnership for the future of product design.”) Personalizing requires designing with an eye for variability within the product. Variability has evolved from manual, hard coded user settings to fully dynamic, self-training products
• #7: “For designers, it means being able to understand users more holistically through their interactions with the interfaces and experiences we build. This understanding will allow us to better anticipate and meet users’ needs, elevate their capabilities and extend their reach.” - Machine Learning for Designers Designing a model for enabling a machine to successfully deliver user experiences (“Machines will increasingly be making decisions within user experiences, and co-designing with them is an essential partnership for the future of product design.”) Personalizing requires designing with an eye for variability within the product. Variability has evolved from manual, hard coded user settings to fully dynamic, self-training products
• #10: The Omada program digitizes the components of the DPP: a digital scale and website and mobile app that deliver weekly lessons, a remote health coach, a support group, and challenges. The program is purchased by employers and insurers (B2B2C) to offer to their employees as a free benefit. The ‘Foundations’ program focuses on building positive behaviors in 4 key areas within the first 16 weeks, after which Participants have access to the tools for the subsequent 2+ years.
• #11: The Omada program digitizes the components of the DPP: a digital scale and website and mobile app that deliver weekly lessons, a remote health coach, a support group, and challenges. The program is purchased by employers and insurers (B2B2C) to offer to their employees as a free benefit. The ‘Foundations’ program focuses on building positive behaviors in 4 key areas within the first 16 weeks, after which Participants have access to the tools for the subsequent 2+ years.
• #12: We have to figure out how to intelligently deliver personalized interventions to help change behavior change in fundamental aspects of PPTs lives…or we cease to exist as a business. Enrollment fee We have to actually help people lose weight (and keep it off) to make money Employers and insurers only pay for PPT who engage and excel in the program
• #14: A brief history
• #16: Personalization refers to giving the users what they need without them having to ask for it. It is an intelligent UX that learns and adapts to the user based on behavior. Customization is when the user sets their preferences among existing choices. Privacy, security, notifications, theme templates — these all fall under customization. The system is not thinking on its own, the user must first do all the work. Personalization has an added feeling of magic, while it delivers 2 things for the user: (1) an intuitive sense of ease (by minimizing steps) and (2) development of a connection and trust with the product over time.
• #17: Personalization refers to giving the users what they need without them having to ask for it. It is an intelligent UX that learns and adapts to the user based on behavior. Customization is when the user sets their preferences among existing choices. Privacy, security, notifications, theme templates — these all fall under customization. The system is not thinking on its own, the user must first do all the work. Personalization has an added feeling of magic, while it delivers 2 things for the user: (1) an intuitive sense of ease (by minimizing steps) and (2) development of a connection and trust with the product over time.
• #18: Personalization refers to giving the users what they need without them having to ask for it. It is an intelligent UX that learns and adapts to the user based on behavior. Customization is when the user sets their preferences among existing choices. Privacy, security, notifications, theme templates — these all fall under customization. The system is not thinking on its own, the user must first do all the work. Personalization has an added feeling of magic, while it delivers 2 things for the user: (1) an intuitive sense of ease (by minimizing steps) and (2) development of a connection and trust with the product over time.
• #19: Personalization refers to giving the users what they need without them having to ask for it. It is an intelligent UX that learns and adapts to the user based on behavior. Customization is when the user sets their preferences among existing choices. Privacy, security, notifications, theme templates — these all fall under customization. The system is not thinking on its own, the user must first do all the work. Personalization has an added feeling of magic, while it delivers 2 things for the user: (1) an intuitive sense of ease (by minimizing steps) and (2) development of a connection and trust with the product over time.
• #20: Personalization refers to giving the users what they need without them having to ask for it. It is an intelligent UX that learns and adapts to the user based on behavior. Customization is when the user sets their preferences among existing choices. Privacy, security, notifications, theme templates — these all fall under customization. The system is not thinking on its own, the user must first do all the work. Personalization has an added feeling of magic, while it delivers 2 things for the user: (1) an intuitive sense of ease (by minimizing steps) and (2) development of a connection and trust with the product over time.
• #30: Related Content: “Things like this” is the most common and easy to execute as it can be done with basic key words. Commonly seen in advertising when items you’ve viewed and those like it follow you around the internet. Recommendations are a slightly richer version of personalization that combine basic contextual data (previously viewed, your ratings, relevant ratings) to suggest options you might like. E-commerce, Netflix, Pandora Chatbots: Sensor-based personalization supercharge their basic functionality and shortcut steps based on information gathered from your location, activity (accelerometer), times of use and other automatically captured inputs. Active Personalization are used by apps that require personalization to function and leverage information manually gathered up front (surveys), interactivity data, and passive data to uniquely adapt the entire of the experience.
• #32: Manual rules-based contextualization allows business users to explicitly define various consumer segments based on input variables. Automated processes display content or offers based on automated filtering of numerous inputs that may be both explicit (e.g., previous purchases and stated preferences) and implicit (e.g., click path, referring site, and search terms).
• #33: A brief history
• #34: This may sound obvious, but how you think to capture, structure, and organize data can go a long way in enabling you to build richer and more effective solutions down the road.
• #35: Previously viewed pages & past purchases — Another personalization cornerstone, seeing what pages/products your users preferred is a nearly foolproof way of noting their personal preferences. Location — Whether through an IP address or a geolocation tracker, where the user is — at least a broad view — is easy to track. This is a useful personalization tool for area-specific recommendations, and for tailoring content to regional tastes. Device — Basic analytics can determine which devices your users choose. If you notice trends, this can lead to more potent adaptive design templates to take advantage of popular devices’ strengths. Previously viewed pages & past purchases — Another personalization cornerstone, seeing what pages/products your users preferred is a nearly foolproof way of noting their personal preferences.
• #36: Sign-up questions — If you’d like to know something in particular about your user, ask them during the sign-up survey. Typically this goes no further than demographic information (gender, age, etc.) but there is room for experimenting. For example, a site for a music player might ask the user’s favorite band or style of music. Personalization questionnaires — For sites that offer a specific service based on the user’s personal information (like Sage above) a personalization questionnaire is necessary. These ask direct questions that the site needs to perform the service. Ratings — Ratings go a step further than previously viewed pages and past purchases by providing more details on the user’s emotional connection. People often view pages (and sometimes make purchases) that do not reflect their true likes and preferences — ratings clarifies which of these are genuine interests, and by how much. When applied correctly, these lead to useful recommendations (like Amazon or Netflix).
• #37: Adapting the experience based on past usage is another method that’s quite effective for personalization. In this sense, the product “learns” about the user over time, increasing its accuracy with more usage (which, if effective, will increase usage even more).
• #38: Adapting the experience based on past usage is another method that’s quite effective for personalization. In this sense, the product “learns” about the user over time, increasing its accuracy with more usage (which, if effective, will increase usage even more).
• #39: This may sound obvious, but how you think to capture, structure, and organize data can go a long way in enabling you to build richer and more effective solutions down the road.
• #40: Historically, personas have been a tried and true means for designers synthesizing the range of user types by a baseline set of important behaviors (based on secondary and qualitative / user research), primarily for the designer to have clarity of who they’re designing for But these are incomplete, because they… Gives you a more complete picture of what you know about users across the gamut (which can trigger new ideas) Allows you to, with each new problem, re-evaluate, prioritize and re-shape personalization needs based on the problem you’re trying to solve (rather than across the system) > done through subgroups / advanced segments (that can be adapted based on the problem / aren’t cross-product) Create advanced user segments — As you gather the data, you’ll see natural patterns and clusters. You can combine what you know about the user’s psychology (motivations/fears/goals/etc.) with behavioral data to create robust segments. To identify friction points to be smoothed over by personalization, try creating user scenarios and customer journeys for each segment. Makes the leap between user design goals and how to build and adapt them based on the data
• #41: Previously viewed pages & past purchases — Another personalization cornerstone, seeing what pages/products your users preferred is a nearly foolproof way of noting their personal preferences. Location — Whether through an IP address or a geolocation tracker, where the user is — at least a broad view — is easy to track. This is a useful personalization tool for area-specific recommendations, and for tailoring content to regional tastes. Device — Basic analytics can determine which devices your users choose. If you notice trends, this can lead to more potent adaptive design templates to take advantage of popular devices’ strengths. Previously viewed pages & past purchases — Another personalization cornerstone, seeing what pages/products your users preferred is a nearly foolproof way of noting their personal preferences.
• #42: Internal profiles give everyone on the product team a rich view of each user It also allows us to evaluate and prioritize the key user attributes we want to target for a given problem What comes out are subgroups that can be targeted and evaluated once the experiment ships
• #43: Internal profiles give everyone on the product team a rich view of each user It also allows us to evaluate and prioritize the key user attributes we want to target for a given problem What comes out are subgroups that can be targeted and evaluated once the experiment ships
• #45: Are you wanting to learn deductively or inductively? What resources (technical, time) are at your disposal? Is the solution very meaningful to the users? Can you segment your users in a meaningful way? Do you have the data set to build an algorithm-based solution / train a model?
• #47: Defined = has a clear (1) user goal and (2) business goal / metric and (3) static location within the experience [Example: Undefined = Does not have all 3 of the above One-Dimensional = lives in one, static place within a product experience Multi-Dimensional = lives across multiple touchpoints Optimization: take a known thing and train a model to deliver it better than a human could Experimentation: take an unknown thing and prove where it should live / exist
• #48: Defined = has a clear (1) user goal and (2) business goal / metric and (3) static location within the experience [Example: Undefined = Does not have all 3 of the above One-Dimensional = lives in one, static place within a product experience Multi-Dimensional = lives across multiple touchpoints Optimization: take a known thing and train a model to deliver it better than a human could Experimentation: take an unknown thing and prove where it should live / exist
• #49: Defined = has a clear (1) user goal and (2) business goal / metric and (3) static location within the experience [Example: Undefined = Does not have all 3 of the above One-Dimensional = lives in one, static place within a product experience Multi-Dimensional = lives across multiple touchpoints Optimization: take a known thing and train a model to deliver it better than a human could Experimentation: take an unknown thing and prove where it should live / exist
• #50: Every design and product solution can be boiled down to one hypothesis statement Most design solutions should be measurable Identify key hypotheses / metrics to design and evaluate against How can this be broken into a small piece to test learn? Narrowest possible frame?
• #51: Every design and product solution can be boiled down to one hypothesis statement Most design solutions should be measurable Identify key hypotheses / metrics to design and evaluate against How can this be broken into a small piece to test learn? Narrowest possible frame?
• #52: Convert the hypotheses into-frame based opportunities
• #53: Defined / Unidimensional [algorithm+rules] : Auto-food feedback (content optimization)
• #54: Defined = has a clear (1) user goal and (2) business goal / metric and (3) static location within the experience [Example: Undefined = Does not have all 3 of the above One-Dimensional = lives in one, static place within a product experience Multi-Dimensional = lives across multiple touchpoints Optimization: take a known thing and train a model to deliver it better than a human could Experimentation: take an unknown thing and prove where it should live / exist
• #55: Every design and product solution can be boiled down to one hypothesis statement Most design solutions should be measurable Identify key hypotheses / metrics to design and evaluate against How can this be broken into a small piece to test learn? Narrowest possible frame?
• #56: Every design and product solution can be boiled down to one hypothesis statement Most design solutions should be measurable Identify key hypotheses / metrics to design and evaluate against How can this be broken into a small piece to test learn? Narrowest possible frame?
• #57: Convert the hypotheses into-frame based opportunities
• #58: Defined / Unidimensional [algorithm+rules] : Auto-food feedback (content optimization)
• #60: Set up competitive metrics Measure against hypotheses Look at primary vs. secondary If primary metrics are succeeded: enhance or ‘productize’ If primary metrics aren’t work but secondary are: Decide to re-purpose frame and refocus based on signal
• #62: A brief history
• #63: Qualitative
• #64: you can’t test or evaluate data for that which doesn’t exist Decisions should be made both on the quantitative factors AND the why’s behind them...that forms a complete story Iterate, test more, extend the design, balance will the full, projected potential of the idea and find the balance) Iterate on the data analysis like you would the design. What else can I analyze? What else should be or wasn’t measured?
• #65: Qualitative
• #67: Hey and thanks for joining this MasterClass. I’m really excited to talk today about this exciting future Quick poll: how many designers / product people / founders / marketers / podcasters?
• #68: Escalating healthcare prices has suggested a need to start earlier and focus on prevention and management Transition from treatment to incentives that encourage long term, value based-care Paired with a patient empowerment movement: patients are taking control of their own health AND being asked to do by doctors
• #69: On the far left, you have those who are already generally healthy and likely to remain so. The wellness population is largely likely to remain that way, with or without intervention. On the far right, you have those who are unfortunately already suffering from a chronic disease that requires on-going and costly management, like type 2 diabetes or heart disease. Big drug companies already targeting, employers / insurers already having to cover. (and irreversible). But there’s a very important group in between – those who are at high-risk of developing certain chronic but largely preventable disease. On the brink of tipping over into a very dangerous and very expensive condition like type 2 diabetes or heart disease There’s still time to intervene and have a different outcome. Experts predict that chronic disease will be the biggest driver of healthcare costs for the foreseeable future, especially if you do nothing.
• #70: People that don’t yet have a disease = a different model than traditional healthcare (not another drug) This population doesn’t need a pill or wellness initiative, they need a professional behavioral intervention. There’s no drug for changing their habits (but if they don’t change their habits, they’ll need one)
• #73: $474 billion a year spent on type 2 and cardiovascular disease ($254b type 2 / $320b cardiovascular) Those with chronic disease cost our system over 5x that of a healthy individual And those costs get passed along to us (we all have to pay more to cover it when everyone else is getting sicker)
• #74: Not to mention the very real human costs. Consider what someone with diabetes faces each and every day (and this is just a partial list)… Medications Fatigue Breathing difficulty Fractures Insulin shots Blood tests Anxiety and depression Scrutinizing food labels
• #75: Landmark Diabetes Prevention Program (DPP) clinical trial validated: health coach lead, intensive behavioral counseling as standard of care for those at risk of weight-related chronic disease.
• #76: But this new gold standard for behavioral medicine involved in person interventions. Some challenges associated with scaling this model… To serve the 87M Americans who need behavioral counseling, given a group size of 20, we would need 4.35M health coaches Access and reach (remote and rural participants) Availability, not dependent on location or synchronous schedules Lack of monitoring and analysis
• #78: Founded as part of an internal project with the global design firm IDEO in 2011 With the intention of building a digitized, human-centered version of healthcare to be able to deliver the DPP at scale.
• #79: The only lever we have are the moments when people are in the program (and the experiences people have in these moments) Design is the intervention. We need to design to enable using the highest impact touchpoints as possible (timely, salient)
• #80: More able to meet PPT where they actually stand than past RCTs (throughout their day to day lives) The model we’re building is a v2.0 of changing behavior extensible to other chronic diseases and behavioral issues)
• #81: Hey and thanks for joining this MasterClass. I’m really excited to talk today about this exciting future Quick poll: how many designers / product people / founders / marketers / podcasters?
• #82: Behind the scenes, we’re also building a baseline (app, survey, points of engagement) You can notice an interesting “pointy” structure around 100-130 days For context, there are 112 days in the 16 week Omada program…which could be a hint about the relationship between frequency of weigh in and weight loss.
• #83: 2008 study, with an N of over 1680 Tracking behavior creates self-awareness (think more about the before, during and after of a behavior) Awareness has a ripple effect on behaviors throughout the day And more conscientiousness about an activity has the ability to change the individuals mindset in how they approach that activity
• #84: On the back end, we’re experimenting too Nudge, not judge Feedback shouldn’t simply be informational; it should be actionable. Tell us what you (really) did today, and we can help you eat better tomorrow.”
• #85: Asking open ended questions (“how are you feeling about your choices today?”) vs. direct, specific questions (“what was in that protein shake?) lowers the probably of food-tracking by 10%.
• #86: Like any product, we want to make something that people will use and benefit over time and fight the natural decay of any product
• #89: Through techniques like this, we take the randomized control trials process and make it as lean as possible to maximize learning Turn Omada into a system that improves with scale Pave the way for true personalization of programs, optimize for the populations that benefit most 0.4% is roughly 8% of an average 5% weight loss between the week 16 weight loss of the control group (mu_x)- that of the treatment group (mu_y)
• #90: One and two-year data from a longitudinal cohort we’ve been following since 2012
• #91: The notion of making ‘the right product’ is a notion of the past. We’re building an experience that needs to adapt to the individual in the moment. Meet someone where they are in the moment AND change them. It requires a living organism. Anticipate people to move through emotional states with the product (it may be different 2 weeks from now) Build a dynamic system (that provides users with a variety of techniques to experiment with and gather data on) Create multiple channels (especially for learning) to cater to the broad range of types of learners your users might be Meet people where they are (readiness to change, especially for chronic disease)