UCA: Data Gathering Techniques. Selection criteria

Editor's Notes

• #2: Hello, my name is Cristina Cachero, and I am associate professor at the University of Alicante. In this video we are going to dive into the parameters that influence the selection of one data gathering technique or another.
• #3: This selection of the most suitable technique depends on several parameters: which data need to be collected? How many people will be involved in the research? Do we need specific data (what is exactly happening, how, when and where) or exploration data, more qualitative in nature (for example, why questions). How complex is the issue we are trying to understand? Which are our budget restrictions? Do we need to collect information about user errors or misconceptions? Answering these and other questions can help us create a decision tree to select the most appropriate technique.
• #4: Some companies, such as Human Factors International (www.hfi.com), in an attempt to simplify this decision process for novice usability analysts have created a cheatsheet that, for a number of methods, indicates their suitability depending to the answer to these questions. For example, imagine that we need to collect data about the features needed for the new application to better support the customer service of our company.This would incline us towards working with a direct method, where you can probe the user. Surveys could be another option. If we now add that we have 10 people available to work with, this would discard surveys, since there are too few subjects to make it cost-worthy. Also imagine that this is the beginning of the study, so we are interested in exploratory data. Again, this situation inclines us towards more conversational-like methods, such as interviews or observation followed by discussion. We can assume that there are some complex issues involved in the service (complex task flows with many exceptions). This strongly suggests the use of interviews and/or observation, where we can ask for further clarifications when needed. If, furthermore, data on user errors while using the actual system (which is preventing customer service workers from giving correct solutions to the users) is needed, this would make it advisable to use observation, since users are not prone to reporting their own errors... many times the are not even aware of them!
• #5: Now let's work with some examples that help us further get acquainted with the different data gathering techniques mentioned in this video. Imagine that our software application to support the customer service is being redesigned in an iterative manner. The goal of the first increment consists in fixing the most important complaints about the current design. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
• #6: Now imagine we want to develop an on-line tool to help people decide when to go to the emergency room in a hospital and when to wait for the doctor to fix an appointment with them. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
• #7: In our third setting, imagine that there is a lot of people coming to your e-commerce website, but few of them are buying and turning into customers. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
• #8: In our fourth example imagine that we want to design a data entry device for ambulance workers for them to register what happened in each call and check the patient’s medical records. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
• #9: The last situation involves an internal operations application that is so difficult to use that important company processes are being ignored, critical data are not being captured, and so on. What would be, according to you, the most suitable data gathering technique? Please, stop the video now and write down your answer.
• #10: Here are my selection of techniques for the situations exposed. Please, take into account that there are many nuances in the context that may cause the selection of techniques to change. So there is not absolute right or wrong answer. And remember, the best way to go is to try to combine different data gathering techniques so that the reliability of your data increases! . In the customer service app, my first option would be to go through the existing reports they issue, in order to get to grips with the exact information they need to gather and how they summarize it. Then, I would try to interview some customer service workers to check what is working and where do they find difficulties making the system adapt to their needs, and finally I would try to observe at least a couple of workers as the work. Customer Service (and, in general, any job post that requires direct contact with customers) is subject to lots of stress, and therefore understanding the work environment under which those workers operate may change the way in which we tackle the redesign of the user interface. Following the same kind of reasoning, for the app to decide whether to go or not to go to the emergency room I would interview doctors (few) and run surveys on patients (many) in order to get the exact language for the application right. Regarding the e-commerce web site, the first (and most economic-effective) thing to do would be to run any kind of web analytics, to check where customers are abandoning the purchase process. Then, in order to answer why they are leaving, I would go for interviews or user observation. For the design of an ambulance data-entry device, I would use interviews, since there is a complex process involved, and getting into the ambulance to observe the whole process may not be possible. Last but not least, for the internal operations application I would run a round of interviews, since the complexity of the issue is high.
• #11: The importance of selecting the right technique cannot be overstated. As a real example of how not understanding the possible biases introduced by the different techniques can lead to billionaire losses, please read the Wallmart case of study. Remember to take some notes, since you will be asked about it in the quizz that accompanies this part of the course!
• #12: So, this is all for now. Hope you have enjoyed this video. See you in next section!