Q qualitative-and-quantitative-research
- 1. May 2012 Qualitative and Quantitative Research Techniques for Humanitarian Needs Assessment An Introductory Brief
- 2. 2 Table of Contents 1. Introduction to Qualitative and Quantitative Research.................................................... 3 2. Data Sources................................................................................................................. 3 3. Types of Data ................................................................................................................ 4 3.1 Quantitative Information.......................................................................................... 4 3.1.1. Quantitative Research Methods ........................................................................... 5 3.1.2. Strengths and Weaknesses of Quantitative Research.......................................... 6 3.2 Qualitative Information............................................................................................ 7 3.2.1. Qualitative Research Methods ............................................................................. 8 3.2.2. Strengths and Weaknesses of Qualitative Research............................................ 9 4. Data Collection Techniques ......................................................................................... 10 5. Conclusion................................................................................................................... 13 6. Key Resources ............................................................................................................ 14
- 3. 3 1. Introduction to Qualitative and Quantitative Research Within the context of phase I and phase II rapid needs assessments during the initial days of an emergency response, both qualitative and quantitative information is needed to develop a shared understanding of how people are affected by emergencies1 . Quantitative data guides in understanding the magnitude and scale of a humanitarian crisis by providing a numeric picture of its impact upon affected communities. It addresses the questions: how many and how much. Qualitative data, on the other hand, focuses on determining the nature of the impact of a disaster upon affected populations. Qualitative data answers questions of how and why coping strategies have adapted, or failed to adapt, to the changed circumstance. Collection, collation, analysis, and synthesis of qualitative and quantitative information, gathered and analysed using appropriate sources, tools, and methods is the cornerstone of rapid needs assessments that allows decision makers to plan a timely, appropriate, and coordinated emergency response. When undertaking a needs assessment, a combination of different types and sources of data is required to build a holistic picture of the affected population. Sources for information include both primary and secondary data. Types of information include qualitative and quantitative data. 2. Data Sources Primary data is most generally understood as data gathered from the information source and which has not undergone analysis before being included in the needs assessment. Primary data is collected directly from the affected population by the assessment team through field work2 . Primary data is most often collected through face to face interviews or discussions with members of the affected community, but can also be gathered through phone interviews, radio communication, email exchange, and direct observation. Secondary data is information which has typically been collected by researchers not involved in the current assessment and has undergone at least one layer of analysis prior to inclusion in the needs assessment. Secondary data can comprise published research, internet materials, media reports, and data which has been cleaned, analysed and collected for a purpose other than the needs assessment, such as academic research or an agency or sector specific monitoring reports. During phase I of an emergency assessment, the majority of data used to build a shared picture of the disaster affected area and populations comes from secondary sources. This is largely because time constraints during the first few days following a sudden onset disaster prohibit a large scale field data collection exercise. As the emergency evolves and humanitarian stakeholders, and the assessment team, have greater direct access to the 1 The OCHA NATF framework indicates that phase I is the initial 72 hours following a sudden onset disasters, during this time the initial emergency assessment is carried out. Phase II is the first two weeks during which time a rapid assessment is carried out. Due to the extended time frame in protracted emergencies, these phases are less applicable. See IASC, Operational Guidance for Coordinated Assessments in Humanitarian Crises, February 1, 2011. 2 Certain published information, such as census data, can also be considered primary data.
- 4. 4 affected population, the proportion of primary data will increase and the consolidated analysis of both types of data is necessary. This is increasingly true as phase II segues into phase III. Clearly understanding the information gleaned from secondary sources frees the primary data collection from a joint or coordinated needs assessment to focus on key information gaps (issues that are presently unknown) and on ensuring that the voice, needs and priorities of an affected population are captured and shared. All field and desk information gathering activities for needs assessments will: Collect evidence on the impact of the disaster across sectors Produce findings about the disaster which are not already known. Triangulate information collected to confirm, or dispute, findings. Investigate the effect of a change of circumstance (directly or indirectly due to the disaster) on a population. Use a pre-defined set of research techniques to ensure consistency in data collection, analysis and presentation of findings. 3. Types of Data The different types of data required for a needs assessment are most easily understood using the descriptive terms: qualitative and quantitative. Both primary and secondary data can be either qualitative or quantitative. The difference is in the type of information collected, the questions and information requirements that the data is meant to address, and the methods used to analyse it. 3.1 Quantitative Information Quantitative research methods are characterised by the collection of information which can be analysed numerically, the results of which are typically presented using statistics, tables and graphs. For phase I of assessments, the majority of quantitative data collected is secondary data (e.g. affected population figures provided by the government). During phase II, field level questionnaires complement the continued collection of secondary data through the collection of quantitative information using close ended questions, typically in questionnaire format. During phase I and II of an assessment, there will be limited primary quantitative data collected from a joint field assessment process (i.e. a multi sector assessment with the buy- in and support of multiple agencies) because of time and access constraints. Quantitative information collected through primary data collection will be relevant only to the visited sites and cannot be generalized for all affected areas and groups. It will tell little about the big picture due to the limited sample size and sampling methodology. For example, if in 30 sites visited for primary data collection it is found that the number of newly arrived IDPs is twice the total number of pre-disaster inhabitants, this does not mean that in all affected
- 5. 5 communities IDPs now comprise twice the pre-disaster population. However, quantitative information will enhance a better understanding of the situation at the site level and help stakeholders recognise trends resulting from the disaster’s impact. Nevertheless, being able to quantify the magnitude and scope of the crisis is critical to the decisions made in these phases of a disaster. Quantitative information required to feed into this information gap will usually come from government or other official sources, based on pre-crisis census information and population projections for areas known to be affected by the disaster rather than from extrapolation from a small number of surveyed sites. At the early stage of a disaster, demographic information including estimates of numbers of children under 5, pregnant women, older persons, persons with disabilities and other vulnerable groups should also be based on pre-crisis secondary information. 3.1.1. Quantitative Research Methods The aim of the quantitative research method is to test pre-determined hypotheses and produce generalizable results3 . Using statistical methods, the results of quantitative analysis can confirm or refute hypotheses about the impact of a disaster and ensuing needs of the affected population. They can also measure impact according to humanitarian indicators. Conclusions made from the analysis of quantitative data indicate how many are affected, where the greatest area of impact is, and what are the key sector needs. Scientific measurement is key to quantitative research. Because quantitative data is numeric, the collection and analysis of data from representative samples is more commonly used. In its simplest terms, the more representative the sample is, the more likely it is that a quantitative analysis will accurately and precisely reflect a picture of the impact of the disaster when generalized to the whole affected population. However, even a representative sample is meaningless unless the data collection instruments used to collect quantitative data are appropriate, well designed and clearly explained to end users of the data4 . All too often, designers of data collection tools frame qualitative questions quantitatively and vice versa. Data collected using poorly designed questionnaires may solicit an enormous amount of data, but result in much of it being unusable as a result of being too difficult to measure and impossible to generalize for the total affected areas. Larger sample sizes tend to be used for collecting quantitative information, so as to gather as representative a picture as possible. However, in any assessment process, there is a trade-off between the representativeness and diversity of a sample and the efficiency and timeliness with which data can be collected5 . Assessments in phase I and II do not need to be as representative as they need to be rapid6 . Use of large represenative sample sizes does not typically happen until phase III of an assessment when their is sufficient time and access to enable sampling of households and individuals. 3 Marshall, MN 1996, Sampling for Qualitative Research, Oxford University Press, p 522. 4 See ACAPS TB on purposive sampling and site selection for phase 2 assessment. 5 For more information on sampling, see ACAPS, Technical Brief on Sampling and Site Selection, 2011. 6 A comprehensive sampling of affected households or individuals won’t be carried out until phase III when full diversity strata and representation of all affected groups can be included in the sample.
- 6. 6 Previous experience in assessments highlights the fact that measurable amounts of quantitative information is often collected during assessments, but not used. This type of redundant information falls into two main question categories7 : Questions with integrity, but asked by members of an assessment team who lack the capacity and/or time to analyse the responses. For example, the question how much did you spend last week for your food? is a useful question, but with up to a dozen potential answers, no baseline reference to compare to, and limited resources for data analysis is too detailed to be used critically. Questions that are valid, but technically difficult to obtain valid answers to, given the capacities of the enumerators. For example, asking questions pertaining to MUAC measurements are likely to lead to invalid and inaccurate entries, and an eventual discounting of the data, given the expertise, experience and capacity of enumerators. Box 1: Lesson Learned - Haiti8 3.1.2. Strengths and Weaknesses of Quantitative Research The advantage of legitimate quantitative data, that is data which is collected rigorously, using the appropriate methods and analysed critically, is in its reliability. However, the shortcoming of quantitative data is that it fails to provide an in depth description of the experience of the disaster upon the affected population. Knowing how many people are affected and their locations does not provide sufficient information to guide agencies and sectors on what they should plan for in terms of response. Knowing why there is a problem and how people are affected will combine with the numbers and locations to provide insight on how best to tailor the humanitarian response. For example, quantitative data collection may indicate categorically that 200,000 people were affected by a flood in four districts. This information would answer the questions: How many people have been affected by the flood? In how many districts? However, this data does not tell you what priority needs are for affected persons in light of the flood or how the flood has impacted traditional coping strategies. Additional quantitative data could be collected to determine specific needs by asking community members to rank a list of priority needs. But this would still fall short of explaining why these are the priority needs and how that impacts upon and is affected by local cultural and values. It would also 7 Examples taken from the Pakistan McRam 2010. 8 Data from ACAPS’ own field work, 2010. In previous multi cluster needs assessments, pressure from agencies to include a multitude of sector specific questions in quantitatively framed questionnaire tools resulted in assessment teams collecting quantitative data that was neither reliable nor analysable and thus unusable. In the Haiti RINAH, for example, 190 questions were included in the questionnaire, but out of the 76 that were usable and reliable for inclusion in the final RINAH report, none were quantitative.
- 7. 7 fail to provide information about priority needs for humanitarian intervention. To gather this information, an investigator would need to ask an open ended question, such as how has the disaster affected traditional coping strategies used by members of the community? or why are these the priority needs for your community? The main strengths of quantitative data collection are that it provides9 : numeric estimates opportunity for relatively uncomplicated data analysis data which are verifiable data which are comparable between different communities within different locations data which do not require analytical judgement beyond consideration of how information will be presented in the dissemination process. Weaknesses inherent in quantitative data include10 : gaps in information - issues which are not included in the questionnaire, or secondary data checklist, will not be included in the analysis a labour intensive data collection process limited participation by affected persons in the content of the questions or direction of the information collection process. Box 2: Lesson Learned - Pakistan11 3.2 Qualitative Information Qualitative research is by definition exploratory. It is used when we don’t know what to expect, how to define the issues, or lack an understanding of why and how affected populations are impacted by an emergency. Qualitative data like quantitative data is based on empiric investigation and evidence. However, qualitative research explores information from the perspective of both groups and individuals and generates case studies and summaries rather than lists of numeric data. 9 Adapted from the PARK companion, JIPS/ACAPS 2012. 10 Adapted from the PARK companion, JIPS/ACAPS 2012. 11 Data from ACAPS’ own field work, 2010. In Pakistan in 2008 during large scale conflict related displacement, the child protection cluster wanted to know numbers of separated children in each site visited. The sites comprised a small, purposive sample of all of sites in the area and varied significantly in population size and composition. Questions were asked to community groups about numbers of separated children. Asking for this information was taxing for the community groups to answer, and field teams found that male and female groups gave vastly different numeric answers to the question making it both impossible to resolve for each site and resulting in un-analysable information. While it would have been useful to know whether a trend of unaccompanied children resulted from the initial displacement, reliable data on numbers of unaccompanied children could not be generalised from the data, rendering the exercise of collecting this specific piece of data meaningless and wasteful of both time and resources.
- 8. 8 Qualitative data are often textual observations that portray attitudes, perceptions or intentions12 . Conclusions made from collected qualitative data take the form of informed assertions about the meaning and experience of certain (sub) groups of affected populations. The key contribution of qualitative data is that it provides information about the human aspect of the emergency by acknowledging context to the priority needs of affected populations and with it respecting the core principle of needs based assistance and ownership by affected populations. One major challenge for phase I and phase II assessments is finding the right balance in collecting and analysing qualitative information to identify trends and overarching issues for people affected by a crisis and to present this information appropriately. 3.2.1. Qualitative Research Methods Qualitative methods of research and analysis provide added value in identifying and exploring intangible factors such as cultural expectations, gender roles, ethnic and religious implications and individual feelings. Qualitative research explores relationships and perceptions held by affected persons and communities. As a result, smaller sample sizes chosen purposefully can be used for the following reasons13 : The larger the sample size for qualitative data collection is, the more complex, time consuming and multi-layered the analysis will be. For a true random sample to be selected, the characteristics under study of the whole population should be known, which is rarely possible at the early stage of an emergency. Random sampling of a population is likely to produce a representative sample only if the research characteristics are evenly distributed within the population. There is no evidence that the values, beliefs, attitude and perceptions that form the core of qualitative research are normally distributed, making the probability approach inappropriate. Some informants are more likely to provide greater insight and understanding of a disaster’s impact to the assessment team, due to a variety of factors including their social, economic, educational, and cultural position in the commuity. Choosing someone at random to answer a qualitative question would be analogous to randomly asking a passer by how to repair a broken car, rather than asking a garage mechanic. The qualitative sample must be big enough to assure inclusion of most or all of the perceptions that might be important. The smaller the sample size is, the narrower the range of perceptions that may be heard. The larger the sample size, the less likely it is that assessment team would fail to discover a perception that they would have wanted to know. In other words, the objective in designing qualitative research is to reduce the chances of discovery failure as opposed to reducing (quantitative) estimation error. In practice, the number of sample sites or groups becomes obvious as the assessment progresses, as new categories, themes and explanations stop emerging from the data (theoretical saturation). Clearly this requires a flexible assessment design and an iterative, cyclical approach to sampling, data collection, analysis and interpretation. 12 World Food Programme, Emergency Food Security Assessments, Technical Guidance Sheet #8, Introduction to qualitative data and methods for collection and analysis in food security assessments, February 2009, p2. 13 Adapted from 1996, Sampling for qualitative research.
- 9. 9 Data originally obtained as qualitative information about individual items may give rise to quantitative data if they are recoded or categorised numerically. Conversely, data that are originally quantitative are sometimes grouped into categories to become qualitative data (for example, income below $5/day, income between $6 and $20, and income above $20). Data gathered through qualitative methods is often presented in the form of a case study. However, as with all data, results can also be presented in graphs, tables and using other (traditionally) quantitative methods. It is important, though, to realise that just because qualitative information is presented in a graph, it does not suddenly become quantitative. Table 1 illustrates a graph of qualitative information which has been presented quantitatively. The issues presented are clear examples of qualitative information and indicate how affected people feel about the impact of a conflict on their security. The responses from the focus group discussion (FGD) can be quantified according to numbers of FGD participants, and indicate trends, but they cannot be extrapolated to apply to all persons affected by the crisis. Table 1: Perceptions of security for disaster affected persons 3.2.2. Strengths and Weaknesses of Qualitative Research The main strengths of qualitative data collection are that it provides14 : rich and detailed information about affected populations perspectives of specific social and cultural contexts (i.e. the human voice of the disaster) inclusion of a diverse and representative cross section of affected persons in depth analysis of the impact of an emergency a data collection process which requires limited numbers of respondents a data collection process which can be carried out with limited resources. 14 Adapted from the PARK companion, JIPS/ACAPS 2012. 65% 30% 4% 1% Perceptions of Security (data from 10 FGDs) Very Insecure Insecure Somewhat insecure Safe
- 10. 10 Weaknesses inherent in qualitative data are that it15 : results in data which is not objectively verifiable requires a labour intensive analysis process (categorization, recoding, etc.) needs skilled interviewers to successfully carry out the primary data collection activities. 4. Data Collection Techniques Both quantitative and qualitative data are empiric, but collect different kinds of information using different methods. Neither type of data is intrinsically superior to the other. Rather, the key in developing an accurate and holistic picture of the impact of a disaster on an affected population is in ensuring that the type of data collected matches with the method of collection to ensure the best possible data. Asking the right questions in the right ways at the right time and of the right audiences is vital to achieving reliable16 , valid, and usable assessment information. During the immediate days following a sudden onset emergency, assessment teams will be dependent primarily on secondary data. As access to affected populations and areas increases, assessment teams will be able to increase their use of primary data. There are four main ways to collect primary information in the field during phases I and II of an assessment. These include: Direct observation. Key informant interviews (KIs) provide key information on individual perspectives and experiences. Focus group discussions can be effective in identifying the cultural norms and understanding the issues of concern within groups or sub groups in an affected population17 , but demand a highly skilled facilitator. Community group discussions are a more general group discussion with disaster affected persons located at a specific site impacted by the crisis. A community group has less homogeneity than an FGD. Direct observation, also known as participant observation, is a process by which an assessment team collects data on naturally occurring behaviour within their usual context. This is achieved by observing conditions and specific features of an affected site from a range of viewpoints and locations to provide an overall view of the affected area and by noting these observations in a checklist. Observation can put into context information provided by affected persons through key informant interviews of community/focus group discussions18 . For example, latrines may still exist within the community, but the effects of the earthquake may have made them accessible only for able bodied persons. Direct observation by assessment teams also help them compare conditions between sites, 15 Ibid. 16 Reliable information being defined by the degree to which the instrument consistently measures what it purports to measure, and validity referring to the accuracy with which it measures this information. 17 ACAPS, 2011, Direct Observation and Key Informant Interview Techniques. http://www.acaps.org/img/documents/direct- observation-and-key-informant-interview-techniques-direct-observation-and-key-informant-interview-techniques.pdf 18 UNHCR, May 2006, Participatory Assessment in Operations; see also ACAPS, 2011 TB KIDO Techniques.
- 11. 11 something that analysis of information from the affected population at one single site is not able to do. Key informant interviews 19 , or semi structured interviews, provide vital information on individual perspectives and experiences through direct discussion. KI interviews involve a set of key questions which explore the impact of the disaster on the community as a whole. A KI is a primary data source with prior knowledge of the affected community who can provide information on behalf of the community on the impact of the disaster and on the priority community needs. Data from KIs can be combined and analysed to develop an understanding of how an emergency has affected different (sub) groups of a population, why it has so affected them, and what their ensuing priority needs and concerns are and are likely to be over time. As with KI interviews, FGDs are effective in identifying the cultural norms and understanding the issues of concern within groups or sub groups in an affected population which affect community response to a disaster and use of coping mechanisms by those community members. A FGD is a group discussion with persons of similar ages and backgrounds (and usually gender) whose experience of the disaster will likely be similar. Information gathered from an FGD enables analysis and understanding of a selected topic on the basis of common characteristics of the groups20 . For example, an FGD of rice farmers could be used to understand the impact of a flood on their crop; an FGD of mothers of young children could be used to understand post disaster child feeding concerns. Sometimes in the initial days after a crisis it may not be possible to form discrete FGDs that meet specific FGD criteria. In addition, it may not be possible to include someone with FGD facilitation skills in the assessment team. When constraints including time, human resources, and access to people affected by a disaster make it difficult to organise FGDs, a community group can be a useful way to capture some of the views and priorities of the affected population and to see which issues there is consensus on. A community group is comprised of members of the affected community, but is often larger in number and crosses gender, age, ethnicity and other standard FGD diversity parameters in composition. The nature and composition of the group and how it was formed should always be included in the reporting notes. In considering Table 2, it is seen that a qualitative question might ask: What risks do affected women face collecting firewood? or how has the disaster increased risks in water collection? A quantitative question, on the other hand, might ask: how many functional water points were there in the community before the disaster? and how many functional water points are in the community now? 19 ACAPS, 2011 TB KIDO Techniques. 20 UNHCR, May 2006, Participatory Assessment in Operations.
- 12. 12 Table 2: Qualitative and Quantitative Research Methods21 Qualitative Research Method Quantitative Research Method When to use it When in-depth understanding of a specific issue is required To understand behaviour, perception and priorities of affected community To explain information provided through quantitative data To emphasize a holistic approach (processes and outcomes) When the assessor only know roughly in advance what he/she is looking for Recommended during earlier phases of assessments To get a broad comprehensive understanding of the situation To get socio-demographic characteristics of the population To compare relations and correlations between different issues When accurate and precise data is required To produce evidence about the type and size of problems When the assessor knows clearly in advance what he/she is looking for Recommended during latter phases of assessment Objectives and main features To explore, understand phenomena Provides in depth understanding of specific issues Detailed and complete information, contextualization, interpretation and description Perspectives, opinions and explanations of affected populations toward events, beliefs or practices To seek precise measurement, quantify, confirm hypotheses Provides a general overview Provides demographic characteristics Objective and reliable Apt for generalization Objectively verifiable Prediction, causal explanation Data format Data can be observed but not measured Mainly textual (words, pictures, audio, video), but also categorical Data which can be counted or measured. Involves amount, measurement or anything of quantity Mainly numerical and categorical values Answers the questions Answers questions arising during the discussion How? Why? What do I need to look for in more detail? Questions are generally open ended Answers a controlled sequence of questions with predetermined possible answers What? How many? Questions are closed Perspective Looks at the whole context from within Searches for patterns Lends itself to community participation. Seeks depth of perspective though ongoing analysis (e.g. Waves of data) Looks at specific aspects from the outside Methods Individual interviews Key informant interviews Semi-structured interviews Focus group discussions Observation Quick counting estimates Sampling surveys Population movement tracking Registration Structured interviews Sampling Non random (purposive) Random Study design and instruments Flexible, the assessor is the primary instrument for data collection and analysis. Fixed, standards control the assessor’s bias. Questionnaire tool types Checklist with open questions and flexible sequence Predetermined questionnaire with sequence and structure Analysis Use inductive reasoning Involves a systematic and iterative process of searching, categorizing and integrating data Describes the meaning of research findings from the perspective of the research participants Involves developing generalizations from a limited number of specific observations or experiences Analysis is descriptive Uses deductive methods Descriptive statistics Inferential statistics 21 WFP, 2009, p5, see also PARK companion, JIPS/ACAPS 2012..
- 13. 13 Quantitative data is often described as objective while qualitative data is characterised as subjective. This distinction is both misleading and unhelpful since the subjectivity or objectivity of data depends not upon its intrinsic identity but upon the bias of the investigator, the methods used to collect it, and the ensuing analytical process. This can be true for both qualitative and quantitative data. The key to gathering a shared picture of the impact of a disaster affected population is through combining both types of data types; when used along with quantitative methods, qualitative research can help us to interpret and better understand the complex reality of a given situation and the implications of quantitative data22 . 5. Conclusion Quantitative and qualitative information falls upon a continuum and varies according to type of data, collection tool and method of data analysis. The main distinction between the two is that qualitative methods do not seek statistical significance and thus, cannot be extrapolated without relying on judgment. Any extrapolation of qualitative data to larger areas will inevitably be general and not statistically rigorous. Qualitative research methods make up for lack of statistical rigour by providing explanations and understanding23 . Four elements regarding qualitative and quantitative information are key to ensuring use of the right types and right sources of information during the initial days of a rapid needs assessment to improve decision making : 1. Qualitative and quantitative information are both important to understanding the impact of a disaster upon an affected population. However, given time and access constraints, primary qualitative information will most often combine with secondary quantitative information to address knowledge gaps. 2. Collection of both qualitative and quantitative data calls for specific and agreed upon technical methods and requires specific skills sets for accurate collation, collection and analysis. 3. In qualitative research, the data collection instrument is the researcher. Thus, the assessment team is the core element in both the observation of the problem and in the analysis. 4. Qualitative and quantitative data interpretation are inter-changeable; the crucial consideration is in how data is presented and what questions it is being used to answer. With all data collection methods, be they quantitative or qualitative, there is a responsibility by the assessment teams to be accountable to the persons from whom they solicit 22 Family Health International, 2005, Qualitative Research Methods: a data collector’s field guide. 23 WFP, 2009, p3.
- 14. 14 information. Awareness of, and adherence to, key humanitarian principles should be the first step of all needs assessment teams when carrying out field research. Affected persons should be invited to participate in the data collection process rather than feel cajoled or pressured. Information on how, when, where, and with whom the data will be used must also be shared with community research participants. Expectations regarding potential advantages, or disadvantages, of participating in the research process must be managed carefully to ensure that community members do not develop false expectations about humanitarian support or intervention. Failure to communicate clearly with affected populations participating in research cannot only compromise research results but, in some cases, can compromise the safety and security of the affected populations themselves. 6. Key Resources ACAPS, 2011, Direct Observation and Key Informant Interview Techniques. http://www.acaps.org/img/documents/direct-observation-and-key-informant-interview- techniques-direct-observation-and-key-informant-interview-techniques.pdf ACAPS Secondary Data Review Technical Brief, June 2011. http://www.acaps.org/img/documents/secondary-data-review---summary-secondary-data- review---summary.pdf Family Health InternationaI, Qualitative Research Methods: a data collector’s field guide, 2005. UNHCR, Participatory Assessment in Operations, May 2006. World Food Programme, Emergency Food Security Assessments, Technical Guidance Sheet #8, Introduction to qualitative data and methods for collection and analysis in food security assessments, February 2009.