PILOT STUDY.pdf
1 like3,667 views
This document discusses pilot studies and pretesting of surveys. It defines a pilot study as a small-scale trial of the research process to assess feasibility, while pretesting specifically refers to testing survey questions to ensure they are clear and understood by respondents. The key objectives of pilot studies and pretesting are to validate research tools and methods, identify any issues, and improve the quality and rigor of the full research project. Sample sizes of 10-30% of the planned study are typically used for pilot testing.
PILOT STUDY.pdf
- 1. PILOT STUDY AND PRETESTING BY DR EBOT WALTER OJONG SENIOR LECTURER/CHEMICAL PATHOLOGIST
- 2. INTRODUCTION ‘No amount of intellectual exercise can substitute for testing an instrument designed to communicate with ordinary people’ (Backstrom and Hursch, 1963) Surveys must be thoroughly pretested before they are launched (Hulland et al., 2018)
- 3. INTRODUCTION To ensure that the meaning of the question items are clear to the respondents, and that the respondents will clearly understand the questions the way they are designed and intended, it is vey important that a survey questionnaire must be pretested (Sekaran, 2003)
- 4. MEANING OF PILOT STUDY • A mini version or small scale version of the research study aimed at trying or testing the research process and protocols in order to help in assessing the feasibility of the main/proposed research project • Could be a mini version of the entire process of the main research project or of certain aspects e.g. data collection, analysis tools etc. • Examples piloting of observations, interviews, questionnaires or other tools • In experimental studies it could be piloting of the experiments/tests
- 5. OBJECTIVES OF PILOT STUDIES • The general goal of a pilot study is to save some time, effort and money, which can be lost if a major research study fails because of unforeseen attributes • Test adequacy of research instruments • Assess feasibility of a full-scale study • Assess whether the research protocol is realistic and workable • Reveal logistic issues • Collect preliminary data
- 6. OBJECTIVES OF PILOT STUDIES • Ensure whether the sampling frame and technique are effective • Determine sample size (The pilot study sample is inappropriate to be used as the main study sample) • Convince funding bodies that the major study is feasible and worth funding
- 7. OBJECTIVES OF PILOT STUDIES • ‘To see if the beast will fly’ (Teijingen and Hundley, 2002) • ‘Reassessment without tears’ (Blaxter, Hughes and Tight, 1996:121) •‘Do not take the risk. Pilot test first’ (Van Teijilingen and Hundley, 2001:2)
- 8. IMPORTANCE OF PILOT STUDIES • Helps in establishing the feasibility of the research project • Feasibility of data collection and analysis tools/processes • Authentication and improvement of research processes and tools. Validity and reliability of research tools and processes which add rigour to the research • Precautions against possible large scale project failures
- 9. IMPORTANCE OF PILOT STUDIES • Data/results generated from pilot study could be made part of the main research. • Establishing the need/importance of full/large scale studies. • Useful for junior researchers and student researchers. • Helps in obtaining funding for large scale studies.
- 10. HOW ARE PILOT STUDIES CARRIED OUT? QUANTITATIVE STUDIES • Surveys should be carried out on small samples during the pilot process which are subsets of the larger sample/population for the main study. • Generally these mini versions of surveys will establish validity, reliability and feasibility of data collection, analysis and access processes • Highly important/essential
- 11. HOW ARE PILOT STUDIES CARRIED OUT? QUALITATIVE STUDIES • Pilot studies are generally useful for for improvement of data collection tools and processes (interviews, observation protocols) in terms of language, content and data obtained. • This helps in refining and authenticating the data obtained in the main study • Important/useful
- 12. HOW ARE PILOT STUDIES CARRIED OUT? • MIXED METHODS STUDIES • As addressed above • Important/useful
- 13. PILOT STUDY OR PRETESTING •Although pre-testing is often understood as a pilot study, they both serve distinctive purposes. •They are similar but different from each other
- 14. PILOT STUDY OR PRETESTING •A pilot study is a mini-version of a full-scale study or a trial run done in preparation of the complete study to be undertaken in order to ensure that a full-fledged study will be carried out successfully; a feasibility study. •It is a try-out not only of research techniques and methods but also of questionnaires and interviews
- 15. SAMPLE SIZE FOR PILOT STUDY •A sample between 25 and 100 individuals (Cooperand Schindler, 2011) • A range from 10 to 30 individuals are enough for a pilot test (Hill, 1998) •A sample size should be 10 per cent of the sample projected for the main study (Connelly, 2008)
- 16. WHAT IS PRE-TESTING PROCESS? •Objectives of pre-testing To ensure whether a) the wording of the questions is correct b) the sequence of questions is correct c) the respondents have clearly understood all the questions d) Additional questions are needed or some questions should be eliminated e) the instructions are clear and adequate
- 17. WHAT IS PRE-TESTING PROCESS? •Fundamental issues of pretesting a questionnaire a) The specific items to be pretested b) The method to be used to conduct the pretest c) The person/s who should conduct the pretest d) The participants who should be the subjects of the pretest e) The sample size needed for the pretest
- 18. THE SPECIFIC ITEM TO BE TESTED Items about the questionnaire itself •Format of the questions •Layout •Length •Number of lines in case respondents need to write a response, and the sequencing of questions Items about specific questions •Whether the respondent interpreted the question differently or whether the respondents did not understand the statement very well
- 19. THE METHOD TO BE USED TO CONDUCT THE PRETEST Cognitive (personal) interviews Debriefing method Protocol method Telephone interview Mail self-reports
- 20. THE METHOD TO BE USED TO CONDUCT THE PRETEST Cognitive (personal) interviews Debriefing method The respondent is asked to answer the questionnaire completely while the interviewer observes any indication gestures of confusion from the respondent After completing the questionnaire, the interviewer will ask the respondent for any possible problems which he/she may have encountered while filling out the questionnaire
- 21. THE METHOD TO BE USED TO CONDUCT THE PRETEST Cognitive (personal) interviews Protocol method The respondent is asked to express his or her thoughts once it occurs while answering the questionnaire A recorder will be helpful to make carefulnotes about the verbalizations of the respondents thoughts
- 22. THE PERSON TO CONDUCT THE PRETEST Only the best interviewers should conduct pretesting as they have the keen ability to observe any uneasiness, confusion and resistance among the respondents The selection of interviewers should also involve different levels of competence
- 23. THE PARTICIPANTS DURING THE PRETEST Individuals who are similar to the target population of the study Individuals who are typical or representatives of the actual population to be studied Individuals at a site different from the actual study site
- 24. THE SAMPLE SIZE FOR THE PRETEST If the questionnaire is long and complex, it would need a bigger sample for pretesting If the target population of the study is unsophisticated, a bigger sample for pretesting size is also necessary
- 25. SUMMARY PILOT STUDY: To try out the research questions, methodology, research instrumentor anu aspect of the research PRE-TESTING: To ensure that the instrument is clearly understood by the respondents