Generating researchable questions

Editor's Notes

• #5: The method of question generation that we are going to focus on in this lecture comes from two qualitative processes – the first being observation where the field is surveyed and then refined observations occur to generate rich and thick description of a phenomena. The second process drawn upon being qualitative analysis where you first survey a corpus of data and then refine your coding categories again to generate rich descriptions. So from observation methods and qualitative analysis coding we take three key steps, or phases of question generation, which occur once you have found a topic of interest Scoping – looking over your topic of interest broadly and noting all questions AND ‘problems’ – i.e. problem identification Refining your topic – group themes and questions Generating questions – look over grouped questions and develop a meta questions Most of the time the cycle of question generation finishes at this third step – the generation of a master question. Although you can continue to scope and refine until you get an appropriate question. You will often find that in PhD research you go around the cycle a few times as you become familiar with the literature and eliminate questions that are answered in the literature. By generating questions in this manner, you also prepare yourself for an qualitative analysis – it is good practice for the coding/subcoding of data Put class into interrest groups
• #6: Okay, let’s work through the steps. When we embark on a masters or phd piece of research, we do not come to the research knowing nothing about a topic. Indeed, you will find that many potential supervisors like you to already have some knowledge of a topic. It is not required that you are an expert of a topic but you should be prepared to become an expert (this is part of the research process in thesis construction). So, it is not expected that you have read all the literature in the field and that you already know the answer to every research question posed – but it is expected that you have some ideas. Sometimes the best questions are actually generated by personal experience and personal observations. This first stage of question generation involves a great deal of reflective brainstorming. It’s where you note your topic of interest down and then brainstorm as many questions and problems as possible that interest you or you can think of. Try to frame your problems as questions through – these give you points in which you can answer and acknowledge your own development of knowledge. In doing this you are using your personal experiences, ambitions, observations and the any research or literature you have read. You can do this by yourself but you can also do this with colleagues, peers, and supervisors. Basically, in this stage, you are scoping the field – you are getting a feel for where you are entering and the possibilities of research. Brainstorm questions – one general research question and one evaluation question
• #7: In the second stage – the refining of questions – you actually group questions along the lines of similarities and differences. You first group questions within central themes. In this refining stage you are grouping/subgrouping and arranging questions to find a specific focus or to cover the types of research you could undertake. As you are doing this you are eliminating questions that don’t interest you or seem to be exhausted by research. If you are doing this in a PhD context you need to pay particular attention as to whether or not your question has been answered. A phd must have an element of originality – it is often your question that leads to this. The refining stage is the most important and you should find it quite fluid as it moves between refining and questioning. Show theme oht 1 and 2 and explain that we are going to continue to refine as we move to the questioning step However, it is always important to remember that moments of ‘brilliance’ often occur during brainstorming – you might find as you look through questions, group them, and identify the ones answered by literature, that there are questions that begin to appear to you as being interesting and possible – you might choose to take one of these questions and just work on refining it through developing sub-questions and checking with the literature. In my PhD I thought I had a very good driving question – but it wasn’t until I looked broadly at the literature that I realised that my question was actually answered and needed development.
• #8: In this final stage you are attempting to develop one or two overarching question. Often these questions come out of the aim for your research. This involves you looking at your subquestions and then identifying the aim of the research or the underlining hypothesis. Once you have an aim it is relatively easy to look at both your subquestions and your aim and find the questions that relate specifically with what you want to do. These questions may come from the subquestions or they may come from the aim but reflect the subquestions. However, you might find that your aim comes the opposite way – from the development of key questions that lead to an aim. No matter what you do, it is important to realise that this stage of research is very important. You need to recognise that a single piece of masters or phd research cannot answer every question, that theses have word limits which you need to go to, and you can’t answer every question about a topic in 50,000 words or less so you do need to focus. A good piece of research has a clear focus which each stage of the research is directly related to. Your main research questions do not eliminate subquestions; instead they provide a focus to subquestions. In a sense they provide the skeleton of the research on to which questions, paradigms, and methodologies are attached.
• #9: Once you have your key questions you then situate your questions within a paradigm of research and develop an appropriate methodology taking into account any constraints. Hence, it is important to recognise that it is not the paradigm or the methodology that drives the research but it is the question. Returning to my Masters research this is probably where I went wrong – I first selected a paradigm of research (post-strucuturalism) and then designed a project that fitted within a paradigm. This worked but it is questionable as to whether the research is effective or whether the research would stand up under tougher scrutiny. Indeed, later, when I had my first meeting with my PhD supervisors they asked me what I was interested in – I talked about representations of youth crime and then said that if I could I would like to do an archaeological discourse analysis – I was quickly corrected and told that it was really about selecting the appropriate methodology for my question not dibble-dabbling in research methodologies. This does not mean that you cannot prefer to use a particular methodology or paradigm – but it must match your question, it must fit within the constraints of your research, it must lead to a cohesive study which, to use a qualitative term, leads to an authenticity of the story you want to tell. In short, there must be a fusion of questions, theory, and methodology which is not hindered by the constraints surrounding your research. So what are these paradigms and what sorts of questions do they lead to?
• #10: According to Denzin and Lincoln there are six key paradigms of research (five of which come from a qualitative focus and one from a quantitative focus). Each of these paradigms involves applying a particular methodology and this methodology is suited to a particular type of question. For example, a positivist paradigm is concerned with a quantifiable number – the focus of this paradigm is statistics and the types of questions asked by this paradigm involve a degree of counting and quantifying. A constructivist paradigm is more concerned with a phenomenological story – how are truths constructed by particular groups of people. In this paradigm a ‘how many …’ question makes no sense and cannot be answered authentically Feminist, ethnic and marxist paradigms focus on the exploration of differences and inequalities but not at a quantifiable level but at a phenomenological level – at the level of experience and perception. It might draw upon statistical data but there is an element of critique and questioning. Finally cultural studies leads itself to theoretical questions of critique even further – rather than just exploring differences, cultural studies seeks to question difference and the knowledge on which difference is constructed. It is this questioning of knowledge and truth that makes cultural studies perspectives different to the other four qualitative approach. Often researchers applying a cultural studies focus will draw upon post knowledges – like postmodernism, poststructuralism, postcolonialism, postfordism and so on. Each of these paradigms implies a different sort of question – it is important that you situate your questions into the correct paradigm and use the appropriate methodology. For example, if you want to look at the effectiveness of NCEA you need to have quantifiable elements (that is unless you are questioning the concept of effectiveness itself). In a look at effectiveness you might look at it across gender, ethnicity, and socioeconomic variables – but you are doing just that – looking at it across quantifiable variables. You are not focusing on the phenomenology of NCEA (on people’s experiences and perceptions of NCEA) – that’s a totally different set of questions. It would only be within a mixed paradigm that you would include quantitative and qualitative elements.
• #11: In a positivist paradigm we are concerned with providing quantifiable evidence the type of aim would focus on something that can be counted. An aim might also be a direct hypothesis which we aim to test and quantify using experimental design. In this case we want to track differences – our questions need to reflect this aim – so we ask directly about differences. We must also consider the fact that changes have occurred to NZ qualifications over the last few years (which inhibit the reliability of the study) and that we really only have two years (a masters thesis) to study in. Taking all these questions into account we might do a textual analysis of outcome data from the MOE and, in order to triangulate findings we might do a survey of secondary school assessment data. We probably will not interview students as the data obtained will probably not benefit in the answering of key questions. We also need to be allocating our resources efficiently and effectively – given that we need to ensure that our findings are both valid and reliable we need to attempt to survey or gather as much data as possible – we could really only do this in a masters by working with data that is already accessible or focusing on a large survey sample that is representative of the schools in which we would like to generalise our findings to.
• #12: In a constructivist paradigm we are concerned with exploration and theory construction so our aims are not about producing quantifiable results. We might want to explore a particular aspect of a topic (such as the ways in which students manage their own learning and assessment needs). Our questions are directly related to this aim and suggest that we will actually involve ourselves in talking with students (even if this is only as a pilot to generate survey codes). There may be some quantifiable aspects of our research but these are not framed within an experimental design. Any quantifiable elements in our research design would be there to ensure that we could tell a more representative story of student self-management in schools. Again we may have constraints on our research such as finances. Although the most authentic piece of research would involve talking to students across a good cross section of schools, we might only have enough money to focus on one or two schools. This would suggest that a case study would be better suited to our research than a survey as we can target our funding and resources to ensure that we can gather a rich corpus of data. A case study can be authentic in itself as it recognises the limitations of accessibility whilst also making these limitations a strength – ie although the findings are limited to one case the case in itself might benefit from the findings and other groups can use the findings to inform their practices. However, a case study would also suggest that data is going to be gathered in more than one way – so you would need to combine techniques of observation, interviewing, focus groups, textual analysis, survey etc and/or you would need to combine sources of data (such as teachers, parents, and students).
• #13: In a constructivist paradigm we might also construct a theory. This might be around the same sorts of questions but it involves aspects of grounded theory particularly involving open-ended data gathering and analysis that allows for a multitude of ideas to come through. For example in a direct exploration you might ask students to describe how they use pre-determined strategies (such as timetabling). In the development of a theory you don’t give students clear pre-defined categories but, instead, ask students in an open ended fashion – what they do to manage their own learning needs. Perhaps one of the biggest constraints to this methodology is time – data gathering takes a lot of time (as you need a rich and thorough corpus of data) and analysis takes longer than normal. So you might use a case study again but limit the instruments of data gathering but allowing enough for triangulation (perhaps interviewing a variety of people within a particular school). Alongside time, ‘experience’ also may place limitations on your research. For example, you will need to involve yourself in unstructured interviewing and observations – unstructured techniques of data-gathering can be very difficult for researchers just starting out on a research career as you need to have communication skills that enable participants to generate rich descriptions and stories without you leading the story in a particular direction.
• #14: In a feminist paradigm we may have similar constraints and we are driven by a constructivist agenda – one of exploration. But, particularly in a Masters thesis, our methodology needs to suit the genre of feminist writing and research. It is within a Phd that you might introduce different strategies and approaches that are well argued and suit your questions and aims of research. In a piece of feminist research faced with constraints of the ‘system’, finances and time, is we might find that narrative story telling (based on unstructured interviewing) and focus groups might suit the type of aim and types of questions we would generate. We also need to recognise and debate power issues in the research (whose doing the research, what organisation etc). As we step within these next four paradigms (feminist, ethnic, marxist, and cultural studies) we also need to ensure that we are well versed in the theory and literature of the approach and paradigm we are using. For example, in my own PhD research which is based on a discourse analysis of meaning, I needed to demonstrate that I understood key texts such as Foucault’s Discipline and Punish and Archaeology of Knowledge. In a feminist piece of research that we need to make sure that we know the ideas of authors such as Michelle Fine and Patti Lather. In a way there are key texts to the qualitative approaches and theories we choose to use and we need to show an understanding of these texts and we need to let these texts inform our research and analysis
• #15: We can find the same story in Ethnic research – this time we might draw heavily on unstructured interviewing and kaupapa research. We also need to recognise and debate power issues in the research (whose doing the research, what organisation etc). In New Zealand, we cannot apply a ethnic paradigm to our research without mention of Russell Bishop, Graham Smith, and Linda Tuhiwai Smith.
• #16: In a marxist paradigm we are mainly concerned with inequalities (which are traditionally drawn on economic lines but can be along a variety of structural dimensions such as gender, ethnic and age. In this example, we want to explore the ways that NCEA might enforce structural inequalities. Given the constraints of our research, this might involve some textual analysis of quantitative data – but not at an objective level but at a level of power analysis (where concepts of hegemony come into play). This means that we need to be looking at the data critically. Now in this hypothetical scenario, we are not only constrained financially but also in our location – where we live in a rural setting. This means that we do not have access to many students and schools. The methodology selected reflects this – first we are going to do a structural analysis of textual data (again probably MOE and ERO data) but we are then going to supplement this data with a case study. Like feminist research and ethnic research there are many key authors, a couple that we might use to inform our analysis here are Karl Marx, Antonio Gramsci, and Louis Athusser – depending on the approach we are taking.
• #17: Finally, in a cultural studies paradigm we are focused on critique. This should take a structural focus (based on power) but it might also directly critique the ways in which constructs are formulated and how different social groups are positioned within these constructs. In a sense, cultural studies is a post to our prior qualitative paradigms of constructuralism, feminism, ethnic studies, and marxism, where the word post indicates a further critique and questioning of foundational knowledges. Although there are key authors drawn upon in these approaches (such as Foucault, Derrida, Bhabha, Butler, and Said) there are many theorists and theories that we may choose to use to inform our research. If you take a cultural studies approach be prepared to spend a lot of time thinking – indeed often the thinking takes a large proportion of the study time. In this scenario, our aim directs us to the types of knowledge being assessed and valued by NCEA. Our questions allow us to explore this knowledge and question this knowledge. This requires us to look at NCEA as a qualification which leads us to looking at NCEA in a macro context. Furthermore because we are critiquing NCEA as a qualification it would be beneficial to look at NCEA on the level of policy and practice. We can find, and answer these questions, through a textual analysis. Although we could also interview people, we also need to consider the time factors around our research and the level of depth a cultural studies analysis takes – for this reason we might limit our data to two key sources.
• #18: Finally, perhaps a paradigm that is valued considered highly at this point and time (by researchers and policy makers) is the mixed paradigm. In a mixed paradigm we bring together two or more traditional paradigms. Often this is a mixture of a positivist (or quantitative paradigm) with one of the other qualitative paradigms (often the constructivist). This is what we often see in Ministry funded research which involves a survey and quantitative analysis submitted with qualitative data. In this example, we are involving a quantitative and qualitative component. We are first measuring success and identifying factors through a textual analysis and survey, and then looking at these factors closely through a case study. In groups – students look at aim and question and identify which paradigm there question fits in. Then talk about possible methodologies and reconstruct aim/question to fit within other paradigms. Emphasise that whilst it is possible to change a question to suit a paradigm that it is important, particularly in PhD research, to be passionate about what you are studying as (without a passion) it is hard to keep motivated in down times and often this motivation comes through a passion for a question (rather than approach). Remember that 29% of students who start a PhD finish it within five years and 40% of people just fall off the radar – so it’s important to have a passion.
• #20: I just want to finish this lecture by looking at the key types of methodology found within educational research and where these methodologies fit within the paradigms. Remember this is not an absolute but if you are just starting out in research you really need to know what you are using – if you are uncertain, it is best to select the types of methodologies that are suited to your paradigm (rather than the types of methodologies you would like to use). Again it is about letting the question drive your methodology. I would also say don’t be put off by methodologies because you think you can’t do the analysis – staff are here to help you and there are software packages. For example, you don’t have to do chi squares and regression analysis by yourself – you can enter the numbers into a program like SPSS and let it do it for you. It’s the same with qualitative research, we have programs available like N7 which enable you to do qualitative analysis and theory building. Looking at methods of data gathering Textual analysis can be used across all the paradigms but is suited to positivist, feminist, marxist, and cultural studies. The one thing you have to remember with textual analysis is just that – it is an analysis – it is not a review of literature. We will go on to this in next week’s lecture. Survey research can also be used across the paradigms but it is best suited to positivist and constructivist paradigms. You could also use it (in a critical way) in cultural studies Observation which is a traditional form of data gathering is suited to positivist and constructivist paradigms Finally interviewing is suited to all paradigms – but the type of interview you do (from structured to unstructured) varies to suit the paradigm