Connecting Research Questions and Research Method

Ferran Giones
2018
SDU - MADS CLAUSEN INSTITUTE
Making research that
works
Searching for the problem-method fit
PhD Workshop – June 2018

MADS CLAUSEN INSTITUTE
Agenda
• Challenge
• We want to make “good” and relevant research, but with often
limited resources (method & data).
• What are the key components to get started?
• How do we think about our research questions?
• Making explicit the problem-method fit.
11 May 2017
2

Going back to the basics…

We want to make “good”
research…
• How big a role do unconventional
combinations of existing knowledge
play in the impact of a scientific
paper?
• Unexpected influence of
”conventionality” – shows importance
of connecting with what is out there:
.. Newton presented his laws of
gravitation using accepted geometry
rather than his newly developed calculus,
despite the latter’s importance in
developing his insights…
11 May 2017
4
Uzzi, B. et al. (2013) ‘Atypical combinations and scientific impact’, Science, 342(6157), pp. 468–472. doi: 10.1126/science.1240474.
https://twitter.com/Keld_Laursen/status/1006152889339916289

Novelty & Convention for Impact
11 May 2017
5
(hit means top 5% in their discipline)

Theory Empiricism
Problem/
Phenomenon
„Interesting“
Knowledge
We need at least three elements:
Real-world questions
Relevance of the question
How to explain, predict and
cause the problem still
remains open
6Materials from Tim Schweisfurth (SDU – TUM)

Theory Empiricism
Problem/
Phenomenon
Knowledge
These elements build knowledge
Generally falsifiable system of
assertions
System of Hypotheses
Quality criteria for theories

Theory Empiricism
Problem/
Phenomenon
Knowledge
With a strong or not so strong
empirical component...
Operationalization (define
measurement of concepts/
make concepts measurable)
Rigorous Reality-Test
Testing with random sample

Methodology and research
design
• Epistemological position may remain implicit in research (Creswell 2009;
Slife and Williams 1995)
• Applied methodology becomes apparent to the reader
• Methodology is “the strategy, plan of action, process, or design lying
behind the choice and use of particular methods” (Crotty 1998, p. 3)
• Needs to be justified by highlighting the fit between research questions, research
design, and contribution (Edmondson and McManus 2007)
• Methodological decisions are located on a higher level than the applied methods
(Bryman 1984)
Materials from Tim Schweisfurth (SDU – TUM)

Epistemology
• Objectivism
 Meaning and reality exist independently from any consciousness
 Humans discover meaning of objects which has been there before
 There is one objective truth independent of individual biases
 Objective truth can be discovered
 E.g. trees exist in the forest – if recognized by humans or not
• Constructionism
 There is no objective truth to be discovered
 Meaning comes into existence with our cognition and is constructed in our mind
 Different people construct reality in different ways
 Subject and object interact in the process of cognition
• Subjectivism
 Meaning is imposed on the object by the subject
 No interaction between subject and object in cognition
 Object does not contribute to the cognition process
 Meaning ascribed to object comes from somewhere else (dreams, God, religious beliefs,…)
10
What can we know about what exists?

The methodological field
11 Materials from Tim Schweisfurth (SDU – TUM)

Scientific paradigms
12
Positivist Postmodernist/Interpretive
Discovery of universal laws governing social
world.
Discovery of how people make sense of their
social worlds.
A fixed social reality exists that may be
measured and described.
Many social realities exist due to varying
human experience.
Human behaviour is both rational and
predictable.
Human behaviour is context bound and
variable.
Positivist science is capable of uncovering
‘truth’.
Common sense provides insight into social
realities.
Discovery of social fact is achieved through
reason.
Understanding of social reality is achieved
through rich contextual description.
Objective, value-free study is crucial in social
research.
Recognition of subjectivity in social research
is important.

Quick check:
Can you share your current research question?
(spend 5 minutes if you need).
• It can be from your proposal, last paper, working idea…
Can we infer which epistemology is behind the research
question?
• What type of research design could fit? Why?
11 May 2017

Fitting the research
problem/question with a method

It’s all about connecting…
Creswel, J. W. (2008) ‘The Selection of a Research Approach’, Research design: qualitative, quantitative,
and mixed methods approaches, pp. 3–22.

Methodology and research
design
• State of existing knowledge and prior theory are the defining factors
(Edmondson and McManus 2007)
• Research with little prior theory available goes along well with qualitative
methods
• Qualitative methods allow for a high amount of richness in data, which is
needed to explore new phenomena and to build theory about emerging
constructs and their relationships (Bryman 1984; Gephart Jr 2004)
• Where theory is mature, quantitative methods are better suited to answer
research questions
• Hypotheses can be deducted from existing knowledge and then be tested
empirically
• Examples for methodologies are experimental research, survey research,
ethnography, grounded theory, heuristic inquiry, mixed-method research

Edmondson, A. and McManus, S. (2007) ‘Methodological fit in field reserach’, Academy of Management Review, 32(4), pp. 1155–1179. doi: 10.5465/AMR.2007.26586086.

Research designs
“Research designs are plans and procedures
for research that span the decisions from broad
assumptions to detailed methods of data
collection and analysis. […] The selection of a
research design is also based on the nature of
the research problem or issue being
addressed, the researchers’ personal
experiences, and the audiences for the
study”
Creswell, 2009
18

Methodological fit
19

20

Quick check:
Using your research question and theory (or potential theory/ies)
• Can you look back at the graph, try to find where you are.
• Is there something missing?
• What can be criticized of this assessment?
11 May 2017
21

Nascent theory studies
22

Intermediate theory studies
23

Mature theory studies
24

When things do not go well…
25

Further steps – towards the fit
Unit of analysis – does it fit?
• Individuals
 E.g. How does personality facilitate innovative behavior?
• Groups
 E.g. How does team composition facilitate innovative team behavior?
• Organizations
 E.g. How does absorptive capacity facilitate innovative performance?
• Interactions
 E.g. How do status differentials impact the response time to inbound requests?
• (Social) artifacts
- E.g. Does the tenor of announcements of product retraction affect media coverage about firms?
• Level of analysis informs theory choice (vice versa)

Temporal dimensions
• Time dimension is important for the establishment of causation
• Cross-sectional studies
 Observation of sample at one point in time
 Snapshot of a timeframe
 Causation cannot be established
 High potential of common method bias
• Longitudinal studies
 Capturing the evolution of the phenomenon over time
 Possibility to establish causation
 Collecting data at different points in time
 Difficult for survey research
 E.g. trend analyses, cohort studies, panel studies
29
Further steps – towards the fit

Example:
Reviewer‘s considerations of causation
Directionality. You cite “Employees who identify with users are more likely to view their work as a calling
because it overlaps to a larger degree with their outside lives (Wrzesniewski, McCauley, Rozin, &
Schwartz, 1997).” The distinction mentioned earlier set aside, I have no memory of Wrzesniewski showing
any causality in their paper. The directionality might be the opposite (i.e. employees who view their
work as a calling might identify with, not as, users).
Directionality. You claimed “direct effect of A on B” in your abstract, which is not supported in your
data. I see that in the findings section, you mention relations and direct relationships.

Example:
Reviewer‘s considerations of research design
The paper is somewhat "stuck in the middle" between an inductive approach and a deductive
approach. There are two issues: (a) developing a set of propositions from one case only: there is a general
agreement in the literature on case study research (see e.g. the Special Issue on case study research in
AMJ, 2007) that to develop propositions on one case only requires very sound theoretical pre-work. (b)
validating those propositions with three more cases: I think here you have a misunderstanding of qualitative
research if you try to validate/test propositions with three more cases. You are mixing up two fundamentally
different logics of research namely the inductive and the deductive approach. I rather suggest that you
follow a purely inductive approach. This would mean that you align the structure of your paper as
follows: specification of the research question to better position it against prior work, to develop a
preliminary theoretical framework (identify theories and literature streams that relate to your study), design
a qualitative study with multiple cases, report the findings from case comparison, and then –as a
synopsis of your preliminary framework and the findings- refine theory or develop new theory. I suggest
to drop the emphasis on developing and validating (!) (so many) propositions and to focus more on
the identification & elaboration of interesting and novel aspects/phenomenon instead.

All pieces need to fit…

Back to the beginning:
Building ”good” research
questions

Where do the ideas come from?
Everyday life
• What are the most effective ways to teach research methods?
• What personal characteristics make a favorable impression in
a job interview?
• How could commercial traffic flows in Bogota be routed more
efficiently?
40

Where do the ideas come from?
Practical issues or needs
• Why are most heart attacks on a Monday morning?
• How can logistic service providers ensure sustainable logistics?
• Why do so few companies use activity based costing?
• Why do innovation projects fail so often?
• Why are some chemical companies more successful in reducing the
procurement costs than others?
• Why are there so many conflicts between engineers, scientists and marketing
people when building
Past research
• Knowledge develops in small steps. Rarely does one study answer all the
questions to the research topic.
Theory
41

Sample Research Questions
Unclear: Why are social networking sites harmful?
Clear: How are online users experiencing or addressing privacy issues on such social
networking sites as MySpace and Facebook?
The unclear version of this question doesn’t specify which social networking sites or suggest
what kind of harm the sites are causing. It also assumes that this “harm” is proven and/or
accepted. The clearer version specifies sites (MySpace and Facebook), the type of harm
(privacy issues), and who the issue is harming (users). A strong research question should
never leave room for ambiguity or interpretation.
42
GEORGE MASON UNIVERSITY WRITING
CENTERMaterials from Tim Schweisfurth (SDU – TUM)

Unfocused: What is the effect on the environment from global warming?
Focused: How is glacial melting affecting penguins in Antarctica?
The unfocused research question is so broad that it couldn’t be adequately answered in a
book-length piece. The focused version narrows down to a specific cause (glacial melting), a
specific place (Antarctica), and a specific group that is affected (penguins). When in doubt,
make a research question as narrow and focused as possible.
43

Too simple: How are doctors addressing diabetes in the U.S.?
Appropriately Complex: What are common traits of those suffering from diabetes in
America, and how can these commonalities be used to aid the medical community in
prevention of the disease?
The simple version of this question can be looked up online and answered in a few factual
sentences; it leaves no room for analysis. The more complex version is written in two parts; it
is thought provoking and requires both significant investigation and evaluation from the writer.
As a general rule of thumb, if a quick Google search can answer a research question, it’s
likely not very effective.
44

Research questions – too broad
or too narrow?
Question A: What marketing strategies does the Coca-Cola company currently apply?
Question B: What is the Coca-Cola company's future marketing plan?
Question C: What marketing strategies has the Coca-Cola company used in the past?
Question A: What impact has deregulation had on the airline industry?
Question B: What percentage of commercial airline crashes were traced to negligent
maintenance during the 10 years immediately preceding and following deregulation?
Question C: What impact has deregulation had on commercial airline safety?
Question A: What are the 14 different disease-causing genes that were discovered in 1994?
Question B: What is the importance of genetic research in our lives?
Question C: How might the discovery of a genetic basis for obesity change the way in which
we treat obese persons, both medically and economically?
45

Research questions – too broad
or too narrow?
Question A: Do students trained in management or engineering start their first job with more
developed skills?
Question B: Do students trained in management or engineering start their first job with more
highly developed analytical skills?
Question C: Do students trained in management or engineering start their first job with more
knowledge about 2x2 matrices?
Question A: How can adult children of workaholics most positively interact with their workaholic
parents?
Question B: How do adult children of workaholics interact with their workaholic parents?
Question C: What is the major emotional reaction of adult children of workaholics to their
workaholic parents?
46

Research Questions
What is the most important criterion for good research questions?
How has this answer changed in the last years for you?
47

Criteria for good research
questions
A good research question:
Is feasible, doable
Is clear
Is significant
Defines the investigation
Sets boundaries
Provides direction
Helps produce good research
Is ethical
While it is important to know what you are looking for, it is a mistake to let this initial focus blind you from
other unanticipated questions which are perhaps more interesting, important or manageable than the initial
one.
Questions can be refined and new issues may emerge while reviewing literature or collecting and analyzing
data.
48

Strategies to build
research questions

Generating research questions
Most common strategies (at least from organization studies)
11 May 2017
51
Sandberg, J. and Alvesson, M. (2011) ‘Ways of constructing research questions: gap-
spotting or problematization?’, Organization, 18(1), pp. 23–44. doi:
10.1177/1350508410372151.

Confusion spotting
The main focus in this way of constructing research questions is
to spot some kind of confusion in existing literature. Previous
research on the topic exists, but available evidence is
contradictory.
The research question aims to sort out the identified confusion in
the literature and to explain it.
The main version of this mode of constructing research questions
was to search for competing explanations in existing literature
11 May 2017
52
10.1177/1350508410372151.

Neglect spotting
Spotting something neglected in existing literature is the most
common mode of constructing research questions in our sample.
It tries to identify a topic or an area where no (good) research has
been carried out.
There is virgin territory—a white spot on the knowledge map—
that produces an imperative for the alert scholar to develop
knowledge about the neglected area(s).
It is possible to distinguish three specific versions of neglect
spotting, namely, spotting an overlooked area, an under-
researched area and a lack of empirical support.
11 May 2017
53
10.1177/1350508410372151.

Application spotting
Spotting a new application in existing literature is a third basic
mode of constructing research questions.
It searches mainly for a shortage of a particular theory or
perspective in a specific area of research.
The research task is to provide an alternative perspective to
further our understanding of the particular subject matter in
question.
11 May 2017
54
10.1177/1350508410372151.

Problems with gap-spotting
• Is easy (or more than other options).
• Uncontroversial and safe.
• A powerful tradition indicates knowledge accumulation.
• Fits with institutions and journal formats.
• Alternatives are difficult.
11 May 2017
55
10.1177/1350508410372151.

Alternative approaches
Critical confrontation
Here, the researcher criticizes a theory or a field based on the
identification of some shortcomings. The critical confrontation may
take the form of application spotting, such as when a particular
area lacks a critical perspective.
New idea
This simply means that the author emphasizes a new idea. The
author claims innovation and does not follow any route marked by
the map of a literature review of what has not already been
studied.
This means sidestepping both building on and challenging
existing studies. It calls for original thinking and a high level of
self-confidence. However, this approach is not common in
contemporary management studies.
11 May 2017
56
10.1177/1350508410372151.

… and problematizing
11 May 2017
57
Alvesson, M. and Sandberg, J. (2011) ‘Generating research questions through
problematization’, Academy of Management Review, 36(2), pp. 247–271.

… and problematizing
11 May 2017
58
Alvesson, M. and Sandberg, J. (2011) ‘Generating research questions through
problematization’, Academy of Management Review, 36(2), pp. 247–271.

Wrap-up

Novelty & Convention for Impact
11 May 2017
60
(hit means top 5% in their discipline)

Additional material
For qualitative work:
• Gehman, J. et al. (2017) ‘Finding Theory–Method Fit: A Comparison of Three Qualitative Approaches to
Theory Building’, Journal of Management Inquiry, 143(2), p. 105649261770602. doi:
10.1177/1056492617706029.
11 May 2017
61

New methods for new research
problems
The case of Heuristics Theorizing
11 May 2017
62

- INNOVATION WITHOUT BORDERS
Heuristics Theorizing
14-01-2019
63
Source: Gregory, R.W. & Muntermann, J., 2014. Research Note —Heuristic Theorizing: Proactively
Generating Design Theories. Information Systems Research, 25(3), pp.639–653.

Connecting Research Questions and Research Method

More Related Content

What's hot (20)

Similar to Connecting Research Questions and Research Method (20)

More from University of Stuttgart (13)

Recently uploaded (20)

Connecting Research Questions and Research Method