SlideShare a Scribd company logo
Middle East Technical University 
Informatics Institute 
Department of Information Systems 
IS 720 Research Methods in Information Systems 
Dissertation Review Report 
Fall 2010 
Mustafa DEĞERLİ - 1382142 
11/4/2010
1 
Preface 
This report has seven clauses, namely General Information, Introduction of the Dissertation, Literature Review of the Dissertation, Methodology of the Dissertation, Results of the Dissertation, Conclusions, Implications, Recommendations, and Summary of the Dissertation, and Adoption for My Study. 
In General Information clause, Title, Author, Location, Time, Intended Purpose of the Completion of the Dissertation, Contributors, and the Citation of the Dissertation are provided. 
In Introduction of the Dissertation, Literature Review of the Dissertation, Methodology of the Dissertation, Results of the Dissertation, Conclusions, Implications, Recommendations, and Summary of the Dissertation clauses, each of these special chapters of the dissertation are reviewed. Additionally, the outlines of the each chapter of the dissertation are provided and the information provided in each chapter is summarized and commented. In this context, main purpose or outcome of the study (as stated by the author), methodology that is used in the literature review part (inductive vs. deductive or both), research design model (single linear or multi method), and data collection methods and instruments (self developed or adopted or standardized) of the study, and the quality of the last chapter in terms of answering research questions are explained in each related clauses. 
In Adoption for My Study clause, potential of any data collection/analysis techniques that I think I can adopt for my own study and other sorts of adoptions and benefits that I am to take into account in my own study are explained.
2 
Table of Contents 
Preface ..............................................................................................................................................1 
Table of Contents ...............................................................................................................................2 
1 General Information ...................................................................................................................5 
1.1 Title ..................................................................................................................................................... 5 
1.2 Author ................................................................................................................................................. 5 
1.3 Where ................................................................................................................................................. 5 
1.4 When ................................................................................................................................................... 5 
1.5 For ....................................................................................................................................................... 5 
1.6 With ..................................................................................................................................................... 5 
1.7 Citation ................................................................................................................................................ 6 
2 Introduction of the Dissertation ..................................................................................................7 
2.1 Outline of the Chapter ........................................................................................................................ 7 
2.2 Problem Statement ............................................................................................................................. 7 
2.3 Goal ..................................................................................................................................................... 8 
2.4 Hypotheses and Research Questions .................................................................................................. 8 
Hypothesis 1 (H1) .................................................................................................................................. 8 
Research Question 1 (RQ1) ................................................................................................................... 9 
Research Question 2 (RQ2) ................................................................................................................... 9 
Hypothesis 2 (H2) .................................................................................................................................. 9 
Research Question 3 (RQ3) ................................................................................................................... 9 
Research Question 4 (RQ4) ................................................................................................................... 9 
2.5 Assumptions, Barriers and Issues ....................................................................................................... 9 
Assumptions .......................................................................................................................................... 9 
Barriers and Issues .............................................................................................................................. 10 
2.5 Overall Evaluation of the Chapter ..................................................................................................... 10 
Table 1: Maturity Table for the 1st Chapter of the Dissertation ..................................................... 10 
3 Literature Review of the Dissertation ........................................................................................ 11 
3.1 Outline of the Chapter ...................................................................................................................... 11 
3.1 Defining IS/IT ..................................................................................................................................... 11
3 
3.2 Methodology used in the Literature Review .................................................................................... 12 
Figure 1: Literature Review – Deductive Methodology Flow ......................................................... 12 
Figure 2: Literature Review – Inductive Methodology Flow ........................................................... 13 
3.3 Overall Evaluation of the Chapter ..................................................................................................... 13 
Table 2: Maturity Table for the 2nd Chapter of the Dissertation..................................................... 14 
4 Methodology of the Dissertation .............................................................................................. 15 
4.1 Outline of the Chapter ...................................................................................................................... 15 
4.2 Research Design Model..................................................................................................................... 15 
Figure 3: Phases of the Research Design ........................................................................................ 16 
4.3 Overall Evaluation of the Chapter ..................................................................................................... 16 
Table 3: Maturity Table for the 3rd Chapter of the Dissertation ..................................................... 17 
5 Results of the Dissertation ........................................................................................................ 18 
5.1 Outline of the Chapter ...................................................................................................................... 18 
5.2 Results ............................................................................................................................................... 18 
Figure 4: Model Development ........................................................................................................ 19 
Table 4: Content-analytic Summary ................................................................................................ 20 
5.3 Overall Evaluation of the Chapter ..................................................................................................... 20 
Table 5: Maturity Table for the 4th Chapter of the Dissertation ..................................................... 21 
6 Conclusions, Implications, Recommendations, and Summary of the Dissertation ....................... 22 
6.1 Outline of the Chapter ...................................................................................................................... 22 
6.2 Summary and Evaluation of the Last Chapter of Dissertation .......................................................... 23 
Table 6: Constructs, Descriptions and Sub-classifications .............................................................. 24 
Figure 5: Developed Conceptual Model ......................................................................................... 25 
Table 7: Checklist for Conducting IS/IT Evaluations ........................................................................ 26 
6.3 Quality of the Last Chapter Concerning Answers for Research Questions ....................................... 27 
Hypothesis 1 (H1) ................................................................................................................................ 27 
Research Question 1 (RQ1) ................................................................................................................. 27 
Research Question 2 (RQ2) ................................................................................................................. 27 
Hypothesis 2 (H2) ................................................................................................................................ 27 
Research Question 3 (RQ3) ................................................................................................................. 27 
Research Question 4 (RQ4) ................................................................................................................. 27 
Answer for Research Question #1: ..................................................................................................... 28
4 
Answer for Research Question #2: ..................................................................................................... 28 
Conclusion for Hypothesis #1: ............................................................................................................ 28 
Answer for Research Question #3: ..................................................................................................... 29 
Answer for Research Question #4: ..................................................................................................... 29 
Conclusion for Hypothesis #2: ............................................................................................................ 30 
Table 8: Maturity Table for the 5th Chapter of the Dissertation ..................................................... 30 
7 Adoption for My Study ............................................................................................................. 31 
7.1 My Study’s Title and Abstract ........................................................................................................... 31 
Title ..................................................................................................................................................... 31 
Abstract ............................................................................................................................................... 31 
7.2 Adoptions .......................................................................................................................................... 32
5 
1 General Information 
1.1 Title 
The title of the reviewed dissertation is “A Model for the Evaluation of IS/IT Investments.” 
1.2 Author 
The author of the reviewed dissertation is “Paul M. Tuten.” 
1.3 Where 
The reviewed dissertation is completed in “Graduate School of Computer and Information Sciences in Nova Southeastern University.” 
1.4 When 
The reviewed dissertation is completed in “January 2009.” 
1.5 For 
The reviewed dissertation is completed as a “Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Information Systems.” 
1.6 With 
The reviewed dissertation is completed by Paul M. Tuten with the following academic staff: John Scigliano, Ed.D. as Chairperson of Dissertation Committee William L. Hafner, Ph.D. as Dissertation Committee Member Sumitra Mukherjee, Ph.D. as Dissertation Committee Member
6 
1.7 Citation 
The reviewed dissertation’s citation is provided below. 
Tuten, P. M. A model for the evaluation of IS/IT investments. Ph.D. dissertation, Nova Southeastern University, United States - Florida. Retrieved October 31, 2010, from Dissertations & Theses: Full Text. (Publication No. AAT 3344877).
7 
2 Introduction of the Dissertation 
2.1 Outline of the Chapter 
The “Introduction” part of the reviewed dissertation has the following parts: Problem Statement, Goal, Hypotheses, Research Questions & Assumptions, Relevance & Significance, Barriers and Issues, Limitations / Delimitations of Study, Resource Requirements, Summary, and Definition of Terms. 
2.2 Problem Statement 
The problem investigated in the study was the complexity and difficulty faced by practitioners in evaluating investments in IS/IT. 
The researcher tries to criticize that evaluation of the investments in IS/IT by practitioners is a complex and difficult task to do under the current conditions.
8 
2.3 Goal 
The researcher’s goals were to examine IS/IT evaluation, together with its approaches, techniques, and methods, as well as their application in organizations, and to develop a theoretical model to offer guidelines for organizations to occupy contextually-sensitive evaluation methods. 
2.4 Hypotheses and Research Questions 
The researcher proposed two hypotheses and two research questions for each of these hypotheses. Specifically, there are two hypotheses and four research questions in the related reviewed dissertation. 
Hypothesis 1 (H1) 
Existing models of IS/IT evaluation are inadequate because they fail to include all of the relevant constructs: the purpose of conducting the evaluation (why); the subject of the evaluation (what); the specific aspects to be evaluated (which); the particular evaluation methods and techniques used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the evaluation (who); and the external and internal environmental conditions under which the organization operates (where).
9 
Research Question 1 (RQ1) 
What models of the IS/IT evaluation process are presented in the literature? 
Research Question 2 (RQ2) 
How do the constructs (identified in H1) relate to the process of IS/IT evaluation? 
Hypothesis 2 (H2) 
An improved conceptual model of IS/IT evaluation provides an effective tool for describing and analyzing evaluation practices. 
Research Question 3 (RQ3) 
Is the researcher’s conceptual model valid for describing IS/IT evaluation practices? 
Research Question 4 (RQ4) 
What guidelines may be derived from using the researcher’s conceptual model as an analytical tool to existing IS/IT evaluation case studies? 
2.5 Assumptions, Barriers and Issues 
Assumptions 
The researcher provides three major assumptions in the dissertation. 
1. Putting on one side philosophical and epistemological arguments about the “true” nature of reality, the researcher assumes that individuals’ perceptions or interpretations of reality drive their actions. 
2. The researcher believes that the conceptual model of IS/IT evaluation should be. That is, it should be able to describe equally well the activities of individuals regardless of the correctness or value of their actions.
10 
3. Despite the need for contextual appropriateness in IS/IT evaluation, the researcher assumes that practitioners require a sufficient degree of methodological guidance with the aim of “get-the-job-done” successfully as well. 
Barriers and Issues 
Purposely, the following barriers and issues were identified by the researcher. Philosophical challenges inherent in conceptual modeling. Philosophical and practical difficulties associated with assessing theoretical contributions. The intractable challenges inherent in IS/IT evaluation. The mystery of balancing contextual-sensitivity with sufficient methodological guidance. The potential lack of industrial awareness and use by practitioners. 
2.5 Overall Evaluation of the Chapter 
As this chapter of the dissertation explicitly and clearly provides information about the problem statement, goal, hypotheses, research questions & assumptions, relevance & significance, barriers and issues, limitations and delimitations, and the resource requirements of the dissertation study, this chapter is evaluated as “Fully Implemented.” 
Table 1: Maturity Table for the 1st Chapter of the Dissertation 
Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction 
X
11 
3 Literature Review of the Dissertation 
3.1 Outline of the Chapter 
The “Literature Review” part of the reviewed dissertation has the following parts: Defining IS/IT Evaluation, Why: The Purpose of Evaluation, Where: Extra- and Intra-Organizational Environmental Conditions, When: The Timing of Evaluation, What: The Object of Evaluation, Who: The People Involved in Evaluation, Which: Evaluation Criteria/Measures, How: IS/IT Evaluation Methods, Techniques, and Approaches, Examining the Puzzle: Understanding Evaluation in Context, Literature Review: Key Themes, and Summary. 
3.1 Defining IS/IT 
The researcher first acknowledges distinction between information systems (IS) and information technology (IT), as IT is referred as an organization’s hardware, software, and related infrastructure, and IS is referred as the design of information flows that attempt to meet an organization’s informational needs. Additionally, the researcher notes that in theory, IS may or may not be primarily based on information technology, in practice, however, IS -especially ones subjected to a formal evaluation process- contain some IT element. For the purpose of this study,
12 
the researcher generally used the terms interchangeably and noted any particular instances in 
which a distinction between the concepts was to the point. 
3.2 Methodology used in the Literature Review 
The researcher used both deductive and inductive methodology in the literature review part of 
the dissertation. From the deductive perspective, the researcher explored IS/IT evaluation by 
examining the underlying assumptions, professional practices, and continuing concerns of both 
practitioners and academicians. For this purpose, the researcher deconstructed IS/IT evaluation 
into a mass of contextual elements. In this context, each of these elements was considered 
independently and then, in relation to each other. Subsequently, the researcher identified themes 
that span this varied body of literature with the purpose of drawing provisional conclusions about 
the current state-of-the-art. 
IT/IS Evaluation Deconstruction 
Contextual Elements 
Drawing 
Provisional 
Conclusions 
Figure 1: Literature Review – Deductive Methodology Flow 
Mostly, the researcher demonstrated that the contextual elements of IS/IT evaluation must be 
better understood in order to advance the field’s effectiveness and significance. Eventually, as
13 
the researcher note, this improved understanding should take the form of a conceptual model of 
IS/IT evaluation, which may be utilized for both descriptive and normative purposes. 
From the inductive perspective, the researcher synthesized the fragmented, disjointed, and 
contradictory insights into a unified-whole, including following themes: 
IS/IT Evaluation is Problematic for Researchers and Practitioners 
IS/IT Evaluation is about More Than Estimating or Measuring Outcomes 
IS/IT Evaluation Practice is (and should be) Pragmatic 
IS/IT Evaluation is Moving beyond the Positivist / Interpretivist Dualism 
IS/IT Evaluation Involves Many Complex, Related Contextual Elements 
IS/IT Evaluation Needs a Theory for Descriptive and Normative Purposes 
Fragmented Insights 
Disjointed Insights 
Contradictory Insights 
Synthesize 
Unified-whole Themes 
Figure 2: Literature Review – Inductive Methodology Flow 
3.3 Overall Evaluation of the Chapter 
In the literature review, the researcher established that IS/IT evaluations are comprised of 
several, interconnected contextual elements that shall be better understood.
14 
These elements, as noted by the researcher, include: the purpose of conducting the evaluation (why); the subject of the evaluation (what); the specific aspects to be evaluated (which); the particular evaluation methods and techniques used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the evaluation (who); and the external and internal environmental conditions under which the organization operates (where). 
Each of these constructs was examined in detail by the researcher in the literature review part of the dissertation, including special emphasis on the specific criteria or measures of evaluation (which question), and the methods or techniques of evaluation (how question). 
Additionally, the researcher reviewed existing models which depicted the process of IS/IT evaluation, as he note down the limitations and differences found in each example. 
Owing to above details, this chapter is also evaluated as “Fully Implemented.” 
Table 2: Maturity Table for the 2nd Chapter of the Dissertation 
Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction 
X Chapter 2 Literature Review 
X
15 
4 Methodology of the Dissertation 
4.1 Outline of the Chapter 
The “Methodology” part of the reviewed dissertation has the following parts: Step 1: Conduct Comprehensive Literature Review Step 2: Develop IS/IT Evaluation Conceptual Model Step 3: Validate and Apply the IS/IT Evaluation Conceptual Model Step 4: Report Study Results Limitations / Delimitations / Assumptions Summary 
4.2 Research Design Model 
The researcher utilized a multiphase approach in the study. There are four main stages for the related dissertation’s research. 
These are: The implementation of all-inclusive literature review, The development of the conceptual model of IS/IT evaluation to provide better understanding of the process’s entity constructs and their possible relationships, The validation of the conceptual model via a meta-analysis of multiple case studies and the development of guidelines for conducting contextual evaluations within particular organizations, and The reporting of the results of this study.
16 
Comprehensive 
Literature Review 
Development of 
Conceptual Model Validation of 
Conceptual Model 
Reporting of 
the Results 
Figure 3: Phases of the Research Design 
4.3 Overall Evaluation of the Chapter 
The researcher conducted a comprehensive literature review to address Hypothesis 1 and endow 
with answers to Research Question 1 and Research Question 2. 
After this, by using the literature review findings the researcher developed the conceptual model 
in the study. 
The researcher’s conceptual model represents the innermost work of art of the study, to provide 
the decisive link between Hypothesis 1 and Hypothesis 2. 
Subsequently, the researcher validated the conceptual model based upon case studies and then 
utilized it to develop normative guidelines meant for conducting evaluations, in that way
17 
addressing Hypothesis 2 by means of answering Research Question 3 and Research Question 4 correspondingly. 
After all, the researcher reported the findings associated with each of the hypotheses and research questions and discussed implications for both future academic research and professional practices. 
As seen above, in four steps the researcher is done. That is to say, the research is completed in these four steps by justifying two hypotheses with reference to four research questions. Please see Sub-clause 2.4 of this document for hypotheses and research questions. 
In consequence of above details, this chapter is also evaluated as “Fully Implemented.” 
Table 3: Maturity Table for the 3rd Chapter of the Dissertation 
Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction 
X Chapter 2 Literature Review 
X Chapter 3 Methodology 
X
18 
5 Results of the Dissertation 
5.1 Outline of the Chapter 
The “Results” part of the reviewed dissertation has the following parts: Assembling the Puzzle: A Conceptual Model of IS/IT Evaluation Introduction to Case Study Analysis for Model Validation Case Study #1: UK Insurance Company’s IS/IT Evaluation Practices Case Study #2: Dutch Insurance Company’s IS/IT Evaluation Practices Case Study #3: UK Manufacturing Company’s Evaluation of IS Infrastructure Case Study #4: UK Manufacturing Company’s Evaluation of MRPII System Case Study #5: US Department of Defense Evaluation of an E-Business System Validating the Conceptual Model: Is it a “Good” Theoretical Contribution? In Search of Normative Guidelines: Cross-Case Analysis Summary 
5.2 Results 
Throughout the Results chapter, the researcher reported the results using a sequence of events style. The reason for the researcher to do this is to reflect the iterative and cyclical processes associated with analyzing, writing, and reflecting throughout this study. Furthermore, the researcher selected this sort of reporting structure in response to Yin’s (2003) call to use the writing and editing process as an analytical tool and mechanism to clarify thoughts. 
In the chapter, the researcher described the construction of a conceptual model of the process of IS/IT evaluation based on the findings from the literature review. The researcher began by describing the limitations associated with existing conceptual models. Next, the researcher
19 
identified seven constructs associated with the context of an evaluation. Finally, the researcher 
developed these constructs into a conceptual model, which resulted after multiple iterations of 
model development. 
Describe Limitations 
of Existing Models 
Identify Constructs 
of Evaluation 
Develop 
Conceptual Model 
Figure 4: Model Development 
After developing conceptual model, the researcher validated it using a multi-case study analysis. 
In this context, the researcher followed the steps explained in Methodology part, and reviewed 
and coded five case studies to discover confirming or disconfirming evidences. 
The researcher confirmed that the conceptual model that is developed represented a “good” 
theoretical contribution based on Whetten’s (2002) standard, which required the conceptual 
model to be complete and systematic in its rationalization. 
In conclusion, the researcher performed a cross-case analysis to identify elements that could 
serve as the basis for methodological guidelines for conducting more contextually appropriate 
IS/IT evaluations.
20 
In this context, the researcher described similarities and differences among the narratives of the case studies. Likewise, the researcher also identified four “drivers” of contextual evaluations, as well as at least two examples of how each driver was implemented in practice, Table 4. Based upon these findings, the researcher constructed IS/IT evaluation guidelines that are described in the last chapter o the dissertation, including a checklist for practitioners support. 
Table 4: Content-analytic Summary 
5.3 Overall Evaluation of the Chapter 
Using the procedures explained in Methodology chapter of the dissertation, the researcher resented a proposed conceptual model of the process of IS/IT evaluation based on the study’s literature review,
21 
validated the proposed model using a multi-case study analysis, and performed a comprehensive cross-case analysis to identify key observations that informed the researcher’s proposed methodological guidelines. 
The above steps are satisfactorily done with enough information. Therefore, this chapter is evaluated as “Fully Implemented.” 
Table 5: Maturity Table for the 4th Chapter of the Dissertation 
Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction 
X Chapter 2 Literature Review 
X Chapter 3 Methodology 
X Chapter 4 Results 
X
22 
6 Conclusions, Implications, Recommendations, and Summary of the Dissertation 
6.1 Outline of the Chapter 
The “Conclusions, Implications, Recommendations, and Summary” part of the reviewed dissertation has the following parts: Conclusions Hypothesis #1: The Contextual Elements of an Evaluation Research Question #1: Models of IS/IT Evaluation in the Literature Research Question #2: Contextual Elements and the Evaluation Process Hypothesis #1: Conclusion Hypothesis #2: Validity and Usability of an Improved Conceptual Model Research Question #3: Is the Conceptual Model of Evaluation Valid? Research Question #4: What Guidelines May Be Derived from the Model? Hypothesis #2: Conclusion Reflections on Validity: Limitations, Assumptions, and Concerns Implications Recommendations Recommendations for Future Research Recommendations for the Improvement of Professional Practice Summary: Recommendations for the Improvement of Professional Practice Summary
23 
6.2 Summary and Evaluation of the Last Chapter of Dissertation 
Having accomplished a comprehensive literature review, defined a research method, and discovered a number of findings; 
the researcher finalized the study by drawing a number of conclusions and recommendations. Additionally, the researcher considered the implications of the study’s outcomes for practitioners and researchers. 
The researcher utilized the model to develop a series of guidelines and a checklist to aid organizations in conducting context-based IS/IT evaluations. 
In particular, the researcher provided recommendations to assist evaluators in: Engaging in critical and reflective practice Increasing stakeholder participation Aligning evaluation criteria (which) with the organizational context Aligning evaluation methods (how) with the organizational context Learning from experience 
In addition to providing guidelines for improved professional practice, the researcher set forth a series of recommendations for subsequent academic research. These recommendations included a call for further validating the conceptual model, making additional refinements and/or extensions to it, developing a comprehensive theory of IS/IT evaluation rooted in the conceptual model, and the exploring the implications for pedagogy of the researcher’s findings.
24 
To summarize the literature review findings, the researcher developed a summary including description of each construct and any relevant sub-classifications available in the literature. See Table 6 for details. 
Table 6: Constructs, Descriptions and Sub-classifications 
The researcher’s conceptual model describes the interactions between the unique conceptual elements above comprising the process of IS/IT evaluation. See Figure 5 for the developed conceptual model.
25 
Figure 5: Developed Conceptual Model 
The researcher believes that the development of a holistic and full-bodied conceptual model that resulted from the study serves as a significant step in advancing of IS/IT evaluation theory. 
Additionally, the researcher’s guidelines and checklist to assist practitioners in conducting context-based IS/IT evaluation offers a significant contribution to industrial practice. See Table 7 for checklist for conducting better IS/IT evaluations developed in the scope of the dissertation.
26 
Table 7: Checklist for Conducting IS/IT Evaluations
27 
6.3 Quality of the Last Chapter Concerning Answers for Research Questions 
Hypothesis 1 (H1) 
Existing models of IS/IT evaluation are inadequate because they fail to include all of the relevant constructs: the purpose of conducting the evaluation (why); the subject of the evaluation (what); the specific aspects to be evaluated (which); the particular evaluation methods and techniques used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the evaluation (who); and the external and internal environmental conditions under which the organization operates (where). 
Research Question 1 (RQ1) 
What models of the IS/IT evaluation process are presented in the literature? 
Research Question 2 (RQ2) 
How do the constructs (identified in H1) relate to the process of IS/IT evaluation? 
Hypothesis 2 (H2) 
An improved conceptual model of IS/IT evaluation provides an effective tool for describing and analyzing evaluation practices. 
Research Question 3 (RQ3) 
Is the researcher’s conceptual model valid for describing IS/IT evaluation practices? 
Research Question 4 (RQ4) 
What guidelines may be derived from using the researcher’s conceptual model as an analytical tool to existing IS/IT evaluation case studies?
28 
Answer for Research Question #1: 
For this purpose, the researcher wanted to identify conceptual models of the context and process of IS/IT evaluation and the researcher distinguished between these meta-models of evaluation versus more specific models of a particular evaluation method. Set this limitation, the researcher found a few instances. Moreover, the vast majority of the examples were rooted in the work of Symons (1990. Overall, the researcher found five meta-models of the process of IS/IT evaluation. In each of these models, one or more of the seven evaluation constructs identified in the researcher’s literature review were not there. 
Answer for Research Question #2: 
In Methodology chapter of the dissertation, the researcher presented underlying principles for including each of the seven identified conceptual elements of IS/IT evaluation, and a comprehensive literature review specific to each construct. Based upon these findings, the researcher followed Whetten’s (2002) methodology for developing theoretical contributions, such as conceptual models. Following numerous iterations and revisions, the researcher created the conceptual model of IS/IT evaluation that appeared consistent with the findings of the literature review. 
Conclusion for Hypothesis #1: 
The researcher concluded that the results support the first hypothesis in this study. The process of IS/IT evaluation consists of seven contextual elements. Although there are number of existing models of IS/IT evaluation in the literature, these models mostly failed to explicitly include all of these relevant constructs. For that reason, the existing models have not either sufficient or inclusive explanations of the process of IS/IT evaluation in organizations. In contrast, the
29 
researcher in this study utilized these existing models, and the findings of the literature review in order to develop an alternative conceptual model of IS/IT evaluation that included all seven contextual elements with sufficient or inclusive explanations. 
Answer for Research Question #3: 
In the study, the researcher wanted to ensure that the proposed conceptual model represented a “good” theoretical contribution. To that end, the researcher applied Whetten’s (2002) standard for strong theoretical contributions. To assess whether or not the proposed model of IS/IT evaluation represented a good theoretical contribution, the researcher explored its descriptive ability with respect to the previously published case studies. In all of the examined cases, the researcher established that the conceptual model’s constructs were valid and relevant. Thus, the researcher concluded that the conceptual model did not contain superfluous constructs. Therefore, the researcher concluded that the conceptual model was sufficiently complete. The researcher provided that the model provides a systematic description of the process of IS/IT evaluation. 
Answer for Research Question #4: 
Having established the descriptive validity of the conceptual model, the researcher investigated its application as an investigative tool. In order to accomplish this task, the researcher performed a cross-case analysis using the conceptual model as a framework, thereby facilitating comparisons across the various instances. In doing so, the researcher found clear evidence supporting the assertion.
30 
Conclusion for Hypothesis #2: 
The researcher noted that there is adequate evidence supporting the second hypothesis in the study. The researcher’s conceptual model represents an effective tool for both describing and analyzing evaluation practices. The researcher’s application of the conceptual model to the cases examined in this study yielded a number of normative guidelines for contextually appropriate IS/IT evaluation practices. 
As the main research questions receive explicit answers and the whole study is well-summarized in the last chapter, this chapter is also evaluated as “Fully Implemented.” 
Table 8: Maturity Table for the 5th Chapter of the Dissertation 
Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction 
X Chapter 2 Literature Review 
X Chapter 3 Methodology 
X Chapter 4 Results 
X Chapter 5 Conclusions, Implications, Recom., and Summary 
X
31 
7 Adoption for My Study 
7.1 My Study’s Title and Abstract 
Title 
Identifying the Managers’ Consciousness about the Importance of Management of Information Technologies and Its Relation to the Success of Organizations 
Abstract 
Today, organizations greatly spend money on hardware and software systems or more precisely on information technologies (IT), grown in power and ubiquity recently. The fundamental reason for this is that organizations view IT as more and more decisive to their success in the sector. IT organizations’ chief executive officers (CEOs) and senior managers persistently note that they interpret IT as a strategic asset to be successful and thereby leader in the sector. However, this is something problematic. For a business asset to be interpreted as strategic it has to be scarce and it ought to require exceptional and grounded interpretation and application. Given that IT ubiquity everywhere, it is not possible to interpret IT as a strategic business asset. Yet, it is possible to do so for the management of IT. Today’s organizations may create a sustained competitive advantage by means of management of IT rather than IT of itself. 
This thesis aims to identify the CEOs’ and senior managers’ consciousness about the importance of management of IT and its relation to the organizations’ success in the sector. In this context, first the 50 IT organizations’ CEOs and senior managers are provided and asked to complete a questionnaire to know their consciousness about the importance of management of IT and their organizations’ culture and success. After this, results of these questionnaires are analyzed and interpreted. In addition, theoretical and practical evidences for this consciousness and its impacts
32 
to success are provided and exemplified. As a result, a consciousness frame is proposed for the IT organizations’ CEOs and senior managers to appreciate the importance of management of IT rather than IT of itself. 
7.2 Adoptions As my study also requires contextual and conceptual analysis and comparisons, just like the dissertation that I reviewed, I am to utilize a multiphase approach in my study. To develop my own IT management consciousness framework survey, I will follow the similar way that the dissertation’s owner followed in the development of the conceptual model of IS/IT evaluation to provide better understanding of the process’s entity constructs and their possible relationships. For the validation of my framework, I will follow the similar way that the dissertation’s owner followed the validation of the conceptual model via a meta-analysis and cross- analysis of multiple case studies and the development of guidelines for conducting contextual evaluations within particular organizations. This dissertation, both in content and context will let me continue my study as I also plan to a similar but not the same occurrence in my thesis. I also will use the outcome of this study as an input for my thesis as I am to investigate the IT investments versus IT management consciousness relations.

More Related Content

PPTX
The literature review
PDF
マイナンバーたな卸し表を作ろう
PPT
Apa formatting guidelines for dissertation
DOC
Writing a dissertation
PPT
How To Write Your Research Dissertation
PPTX
Expressionism Modernism Sustainable Architecture Rookery Walt Disney Concert ...
PPT
APA style guide powerpoint
PPTX
Thesis writing using apa format
The literature review
マイナンバーたな卸し表を作ろう
Apa formatting guidelines for dissertation
Writing a dissertation
How To Write Your Research Dissertation
Expressionism Modernism Sustainable Architecture Rookery Walt Disney Concert ...
APA style guide powerpoint
Thesis writing using apa format

Similar to Mustafa Degerli - 2010 - Dissertation Review - IS 720 Research Methods in Information Systems (20)

DOCX
Final Assignment Written assignment (Research Proposal)
PDF
Course Summary - Dissertation I: Principles of Research and Writing
PPT
Seminar Course instruction .ppt
PPT
Multiview Methodology
DOCX
MITS6011 MITS6012 Advanced Research T.docx
PPTX
Selection of Dissertation Topic and Searching for Literature
DOCX
My Thesis Guide
PPTX
Desktop PPT edited automatically Template.pptx
DOCX
Annotation_M1.docxSubject Information SystemsJoshi, G. (2.docx
DOCX
Research Area. Importance of physical security as it relates to fa.docx
DOCX
CS631 Formal Research Report or QA (Online)The Formal Research Re
PPT
Doing a Literature Review - Part 1
DOCX
INFO 4710 Information Technology Management – Spring 2020 .docx
DOCX
Graduate Studies in computer Information Systems Guidelines .docx
PPTX
Research Proposal Research Proposal.pptx
DOCX
TOPIC Write an original research report consisting of one of the .docx
PDF
Example questions from doctoral defense of Mikko Vesisenaho
DOCX
INFO 4710 Information Technology Management – Spring 2020 .docx
PPT
Intro to research modules
DOCX
Cultural plungeFor this assignment, you will engage in a cultu
Final Assignment Written assignment (Research Proposal)
Course Summary - Dissertation I: Principles of Research and Writing
Seminar Course instruction .ppt
Multiview Methodology
MITS6011 MITS6012 Advanced Research T.docx
Selection of Dissertation Topic and Searching for Literature
My Thesis Guide
Desktop PPT edited automatically Template.pptx
Annotation_M1.docxSubject Information SystemsJoshi, G. (2.docx
Research Area. Importance of physical security as it relates to fa.docx
CS631 Formal Research Report or QA (Online)The Formal Research Re
Doing a Literature Review - Part 1
INFO 4710 Information Technology Management – Spring 2020 .docx
Graduate Studies in computer Information Systems Guidelines .docx
Research Proposal Research Proposal.pptx
TOPIC Write an original research report consisting of one of the .docx
Example questions from doctoral defense of Mikko Vesisenaho
INFO 4710 Information Technology Management – Spring 2020 .docx
Intro to research modules
Cultural plungeFor this assignment, you will engage in a cultu
Ad

More from Dr. Mustafa Değerli (20)

PDF
Dr. Mustafa Degerli - PMI PMP Certification
PDF
AI in Project Management - Course completed by Dr. Mustafa Değerli
PDF
IISEC 22 - Mustafa Degerli
PDF
Privacy Issues in Data-Driven Health Care
PDF
Declarations of Software Engineering Project Managers Managing Remotely: Prov...
PDF
Declarations of Software Engineering Project Managers Managing Remotely: Prov...
PDF
Risk Management Framework - Dr. Mustafa Degerli
PDF
A Comprehensive Overview and Interpretation of Risk and Uncertainty in Projec...
PDF
METU Best PhD Thesis Award
PDF
Mobil Sağlık Uygulamalarına Genel Bir Bakış ve Özgün Bir Mobil Sağlık Uygulam...
PDF
Teknoloji Transfer Ofisleri için Kritik Başarı Faktörleri
PDF
Crafting a CMMI V2 Compliant Process for Governance Practice Area: An Experie...
PDF
A Mobile Health Application for Healthy Living: HWOW (Healthier Work for Offi...
PDF
Etkili Bir Kalite Güvence Sürecinin Parçası Olarak Proje Seviyesindeki Deneti...
PDF
Yazılım Mühendisliği ve Sistem Mühendisliği Süreçlerinin Harmanlanması: ISO/I...
PDF
Olgun Bir Süreç Yönetimi Ekibi Organizasyonu Tasarımı: Deneyimler ve Öneriler
PDF
Bir Süreç Uyarlama Yaklaşımı Örneği: Süreç Uyarlama Matrisi (SUM), Deneyimler...
PDF
Sistem Projelerinde Elektrik & Elektronik Muühendisleri ve Bilgisayar Mühendi...
PDF
Mobil İşletim Sistemleri (iOS ve Android) Açısından Kullanıcı Memnuniyetini E...
PDF
Yazılım veya Yazılım-Yoğun Sistem Mühendisliği İş Süreçleri Açısından Yüksek ...
Dr. Mustafa Degerli - PMI PMP Certification
AI in Project Management - Course completed by Dr. Mustafa Değerli
IISEC 22 - Mustafa Degerli
Privacy Issues in Data-Driven Health Care
Declarations of Software Engineering Project Managers Managing Remotely: Prov...
Declarations of Software Engineering Project Managers Managing Remotely: Prov...
Risk Management Framework - Dr. Mustafa Degerli
A Comprehensive Overview and Interpretation of Risk and Uncertainty in Projec...
METU Best PhD Thesis Award
Mobil Sağlık Uygulamalarına Genel Bir Bakış ve Özgün Bir Mobil Sağlık Uygulam...
Teknoloji Transfer Ofisleri için Kritik Başarı Faktörleri
Crafting a CMMI V2 Compliant Process for Governance Practice Area: An Experie...
A Mobile Health Application for Healthy Living: HWOW (Healthier Work for Offi...
Etkili Bir Kalite Güvence Sürecinin Parçası Olarak Proje Seviyesindeki Deneti...
Yazılım Mühendisliği ve Sistem Mühendisliği Süreçlerinin Harmanlanması: ISO/I...
Olgun Bir Süreç Yönetimi Ekibi Organizasyonu Tasarımı: Deneyimler ve Öneriler
Bir Süreç Uyarlama Yaklaşımı Örneği: Süreç Uyarlama Matrisi (SUM), Deneyimler...
Sistem Projelerinde Elektrik & Elektronik Muühendisleri ve Bilgisayar Mühendi...
Mobil İşletim Sistemleri (iOS ve Android) Açısından Kullanıcı Memnuniyetini E...
Yazılım veya Yazılım-Yoğun Sistem Mühendisliği İş Süreçleri Açısından Yüksek ...
Ad

Recently uploaded (20)

PDF
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
PDF
WOOl fibre morphology and structure.pdf for textiles
PPTX
Final SEM Unit 1 for mit wpu at pune .pptx
PDF
A comparative study of natural language inference in Swahili using monolingua...
PPTX
TLE Review Electricity (Electricity).pptx
PDF
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
PPTX
O2C Customer Invoices to Receipt V15A.pptx
PPTX
Tartificialntelligence_presentation.pptx
PDF
August Patch Tuesday
PDF
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
PDF
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
PPTX
Programs and apps: productivity, graphics, security and other tools
PDF
A contest of sentiment analysis: k-nearest neighbor versus neural network
PDF
Univ-Connecticut-ChatGPT-Presentaion.pdf
PPT
What is a Computer? Input Devices /output devices
PDF
Zenith AI: Advanced Artificial Intelligence
PPTX
Group 1 Presentation -Planning and Decision Making .pptx
PPTX
Modernising the Digital Integration Hub
PDF
Getting Started with Data Integration: FME Form 101
PDF
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf
From MVP to Full-Scale Product A Startup’s Software Journey.pdf
WOOl fibre morphology and structure.pdf for textiles
Final SEM Unit 1 for mit wpu at pune .pptx
A comparative study of natural language inference in Swahili using monolingua...
TLE Review Electricity (Electricity).pptx
TrustArc Webinar - Click, Consent, Trust: Winning the Privacy Game
O2C Customer Invoices to Receipt V15A.pptx
Tartificialntelligence_presentation.pptx
August Patch Tuesday
Video forgery: An extensive analysis of inter-and intra-frame manipulation al...
ENT215_Completing-a-large-scale-migration-and-modernization-with-AWS.pdf
Programs and apps: productivity, graphics, security and other tools
A contest of sentiment analysis: k-nearest neighbor versus neural network
Univ-Connecticut-ChatGPT-Presentaion.pdf
What is a Computer? Input Devices /output devices
Zenith AI: Advanced Artificial Intelligence
Group 1 Presentation -Planning and Decision Making .pptx
Modernising the Digital Integration Hub
Getting Started with Data Integration: FME Form 101
Microsoft Solutions Partner Drive Digital Transformation with D365.pdf

Mustafa Degerli - 2010 - Dissertation Review - IS 720 Research Methods in Information Systems

  • 1. Middle East Technical University Informatics Institute Department of Information Systems IS 720 Research Methods in Information Systems Dissertation Review Report Fall 2010 Mustafa DEĞERLİ - 1382142 11/4/2010
  • 2. 1 Preface This report has seven clauses, namely General Information, Introduction of the Dissertation, Literature Review of the Dissertation, Methodology of the Dissertation, Results of the Dissertation, Conclusions, Implications, Recommendations, and Summary of the Dissertation, and Adoption for My Study. In General Information clause, Title, Author, Location, Time, Intended Purpose of the Completion of the Dissertation, Contributors, and the Citation of the Dissertation are provided. In Introduction of the Dissertation, Literature Review of the Dissertation, Methodology of the Dissertation, Results of the Dissertation, Conclusions, Implications, Recommendations, and Summary of the Dissertation clauses, each of these special chapters of the dissertation are reviewed. Additionally, the outlines of the each chapter of the dissertation are provided and the information provided in each chapter is summarized and commented. In this context, main purpose or outcome of the study (as stated by the author), methodology that is used in the literature review part (inductive vs. deductive or both), research design model (single linear or multi method), and data collection methods and instruments (self developed or adopted or standardized) of the study, and the quality of the last chapter in terms of answering research questions are explained in each related clauses. In Adoption for My Study clause, potential of any data collection/analysis techniques that I think I can adopt for my own study and other sorts of adoptions and benefits that I am to take into account in my own study are explained.
  • 3. 2 Table of Contents Preface ..............................................................................................................................................1 Table of Contents ...............................................................................................................................2 1 General Information ...................................................................................................................5 1.1 Title ..................................................................................................................................................... 5 1.2 Author ................................................................................................................................................. 5 1.3 Where ................................................................................................................................................. 5 1.4 When ................................................................................................................................................... 5 1.5 For ....................................................................................................................................................... 5 1.6 With ..................................................................................................................................................... 5 1.7 Citation ................................................................................................................................................ 6 2 Introduction of the Dissertation ..................................................................................................7 2.1 Outline of the Chapter ........................................................................................................................ 7 2.2 Problem Statement ............................................................................................................................. 7 2.3 Goal ..................................................................................................................................................... 8 2.4 Hypotheses and Research Questions .................................................................................................. 8 Hypothesis 1 (H1) .................................................................................................................................. 8 Research Question 1 (RQ1) ................................................................................................................... 9 Research Question 2 (RQ2) ................................................................................................................... 9 Hypothesis 2 (H2) .................................................................................................................................. 9 Research Question 3 (RQ3) ................................................................................................................... 9 Research Question 4 (RQ4) ................................................................................................................... 9 2.5 Assumptions, Barriers and Issues ....................................................................................................... 9 Assumptions .......................................................................................................................................... 9 Barriers and Issues .............................................................................................................................. 10 2.5 Overall Evaluation of the Chapter ..................................................................................................... 10 Table 1: Maturity Table for the 1st Chapter of the Dissertation ..................................................... 10 3 Literature Review of the Dissertation ........................................................................................ 11 3.1 Outline of the Chapter ...................................................................................................................... 11 3.1 Defining IS/IT ..................................................................................................................................... 11
  • 4. 3 3.2 Methodology used in the Literature Review .................................................................................... 12 Figure 1: Literature Review – Deductive Methodology Flow ......................................................... 12 Figure 2: Literature Review – Inductive Methodology Flow ........................................................... 13 3.3 Overall Evaluation of the Chapter ..................................................................................................... 13 Table 2: Maturity Table for the 2nd Chapter of the Dissertation..................................................... 14 4 Methodology of the Dissertation .............................................................................................. 15 4.1 Outline of the Chapter ...................................................................................................................... 15 4.2 Research Design Model..................................................................................................................... 15 Figure 3: Phases of the Research Design ........................................................................................ 16 4.3 Overall Evaluation of the Chapter ..................................................................................................... 16 Table 3: Maturity Table for the 3rd Chapter of the Dissertation ..................................................... 17 5 Results of the Dissertation ........................................................................................................ 18 5.1 Outline of the Chapter ...................................................................................................................... 18 5.2 Results ............................................................................................................................................... 18 Figure 4: Model Development ........................................................................................................ 19 Table 4: Content-analytic Summary ................................................................................................ 20 5.3 Overall Evaluation of the Chapter ..................................................................................................... 20 Table 5: Maturity Table for the 4th Chapter of the Dissertation ..................................................... 21 6 Conclusions, Implications, Recommendations, and Summary of the Dissertation ....................... 22 6.1 Outline of the Chapter ...................................................................................................................... 22 6.2 Summary and Evaluation of the Last Chapter of Dissertation .......................................................... 23 Table 6: Constructs, Descriptions and Sub-classifications .............................................................. 24 Figure 5: Developed Conceptual Model ......................................................................................... 25 Table 7: Checklist for Conducting IS/IT Evaluations ........................................................................ 26 6.3 Quality of the Last Chapter Concerning Answers for Research Questions ....................................... 27 Hypothesis 1 (H1) ................................................................................................................................ 27 Research Question 1 (RQ1) ................................................................................................................. 27 Research Question 2 (RQ2) ................................................................................................................. 27 Hypothesis 2 (H2) ................................................................................................................................ 27 Research Question 3 (RQ3) ................................................................................................................. 27 Research Question 4 (RQ4) ................................................................................................................. 27 Answer for Research Question #1: ..................................................................................................... 28
  • 5. 4 Answer for Research Question #2: ..................................................................................................... 28 Conclusion for Hypothesis #1: ............................................................................................................ 28 Answer for Research Question #3: ..................................................................................................... 29 Answer for Research Question #4: ..................................................................................................... 29 Conclusion for Hypothesis #2: ............................................................................................................ 30 Table 8: Maturity Table for the 5th Chapter of the Dissertation ..................................................... 30 7 Adoption for My Study ............................................................................................................. 31 7.1 My Study’s Title and Abstract ........................................................................................................... 31 Title ..................................................................................................................................................... 31 Abstract ............................................................................................................................................... 31 7.2 Adoptions .......................................................................................................................................... 32
  • 6. 5 1 General Information 1.1 Title The title of the reviewed dissertation is “A Model for the Evaluation of IS/IT Investments.” 1.2 Author The author of the reviewed dissertation is “Paul M. Tuten.” 1.3 Where The reviewed dissertation is completed in “Graduate School of Computer and Information Sciences in Nova Southeastern University.” 1.4 When The reviewed dissertation is completed in “January 2009.” 1.5 For The reviewed dissertation is completed as a “Dissertation Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in Information Systems.” 1.6 With The reviewed dissertation is completed by Paul M. Tuten with the following academic staff: John Scigliano, Ed.D. as Chairperson of Dissertation Committee William L. Hafner, Ph.D. as Dissertation Committee Member Sumitra Mukherjee, Ph.D. as Dissertation Committee Member
  • 7. 6 1.7 Citation The reviewed dissertation’s citation is provided below. Tuten, P. M. A model for the evaluation of IS/IT investments. Ph.D. dissertation, Nova Southeastern University, United States - Florida. Retrieved October 31, 2010, from Dissertations & Theses: Full Text. (Publication No. AAT 3344877).
  • 8. 7 2 Introduction of the Dissertation 2.1 Outline of the Chapter The “Introduction” part of the reviewed dissertation has the following parts: Problem Statement, Goal, Hypotheses, Research Questions & Assumptions, Relevance & Significance, Barriers and Issues, Limitations / Delimitations of Study, Resource Requirements, Summary, and Definition of Terms. 2.2 Problem Statement The problem investigated in the study was the complexity and difficulty faced by practitioners in evaluating investments in IS/IT. The researcher tries to criticize that evaluation of the investments in IS/IT by practitioners is a complex and difficult task to do under the current conditions.
  • 9. 8 2.3 Goal The researcher’s goals were to examine IS/IT evaluation, together with its approaches, techniques, and methods, as well as their application in organizations, and to develop a theoretical model to offer guidelines for organizations to occupy contextually-sensitive evaluation methods. 2.4 Hypotheses and Research Questions The researcher proposed two hypotheses and two research questions for each of these hypotheses. Specifically, there are two hypotheses and four research questions in the related reviewed dissertation. Hypothesis 1 (H1) Existing models of IS/IT evaluation are inadequate because they fail to include all of the relevant constructs: the purpose of conducting the evaluation (why); the subject of the evaluation (what); the specific aspects to be evaluated (which); the particular evaluation methods and techniques used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the evaluation (who); and the external and internal environmental conditions under which the organization operates (where).
  • 10. 9 Research Question 1 (RQ1) What models of the IS/IT evaluation process are presented in the literature? Research Question 2 (RQ2) How do the constructs (identified in H1) relate to the process of IS/IT evaluation? Hypothesis 2 (H2) An improved conceptual model of IS/IT evaluation provides an effective tool for describing and analyzing evaluation practices. Research Question 3 (RQ3) Is the researcher’s conceptual model valid for describing IS/IT evaluation practices? Research Question 4 (RQ4) What guidelines may be derived from using the researcher’s conceptual model as an analytical tool to existing IS/IT evaluation case studies? 2.5 Assumptions, Barriers and Issues Assumptions The researcher provides three major assumptions in the dissertation. 1. Putting on one side philosophical and epistemological arguments about the “true” nature of reality, the researcher assumes that individuals’ perceptions or interpretations of reality drive their actions. 2. The researcher believes that the conceptual model of IS/IT evaluation should be. That is, it should be able to describe equally well the activities of individuals regardless of the correctness or value of their actions.
  • 11. 10 3. Despite the need for contextual appropriateness in IS/IT evaluation, the researcher assumes that practitioners require a sufficient degree of methodological guidance with the aim of “get-the-job-done” successfully as well. Barriers and Issues Purposely, the following barriers and issues were identified by the researcher. Philosophical challenges inherent in conceptual modeling. Philosophical and practical difficulties associated with assessing theoretical contributions. The intractable challenges inherent in IS/IT evaluation. The mystery of balancing contextual-sensitivity with sufficient methodological guidance. The potential lack of industrial awareness and use by practitioners. 2.5 Overall Evaluation of the Chapter As this chapter of the dissertation explicitly and clearly provides information about the problem statement, goal, hypotheses, research questions & assumptions, relevance & significance, barriers and issues, limitations and delimitations, and the resource requirements of the dissertation study, this chapter is evaluated as “Fully Implemented.” Table 1: Maturity Table for the 1st Chapter of the Dissertation Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction X
  • 12. 11 3 Literature Review of the Dissertation 3.1 Outline of the Chapter The “Literature Review” part of the reviewed dissertation has the following parts: Defining IS/IT Evaluation, Why: The Purpose of Evaluation, Where: Extra- and Intra-Organizational Environmental Conditions, When: The Timing of Evaluation, What: The Object of Evaluation, Who: The People Involved in Evaluation, Which: Evaluation Criteria/Measures, How: IS/IT Evaluation Methods, Techniques, and Approaches, Examining the Puzzle: Understanding Evaluation in Context, Literature Review: Key Themes, and Summary. 3.1 Defining IS/IT The researcher first acknowledges distinction between information systems (IS) and information technology (IT), as IT is referred as an organization’s hardware, software, and related infrastructure, and IS is referred as the design of information flows that attempt to meet an organization’s informational needs. Additionally, the researcher notes that in theory, IS may or may not be primarily based on information technology, in practice, however, IS -especially ones subjected to a formal evaluation process- contain some IT element. For the purpose of this study,
  • 13. 12 the researcher generally used the terms interchangeably and noted any particular instances in which a distinction between the concepts was to the point. 3.2 Methodology used in the Literature Review The researcher used both deductive and inductive methodology in the literature review part of the dissertation. From the deductive perspective, the researcher explored IS/IT evaluation by examining the underlying assumptions, professional practices, and continuing concerns of both practitioners and academicians. For this purpose, the researcher deconstructed IS/IT evaluation into a mass of contextual elements. In this context, each of these elements was considered independently and then, in relation to each other. Subsequently, the researcher identified themes that span this varied body of literature with the purpose of drawing provisional conclusions about the current state-of-the-art. IT/IS Evaluation Deconstruction Contextual Elements Drawing Provisional Conclusions Figure 1: Literature Review – Deductive Methodology Flow Mostly, the researcher demonstrated that the contextual elements of IS/IT evaluation must be better understood in order to advance the field’s effectiveness and significance. Eventually, as
  • 14. 13 the researcher note, this improved understanding should take the form of a conceptual model of IS/IT evaluation, which may be utilized for both descriptive and normative purposes. From the inductive perspective, the researcher synthesized the fragmented, disjointed, and contradictory insights into a unified-whole, including following themes: IS/IT Evaluation is Problematic for Researchers and Practitioners IS/IT Evaluation is about More Than Estimating or Measuring Outcomes IS/IT Evaluation Practice is (and should be) Pragmatic IS/IT Evaluation is Moving beyond the Positivist / Interpretivist Dualism IS/IT Evaluation Involves Many Complex, Related Contextual Elements IS/IT Evaluation Needs a Theory for Descriptive and Normative Purposes Fragmented Insights Disjointed Insights Contradictory Insights Synthesize Unified-whole Themes Figure 2: Literature Review – Inductive Methodology Flow 3.3 Overall Evaluation of the Chapter In the literature review, the researcher established that IS/IT evaluations are comprised of several, interconnected contextual elements that shall be better understood.
  • 15. 14 These elements, as noted by the researcher, include: the purpose of conducting the evaluation (why); the subject of the evaluation (what); the specific aspects to be evaluated (which); the particular evaluation methods and techniques used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the evaluation (who); and the external and internal environmental conditions under which the organization operates (where). Each of these constructs was examined in detail by the researcher in the literature review part of the dissertation, including special emphasis on the specific criteria or measures of evaluation (which question), and the methods or techniques of evaluation (how question). Additionally, the researcher reviewed existing models which depicted the process of IS/IT evaluation, as he note down the limitations and differences found in each example. Owing to above details, this chapter is also evaluated as “Fully Implemented.” Table 2: Maturity Table for the 2nd Chapter of the Dissertation Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction X Chapter 2 Literature Review X
  • 16. 15 4 Methodology of the Dissertation 4.1 Outline of the Chapter The “Methodology” part of the reviewed dissertation has the following parts: Step 1: Conduct Comprehensive Literature Review Step 2: Develop IS/IT Evaluation Conceptual Model Step 3: Validate and Apply the IS/IT Evaluation Conceptual Model Step 4: Report Study Results Limitations / Delimitations / Assumptions Summary 4.2 Research Design Model The researcher utilized a multiphase approach in the study. There are four main stages for the related dissertation’s research. These are: The implementation of all-inclusive literature review, The development of the conceptual model of IS/IT evaluation to provide better understanding of the process’s entity constructs and their possible relationships, The validation of the conceptual model via a meta-analysis of multiple case studies and the development of guidelines for conducting contextual evaluations within particular organizations, and The reporting of the results of this study.
  • 17. 16 Comprehensive Literature Review Development of Conceptual Model Validation of Conceptual Model Reporting of the Results Figure 3: Phases of the Research Design 4.3 Overall Evaluation of the Chapter The researcher conducted a comprehensive literature review to address Hypothesis 1 and endow with answers to Research Question 1 and Research Question 2. After this, by using the literature review findings the researcher developed the conceptual model in the study. The researcher’s conceptual model represents the innermost work of art of the study, to provide the decisive link between Hypothesis 1 and Hypothesis 2. Subsequently, the researcher validated the conceptual model based upon case studies and then utilized it to develop normative guidelines meant for conducting evaluations, in that way
  • 18. 17 addressing Hypothesis 2 by means of answering Research Question 3 and Research Question 4 correspondingly. After all, the researcher reported the findings associated with each of the hypotheses and research questions and discussed implications for both future academic research and professional practices. As seen above, in four steps the researcher is done. That is to say, the research is completed in these four steps by justifying two hypotheses with reference to four research questions. Please see Sub-clause 2.4 of this document for hypotheses and research questions. In consequence of above details, this chapter is also evaluated as “Fully Implemented.” Table 3: Maturity Table for the 3rd Chapter of the Dissertation Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction X Chapter 2 Literature Review X Chapter 3 Methodology X
  • 19. 18 5 Results of the Dissertation 5.1 Outline of the Chapter The “Results” part of the reviewed dissertation has the following parts: Assembling the Puzzle: A Conceptual Model of IS/IT Evaluation Introduction to Case Study Analysis for Model Validation Case Study #1: UK Insurance Company’s IS/IT Evaluation Practices Case Study #2: Dutch Insurance Company’s IS/IT Evaluation Practices Case Study #3: UK Manufacturing Company’s Evaluation of IS Infrastructure Case Study #4: UK Manufacturing Company’s Evaluation of MRPII System Case Study #5: US Department of Defense Evaluation of an E-Business System Validating the Conceptual Model: Is it a “Good” Theoretical Contribution? In Search of Normative Guidelines: Cross-Case Analysis Summary 5.2 Results Throughout the Results chapter, the researcher reported the results using a sequence of events style. The reason for the researcher to do this is to reflect the iterative and cyclical processes associated with analyzing, writing, and reflecting throughout this study. Furthermore, the researcher selected this sort of reporting structure in response to Yin’s (2003) call to use the writing and editing process as an analytical tool and mechanism to clarify thoughts. In the chapter, the researcher described the construction of a conceptual model of the process of IS/IT evaluation based on the findings from the literature review. The researcher began by describing the limitations associated with existing conceptual models. Next, the researcher
  • 20. 19 identified seven constructs associated with the context of an evaluation. Finally, the researcher developed these constructs into a conceptual model, which resulted after multiple iterations of model development. Describe Limitations of Existing Models Identify Constructs of Evaluation Develop Conceptual Model Figure 4: Model Development After developing conceptual model, the researcher validated it using a multi-case study analysis. In this context, the researcher followed the steps explained in Methodology part, and reviewed and coded five case studies to discover confirming or disconfirming evidences. The researcher confirmed that the conceptual model that is developed represented a “good” theoretical contribution based on Whetten’s (2002) standard, which required the conceptual model to be complete and systematic in its rationalization. In conclusion, the researcher performed a cross-case analysis to identify elements that could serve as the basis for methodological guidelines for conducting more contextually appropriate IS/IT evaluations.
  • 21. 20 In this context, the researcher described similarities and differences among the narratives of the case studies. Likewise, the researcher also identified four “drivers” of contextual evaluations, as well as at least two examples of how each driver was implemented in practice, Table 4. Based upon these findings, the researcher constructed IS/IT evaluation guidelines that are described in the last chapter o the dissertation, including a checklist for practitioners support. Table 4: Content-analytic Summary 5.3 Overall Evaluation of the Chapter Using the procedures explained in Methodology chapter of the dissertation, the researcher resented a proposed conceptual model of the process of IS/IT evaluation based on the study’s literature review,
  • 22. 21 validated the proposed model using a multi-case study analysis, and performed a comprehensive cross-case analysis to identify key observations that informed the researcher’s proposed methodological guidelines. The above steps are satisfactorily done with enough information. Therefore, this chapter is evaluated as “Fully Implemented.” Table 5: Maturity Table for the 4th Chapter of the Dissertation Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction X Chapter 2 Literature Review X Chapter 3 Methodology X Chapter 4 Results X
  • 23. 22 6 Conclusions, Implications, Recommendations, and Summary of the Dissertation 6.1 Outline of the Chapter The “Conclusions, Implications, Recommendations, and Summary” part of the reviewed dissertation has the following parts: Conclusions Hypothesis #1: The Contextual Elements of an Evaluation Research Question #1: Models of IS/IT Evaluation in the Literature Research Question #2: Contextual Elements and the Evaluation Process Hypothesis #1: Conclusion Hypothesis #2: Validity and Usability of an Improved Conceptual Model Research Question #3: Is the Conceptual Model of Evaluation Valid? Research Question #4: What Guidelines May Be Derived from the Model? Hypothesis #2: Conclusion Reflections on Validity: Limitations, Assumptions, and Concerns Implications Recommendations Recommendations for Future Research Recommendations for the Improvement of Professional Practice Summary: Recommendations for the Improvement of Professional Practice Summary
  • 24. 23 6.2 Summary and Evaluation of the Last Chapter of Dissertation Having accomplished a comprehensive literature review, defined a research method, and discovered a number of findings; the researcher finalized the study by drawing a number of conclusions and recommendations. Additionally, the researcher considered the implications of the study’s outcomes for practitioners and researchers. The researcher utilized the model to develop a series of guidelines and a checklist to aid organizations in conducting context-based IS/IT evaluations. In particular, the researcher provided recommendations to assist evaluators in: Engaging in critical and reflective practice Increasing stakeholder participation Aligning evaluation criteria (which) with the organizational context Aligning evaluation methods (how) with the organizational context Learning from experience In addition to providing guidelines for improved professional practice, the researcher set forth a series of recommendations for subsequent academic research. These recommendations included a call for further validating the conceptual model, making additional refinements and/or extensions to it, developing a comprehensive theory of IS/IT evaluation rooted in the conceptual model, and the exploring the implications for pedagogy of the researcher’s findings.
  • 25. 24 To summarize the literature review findings, the researcher developed a summary including description of each construct and any relevant sub-classifications available in the literature. See Table 6 for details. Table 6: Constructs, Descriptions and Sub-classifications The researcher’s conceptual model describes the interactions between the unique conceptual elements above comprising the process of IS/IT evaluation. See Figure 5 for the developed conceptual model.
  • 26. 25 Figure 5: Developed Conceptual Model The researcher believes that the development of a holistic and full-bodied conceptual model that resulted from the study serves as a significant step in advancing of IS/IT evaluation theory. Additionally, the researcher’s guidelines and checklist to assist practitioners in conducting context-based IS/IT evaluation offers a significant contribution to industrial practice. See Table 7 for checklist for conducting better IS/IT evaluations developed in the scope of the dissertation.
  • 27. 26 Table 7: Checklist for Conducting IS/IT Evaluations
  • 28. 27 6.3 Quality of the Last Chapter Concerning Answers for Research Questions Hypothesis 1 (H1) Existing models of IS/IT evaluation are inadequate because they fail to include all of the relevant constructs: the purpose of conducting the evaluation (why); the subject of the evaluation (what); the specific aspects to be evaluated (which); the particular evaluation methods and techniques used (how); the timing of the evaluation (when); the individuals involved in, or affected by, the evaluation (who); and the external and internal environmental conditions under which the organization operates (where). Research Question 1 (RQ1) What models of the IS/IT evaluation process are presented in the literature? Research Question 2 (RQ2) How do the constructs (identified in H1) relate to the process of IS/IT evaluation? Hypothesis 2 (H2) An improved conceptual model of IS/IT evaluation provides an effective tool for describing and analyzing evaluation practices. Research Question 3 (RQ3) Is the researcher’s conceptual model valid for describing IS/IT evaluation practices? Research Question 4 (RQ4) What guidelines may be derived from using the researcher’s conceptual model as an analytical tool to existing IS/IT evaluation case studies?
  • 29. 28 Answer for Research Question #1: For this purpose, the researcher wanted to identify conceptual models of the context and process of IS/IT evaluation and the researcher distinguished between these meta-models of evaluation versus more specific models of a particular evaluation method. Set this limitation, the researcher found a few instances. Moreover, the vast majority of the examples were rooted in the work of Symons (1990. Overall, the researcher found five meta-models of the process of IS/IT evaluation. In each of these models, one or more of the seven evaluation constructs identified in the researcher’s literature review were not there. Answer for Research Question #2: In Methodology chapter of the dissertation, the researcher presented underlying principles for including each of the seven identified conceptual elements of IS/IT evaluation, and a comprehensive literature review specific to each construct. Based upon these findings, the researcher followed Whetten’s (2002) methodology for developing theoretical contributions, such as conceptual models. Following numerous iterations and revisions, the researcher created the conceptual model of IS/IT evaluation that appeared consistent with the findings of the literature review. Conclusion for Hypothesis #1: The researcher concluded that the results support the first hypothesis in this study. The process of IS/IT evaluation consists of seven contextual elements. Although there are number of existing models of IS/IT evaluation in the literature, these models mostly failed to explicitly include all of these relevant constructs. For that reason, the existing models have not either sufficient or inclusive explanations of the process of IS/IT evaluation in organizations. In contrast, the
  • 30. 29 researcher in this study utilized these existing models, and the findings of the literature review in order to develop an alternative conceptual model of IS/IT evaluation that included all seven contextual elements with sufficient or inclusive explanations. Answer for Research Question #3: In the study, the researcher wanted to ensure that the proposed conceptual model represented a “good” theoretical contribution. To that end, the researcher applied Whetten’s (2002) standard for strong theoretical contributions. To assess whether or not the proposed model of IS/IT evaluation represented a good theoretical contribution, the researcher explored its descriptive ability with respect to the previously published case studies. In all of the examined cases, the researcher established that the conceptual model’s constructs were valid and relevant. Thus, the researcher concluded that the conceptual model did not contain superfluous constructs. Therefore, the researcher concluded that the conceptual model was sufficiently complete. The researcher provided that the model provides a systematic description of the process of IS/IT evaluation. Answer for Research Question #4: Having established the descriptive validity of the conceptual model, the researcher investigated its application as an investigative tool. In order to accomplish this task, the researcher performed a cross-case analysis using the conceptual model as a framework, thereby facilitating comparisons across the various instances. In doing so, the researcher found clear evidence supporting the assertion.
  • 31. 30 Conclusion for Hypothesis #2: The researcher noted that there is adequate evidence supporting the second hypothesis in the study. The researcher’s conceptual model represents an effective tool for both describing and analyzing evaluation practices. The researcher’s application of the conceptual model to the cases examined in this study yielded a number of normative guidelines for contextually appropriate IS/IT evaluation practices. As the main research questions receive explicit answers and the whole study is well-summarized in the last chapter, this chapter is also evaluated as “Fully Implemented.” Table 8: Maturity Table for the 5th Chapter of the Dissertation Clauses vs. Implementations Not Implemented Partially Implemented Largely Implemented Fully Implemented Chapter 1 Introduction X Chapter 2 Literature Review X Chapter 3 Methodology X Chapter 4 Results X Chapter 5 Conclusions, Implications, Recom., and Summary X
  • 32. 31 7 Adoption for My Study 7.1 My Study’s Title and Abstract Title Identifying the Managers’ Consciousness about the Importance of Management of Information Technologies and Its Relation to the Success of Organizations Abstract Today, organizations greatly spend money on hardware and software systems or more precisely on information technologies (IT), grown in power and ubiquity recently. The fundamental reason for this is that organizations view IT as more and more decisive to their success in the sector. IT organizations’ chief executive officers (CEOs) and senior managers persistently note that they interpret IT as a strategic asset to be successful and thereby leader in the sector. However, this is something problematic. For a business asset to be interpreted as strategic it has to be scarce and it ought to require exceptional and grounded interpretation and application. Given that IT ubiquity everywhere, it is not possible to interpret IT as a strategic business asset. Yet, it is possible to do so for the management of IT. Today’s organizations may create a sustained competitive advantage by means of management of IT rather than IT of itself. This thesis aims to identify the CEOs’ and senior managers’ consciousness about the importance of management of IT and its relation to the organizations’ success in the sector. In this context, first the 50 IT organizations’ CEOs and senior managers are provided and asked to complete a questionnaire to know their consciousness about the importance of management of IT and their organizations’ culture and success. After this, results of these questionnaires are analyzed and interpreted. In addition, theoretical and practical evidences for this consciousness and its impacts
  • 33. 32 to success are provided and exemplified. As a result, a consciousness frame is proposed for the IT organizations’ CEOs and senior managers to appreciate the importance of management of IT rather than IT of itself. 7.2 Adoptions As my study also requires contextual and conceptual analysis and comparisons, just like the dissertation that I reviewed, I am to utilize a multiphase approach in my study. To develop my own IT management consciousness framework survey, I will follow the similar way that the dissertation’s owner followed in the development of the conceptual model of IS/IT evaluation to provide better understanding of the process’s entity constructs and their possible relationships. For the validation of my framework, I will follow the similar way that the dissertation’s owner followed the validation of the conceptual model via a meta-analysis and cross- analysis of multiple case studies and the development of guidelines for conducting contextual evaluations within particular organizations. This dissertation, both in content and context will let me continue my study as I also plan to a similar but not the same occurrence in my thesis. I also will use the outcome of this study as an input for my thesis as I am to investigate the IT investments versus IT management consciousness relations.