PSI e-infrastructures evaluation
Download as PPT, PDF1 like353 views
This document presents a methodology for evaluating Public Sector Information (PSI) e-infrastructures to understand the value generated for various stakeholders, particularly government agencies and users like scientists and journalists. The methodology emphasizes the need for a structured evaluation process that considers both financial and non-financial benefits, and includes the development of value models for effective assessment. An algorithm is proposed for processing stakeholder evaluation data to derive insights and prioritize improvements.
PSI e-infrastructures evaluation
- 1. University of the Aegean – Department of Information and Communication Systems Engineering A methodology for Evaluating PSI e-Infrastructures based on Multiple Value Models Charalampos Alexopoulos, cPhD Euripides Loukis, Associate Professor
- 2. INTRODUCTION e-Science: cross border research collaboration and use of huge high-capacity computing resources – tools for collection, storage, analysis and modeling of data large amount of data is very useful for conducting scientific research in many areas socio-economic benefits understanding of social and economic problems, and also of the effectiveness of various policies government agencies implement for addressing them opening up this resource could amount to about € 40 billion a year in the EU – a new e-market governments start to invest on 5/10/20122 PCI2012 - University of the Aegean
- 3. PURPOSE a systematic evaluation of these PSI e-Infrastructures, aiming at a better understanding and assessment of value they generate a structured and comprehensive evaluation methodology is missing “ The proposed methodology includes initially the definition of one value model for each stakeholder group, which consists of the main dimensions of value the PSI e- infrastructure generates for it, and also the connections among them” 5/10/20123 PCI2012 - University of the Aegean
- 4. BACKGROUND Scoping eInfrastructures Stakeholders Data Acquisition Data Provision Communicatio n Literature Review IS Evaluation TAM IS Success Models E-Services 5/10/20124 PCI2012 - University of the Aegean
- 5. Research Streams Insights IS Evaluation IS’s offer various types of benefits, both financial and non- financial, and also tangible and intangible ones, which differ among the different types of IS it is not possible to formulate one generic IS evaluation method, which is applicable to all IS a comprehensive methodology for evaluating a particular type of IS should include evaluation of both its efficiency and its effectiveness, taking into account its particular characteristics, capabilities and objectives 5/10/20125 PCI2012 - University of the Aegean
- 6. Research Streams Insights TAM identify the characteristics and factors affecting the attitude towards using an IS, the intention to use it and finally the extent of its actual usage perceived usefulness and perceived ease of use determine an individual's intention to use a system with intention to use serving as a mediator of actual system use IS Success Models IS evaluation should adopt a layered approach based on the above interrelated IS success measures (information quality, system quality, service quality, user satisfaction, actual use, perceived usefulness, individual impact and organizational impact) and on the relations among them 5/10/20126 PCI2012 - University of the Aegean
- 7. Research Streams Insights e-Services Evaluation frameworks that assess the quality of the capabilities that the e-service provides to its users frameworks that assess the support it provides to users for performing various tasks and achieving various objectives, or users’ overall satisfaction the above frameworks do not include advanced ways of processing the evaluation data collected from the users, in order to maximize the extraction of value-related knowledge from them 5/10/20127 PCI2012 - University of the Aegean
- 8. Our Approach based on Value Models Ease of Use Experience Performance Data Search & Download Capabilities Data Provision Capabilities Accompl. Users Objectives Use Future Behaviour Users’ Data Analysis Capabilities Efficiency Level Effectiveness Level Fut. Behavior Level Data Users 5/10/20128 PCI2012 - University of the Aegean
- 9. Our Approach based on Value Models Efficiency Level Effectiveness Level Fut. Behavior Level 5/10/20129 PCI2012 - University of the Aegean Data Providers Ease of Use Experience Performance Providers’ Data Analysis Capabilities Data Upload Capabilities Accompl. Providers Objectives Use Future Behaviour
- 10. Value Measures The total of 51 value measures were defined 15 common value measures 22 value measures for users 14 value measures for providers These value measures was then converted to a question to be included in questionnaires to be distributed to stakeholders A five point Likert scale is used to measure agreement or disagreement 2 Questionnaires have been formulated 5/10/201210 PCI2012 - University of the Aegean
- 11. Value Model Estimation Algorithm 1. For each value dimension a composite variable is calculated as the average of its individual measure variables. 2. Average ratings are calculated for all value dimensions (using the composite variables calculated in step 1 3. For each value dimension of the first level we calculate its correlations with all value dimensions of the second and the third levels (using again the composite variables calculated in step 1). 4. Combination of 2 classes of analytics calculated in steps 2 and 3 for the construction of a high-level value model of the PSI e-Infrastructure 5. First Layer Value Dimensions Classification into four groups: low rating – high impact low rating – low impact high rating – high impact high rating – low impact 1. Finally we repeat stages 2, 3, 4 and 5, but this time for the individual value measures/variables instead of the value dimensions’ composite variables. 5/10/201211 PCI2012 - University of the Aegean
- 12. Conclusions 5/10/2012PCI2012 - University of the Aegean12 This paper has presented a methodology for evaluating an emerging class of IS: the PSI e-Infrastructures. These IS aim to support the evaluation of government agencies for opening their data, in order to be used for scientific, commercial and political purposes various groups of users interested in them (e.g. scientists for conducting research, active citizens and journalists for drawing conclusions on previous government activity) The proposed methodology assesses a wide range of types of value generated by PSI e-Infrastructures for these two stakeholders’ groups An algorithm for advanced processing of stakeholders’ evaluation data, which results in the estimation of value models for these two groups and the identification of improvement priorities
- 13. References 5/10/2012PCI2012 - University of the Aegean13 Commission of the European Communities (2009), “Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – ICT Infrastructures for e-Science”, COM (2009) 108 Final, Brussels. Commission of the European Communities (2011), “Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions – Open data: An engine for innovation, growth and transparent governance”, COM (2011) 882 Final, Brussels. Loukis, E. Pazalos, K. Salagara, A. (2012), “Transforming e-services evaluation data into business analytics using value models”, Elsevier, Electronic Commerce Research and Applications 11 (2012), 129–141. Makx Dekkers, Femke Polman, Robbin te Velde and Mark de Vries, MEPSIR – Measuring European Public Sector Information Resources: Final report of study on exploitation of public sector information – benchmarking of EU framework Conditions, report for the European Commission, June 2006 Kim, Y. and Crowston, K. (2011), Technology adoption and use theory review for studying scientists' continued use of cyber-infrastructure. Proc. Am. Soc. Info. Sci. Tech., 48: 1–10. doi: 10.1002/meet.2011.14504801197 Stockdale, R., and Standing, C. (2006), “An interpretive approach for interpreting information systems: a content, context, process framework”. European Journal of Operational Research, 173, , 1090–1102. Irani, Z., and Love, P., (2008), “Information systems evaluation – a crisis of understanding” In Z. Irani and P. Love (eds.), “Evaluating Information Systems – Public and Private Sector”, Butterworth-Heinemann, UK. Schepers, J. and Wetzels, M. (2007). A meta-analysis of the technology acceptance model: Investigating subjective norm and moderation effects. Information & Management, 44, pp. 90-103. Fishbein, M., & Ajzen, I. (1975). Belief, Attitude, Intention, and Behavior. Reading, MA: Addison-Wesley. Lu, J., and Zhang, G. (2003), “Cost benefit factor analysis in e-services”, International Journal of Industry Service Management, 14, 5, 570–595. Fassnacht, M., and Koese, I. (2006), “Quality of electronic servic DeLone, D.H., McLean, E. R. (2003), “The DeLone and McLean model of information systems success: a ten-year update”, Journal of Management Information Systems, 19, 4, , 9–30
- 14. Thank you for your attention 5/10/2012PCI2012 - University of the Aegean14
Editor's Notes
- #9: Such a value model consists of a set of measures assessing different types of value generated by the evaluated e-service, and the relations among them. These value measures are organized in three levels: (i) Efficiency level: it includes ‘efficiency’ measures, which assess the quality of the basic capabilities offered by the e-service to its users, (ii) Effectiveness level: it includes ‘effectiveness’ measures, which assess the extent of use of the e-service and also its outcomes (iii) Future behaviour level: it includes measures assessing to what extent the e-service influences the future behaviour of its users This methodology combines assessment of these multiple types of value generated by the e-service with estimation of the relations among them (with the former and the latter constituting the value model of the e-service), and also an algorithm for defining priorities for capabilities’ improvements.
- #10: Such a value model consists of a set of measures assessing different types of value generated by the evaluated e-service, and the relations among them. These value measures are organized in three levels: (i) Efficiency level: it includes ‘efficiency’ measures, which assess the quality of the basic capabilities offered by the e-service to its users, (ii) Effectiveness level: it includes ‘effectiveness’ measures, which assess the extent of use of the e-service and also its outcomes (iii) Future behaviour level: it includes measures assessing to what extent the e-service influences the future behaviour of its users This methodology combines assessment of these multiple types of value generated by the e-service with estimation of the relations among them (with the former and the latter constituting the value model of the e-service), and also an algorithm for defining priorities for capabilities’ improvements.