![2010 CRC PhD Student Conference
How best to support scientific end-user software
development?
Aleksandra Pawlik
a.n.pawlik@open.ac.uk
Supervisors Dr. Judith Segal
Prof. Marian Petre
Prof. Helen Sharp
Department/Institute Computing
Status Full-time
Probation viva Before
Starting date October 2009
Introduction
End-user software development has received substantial amounts of attention within
both the academic and software engineering communities [1-3]. One of the sub-
groups that can be distinguished amongst end-user developers is that of scientific end-
user developers [4]. A particular set of characteristics differentiates scientists from
other end-user developers. Firstly, working in the field of science often necessitates
the use of various software packages on a daily basis. Secondly, scientists are familiar
with and utilize formal languages as well as particular modelling techniques.
Additionally, the majority of science degree curriculums offered by universities
contain at least one course in programming. Thus, many scientists have some
experience with coding at a relatively early stage of their academic and professional
career. In many cases, conducting a scientific research project means developing a
tailor-made software tool which will address a particular scientific problem. Therefore,
it may seem that scientists are “predisposed” to being effective and successful end-
user software developers more likely to produce a sustainable end-product software.
However, numerous problematic issues related to scientific end-user software
development have been reported by researchers in computing [5, 6], software
engineers [7] and scientists themselves [8]. For the purpose of my research project, I
will make the distinction between two different contexts within scientific end-user
software development:
- Limited Context: when software is developed (usually in a purely academic
environment) in order to address a specific problem within a particular project
which is being run by a limited group of anticipated users;
- Extended Context: when it is expected that the software will be reusable,
maintainable and flexible (i.e. potentially used by an extended group of as yet
undetermined users).
Scientific end-user software development needs, therefore, relevant and effective
support from the software development professionals’ community. Despite the fact
that some related help exists and is available [9], scientists who develop software and
software engineers who collaborate with them at various levels may find scientific
software development problematic. This indicates that the assistance and support
provided may need adjustments and improvements, an objective that may be
approached from different angles. First of all, it is essential to identify and examine
difficulties which may crop up during scientific end-user software development. The
second approach is to investigate and understand the origins of these problems.
Page 78 of 125](https://image.slidesharecdn.com/pawlik-100528095605-phpapp01/85/Pawlik-1-320.jpg)
![2010 CRC PhD Student Conference
Finally, we need to comprehend why the support available for scientific end-users
provided by the software development professionals’ community does not seem to be
working effectively and what steps should be taken to attempt to remedy this. I argue
that these steps need to involve observing the practices applied during scientific
software development in a number of different contexts.
In my PhD research project, I intend to focus on exploring the tools and methods
which scientific end-user developers employ in their work. The answer to the
question ”What techniques do scientific end-user developers use?” should allow me
to identify the ways in which scientists address issues that emerge during software
development. Additionally, I will pay special attention to the methods which scientific
end-user developers find successful. By “successful” I mean those that were
introduced and maintained during part or indeed the whole cycle of software
development, and which resulted in sustainable software. Thus, my second research
question is “What are the problematic and successful applications of tools and
techniques for supporting end-user software developers?". The results of my study
may potentially provide sufficient information which could be used to tailor and
improve ways of assisting scientific end-user development.
Background
A number of researchers investigated the characteristics and issues related to
scientific end-user development. For example, Segal [10] notes that the software
development process consists of short cycles and proposes an “iterative and
incremental” model of scientific software development which is a result of the fact
that the majority of scientific work remains experimental and is based on
approximation models. Moreover, some scientific projects involve tacit knowledge,
something which creates difficulties in establishing requirements and designing
software packages [11]. The experimental nature of these scientific projects, the
application of tacit knowledge and the approximations generated by mathematical
models create a further problem, that of software testing [12] [13].
Some problems are generated by the fact that many scientific end-user developers
make software within a very limited context of usage. The main aim of scientific
projects is to advance science, deliver and publish the findings. The resources (time,
finances and people) allocated to software development within the framework of a
scientific project tend to be insufficient [14]. Therefore, scientists’ reluctance to
apprehend, for example, object-oriented programming languages, and their preference
to implement code in Fortran seems justified. Moreover, by sticking with familiar
programming languages, scientific end-user developers reduce the risk of errors that
might result from the use of languages which are new or unfamiliar to them [6]. Since,
within the scientific working culture [5], software development is not made a high
priority, scientists who develop software packages do not, as a result, receive relevant
credit, something which tends to discourage them from putting more effort into
creating sustainable software [14]. Other factors which contribute to problems with
scientific end-user software development, such as lack of effective project
management or problems with the labour division, may dissuade developers from
making use of any version control systems or configuration management tools [15].
Page 79 of 125](https://image.slidesharecdn.com/pawlik-100528095605-phpapp01/85/Pawlik-2-320.jpg)
![2010 CRC PhD Student Conference
In fact, tailor-made resources relating directly to software engineering techniques and
methods supporting scientific end-user software development are available and being
continuously developed, mainly by software development professionals [16].
However, these resources only receive rather a poor uptake from the scientific
community, as scientists prefer to teach themselves from, for example, generic
textbooks, colleagues, the Internet, and so on [17] [6]. Additionally, as described by
Kelly [18], the chasm that divides the different approaches to software development
between the communities of scientific end-user developers and software development
professionals only serves to cause further discrepancies in the overall communication
between the two groups.
Methodology
I intend to investigate case studies of scientific end-user software development in
which various software engineering techniques and methods were used in covering
the following:
- The transition of turning purely academic (Limited Context) scientific software
packages into commercial ones;
- The transition of turning purely academic (Limited Context) scientific software
packages into open source (Extended Context) ones;
- The development of scientific software which directly involves software
development professionals (Extended Context).
Since this PhD research project is exploratory in nature, qualitative research methods
would seem to be the most appropriate. Moreover, studies in information systems are
highly context-dependent and interpretative [19], something which requires making
use of methods that allow researchers to investigate issues in depth. I will use
interviews and participant observation as the main methods of data collection. The
interviews will be conducted with both scientific end-user developers and software
development professionals who are directly involved, together with scientists, in
scientific software development teams. The former will constitute the majority of the
respondent group whilst interviews with software development professionals will aim
to provide additional information about the application of methods and techniques for
supporting scientific end-user development. Ideally the interviews will be combined
with participant observation enabling me to obtain a fuller picture of the process and
to perceive any issues related to scientific end-user development. Two things will be
crucial in the sampling of the case studies: being able to obtain maximum variation
within the sample, but also the ability to include convenient sampling (e.g. contacting
respondents, access to the fieldwork etc.), something which will doubtless have an
impact on the final construction of the set of case studies.
References
[1] B. A. Myers, M. M. Burnett, S. Wiedenbeck, A. J. Ko, and M. B. Rosson,
"End user software engineering: CHI: 2009 special interest group meeting," in
Proceedings of the 27th international conference extended abstracts on
Human factors in computing systems Boston, MA, USA: ACM, 2009.
[2] H. Lieberman, Paternò, F., Wulf, V., "End user development," Dordrecht, The
Netherlands: Springer, 2006.
Page 80 of 125](https://image.slidesharecdn.com/pawlik-100528095605-phpapp01/85/Pawlik-3-320.jpg)
![2010 CRC PhD Student Conference
[3] M. F. Costabile, P. Mussio, L. P. Provenza, and A. Piccinno, "End users as
unwitting software developers," in Proceedings of the 4th international
workshop on End-user software engineering Leipzig, Germany: ACM, 2008.
[4] J. Segal and S. Clarke, "Point/Counterpoint: Software Engineers Don't Know
Everything about End-User Programming," Software, IEEE, vol. 26, pp. 54-57,
2009.
[5] J. Segal, "Software Development Cultures and Cooperation Problems: A field
Study of the Early Stages of Development of Software for a Scientific
Community," Computer Supported Cooperative Work (CSCW), vol. 18, pp.
581-606, 2009.
[6] R. Sanders and D. Kelly, "Dealing with risk in scientific software
development," Software, IEEE, pp. 21-28, 2008.
[7] V. R. Basili, D. Cruzes, J. C. Carver, L. M. Hochstein, J. K. Hollingsworth, M.
V. Zelkowitz, and F. Shull, "Understanding the high-performance-computing
community: A software engineer's perspective," Software, IEEE, vol. 25, pp.
29-36, 2008.
[8] C. Rickett, S. Choi, C. Rasmussen, and M. Sottile, "Rapid prototyping
frameworks for developing scientific applications: A case study," The Journal
of Supercomputing, vol. 36, pp. 123-134, 2006.
[9] G. Wilson, "Those Who Will Not Learn From History," Computing in Science
and Engineering, vol. 10, p. 5, 2008.
[10] J. Segal, "Models of scientific software development," in Workshop on
Software Engineering in Computational Science and Engineering, Leipzig,
Germany, 2008
[11] S. Thew, A. Sutcliffe, R. Procter, O. de Bruijn, J. McNaught, C. C. Venters,
and I. Buchan, "Requirements Engineering for E-science: Experiences in
Epidemiology," Software, IEEE, vol. 26, pp. 80-87, 2009.
[12] D. Hook and D. Kelly, "Testing for trustworthiness in scientific software," in
Proceedings of the 2009 ICSE Workshop on Software Engineering for
Computational Science and Engineering: IEEE Computer Society, 2009.
[13] S. Easterbrook and T. Johns, "Engineering the Software for Understanding
Climate Change," Computing in Science and Engineering, vol. 26, 2009.
[14] "Reporting Back - Open Middleware Infrastructure Institute Collaboration
Workshops 2010," http://www.omii.ac.uk/wiki/CW10ReportingBack, 2010.
[15] M. Vigder, "End-user software development in a scientific organization," in
Proceedings of the 2009 ICSE Workshop on Software Engineering
Foundations for End User Programming: IEEE Computer Society, 2009.
[16] "Software Carpentry - an intensive introduction to basic software development
practices for scientists and engineers," http://software-carpentry.org/.
[17] G. Wilson, "How Do Scientists Really Use Computers?," American Scientist,
vol. 97, pp. 360-362, 2009.
[18] D. Kelly, "A software chasm: Software engineering and scientific computing,"
Software, IEEE, p. 120, 2007.
[19] H. K. Klein and M. D. Myers, "A set of principles for conducting and
evaluating interpretive field studies in information systems," MIS Quarterly,
vol. 23, p. 67(2), 1999.
Page 81 of 125](https://image.slidesharecdn.com/pawlik-100528095605-phpapp01/85/Pawlik-4-320.jpg)
Pawlik
- 1. 2010 CRC PhD Student Conference How best to support scientific end-user software development? Aleksandra Pawlik a.n.pawlik@open.ac.uk Supervisors Dr. Judith Segal Prof. Marian Petre Prof. Helen Sharp Department/Institute Computing Status Full-time Probation viva Before Starting date October 2009 Introduction End-user software development has received substantial amounts of attention within both the academic and software engineering communities [1-3]. One of the sub- groups that can be distinguished amongst end-user developers is that of scientific end- user developers [4]. A particular set of characteristics differentiates scientists from other end-user developers. Firstly, working in the field of science often necessitates the use of various software packages on a daily basis. Secondly, scientists are familiar with and utilize formal languages as well as particular modelling techniques. Additionally, the majority of science degree curriculums offered by universities contain at least one course in programming. Thus, many scientists have some experience with coding at a relatively early stage of their academic and professional career. In many cases, conducting a scientific research project means developing a tailor-made software tool which will address a particular scientific problem. Therefore, it may seem that scientists are “predisposed” to being effective and successful end- user software developers more likely to produce a sustainable end-product software. However, numerous problematic issues related to scientific end-user software development have been reported by researchers in computing [5, 6], software engineers [7] and scientists themselves [8]. For the purpose of my research project, I will make the distinction between two different contexts within scientific end-user software development: - Limited Context: when software is developed (usually in a purely academic environment) in order to address a specific problem within a particular project which is being run by a limited group of anticipated users; - Extended Context: when it is expected that the software will be reusable, maintainable and flexible (i.e. potentially used by an extended group of as yet undetermined users). Scientific end-user software development needs, therefore, relevant and effective support from the software development professionals’ community. Despite the fact that some related help exists and is available [9], scientists who develop software and software engineers who collaborate with them at various levels may find scientific software development problematic. This indicates that the assistance and support provided may need adjustments and improvements, an objective that may be approached from different angles. First of all, it is essential to identify and examine difficulties which may crop up during scientific end-user software development. The second approach is to investigate and understand the origins of these problems. Page 78 of 125
- 2. 2010 CRC PhD Student Conference Finally, we need to comprehend why the support available for scientific end-users provided by the software development professionals’ community does not seem to be working effectively and what steps should be taken to attempt to remedy this. I argue that these steps need to involve observing the practices applied during scientific software development in a number of different contexts. In my PhD research project, I intend to focus on exploring the tools and methods which scientific end-user developers employ in their work. The answer to the question ”What techniques do scientific end-user developers use?” should allow me to identify the ways in which scientists address issues that emerge during software development. Additionally, I will pay special attention to the methods which scientific end-user developers find successful. By “successful” I mean those that were introduced and maintained during part or indeed the whole cycle of software development, and which resulted in sustainable software. Thus, my second research question is “What are the problematic and successful applications of tools and techniques for supporting end-user software developers?". The results of my study may potentially provide sufficient information which could be used to tailor and improve ways of assisting scientific end-user development. Background A number of researchers investigated the characteristics and issues related to scientific end-user development. For example, Segal [10] notes that the software development process consists of short cycles and proposes an “iterative and incremental” model of scientific software development which is a result of the fact that the majority of scientific work remains experimental and is based on approximation models. Moreover, some scientific projects involve tacit knowledge, something which creates difficulties in establishing requirements and designing software packages [11]. The experimental nature of these scientific projects, the application of tacit knowledge and the approximations generated by mathematical models create a further problem, that of software testing [12] [13]. Some problems are generated by the fact that many scientific end-user developers make software within a very limited context of usage. The main aim of scientific projects is to advance science, deliver and publish the findings. The resources (time, finances and people) allocated to software development within the framework of a scientific project tend to be insufficient [14]. Therefore, scientists’ reluctance to apprehend, for example, object-oriented programming languages, and their preference to implement code in Fortran seems justified. Moreover, by sticking with familiar programming languages, scientific end-user developers reduce the risk of errors that might result from the use of languages which are new or unfamiliar to them [6]. Since, within the scientific working culture [5], software development is not made a high priority, scientists who develop software packages do not, as a result, receive relevant credit, something which tends to discourage them from putting more effort into creating sustainable software [14]. Other factors which contribute to problems with scientific end-user software development, such as lack of effective project management or problems with the labour division, may dissuade developers from making use of any version control systems or configuration management tools [15]. Page 79 of 125
- 3. 2010 CRC PhD Student Conference In fact, tailor-made resources relating directly to software engineering techniques and methods supporting scientific end-user software development are available and being continuously developed, mainly by software development professionals [16]. However, these resources only receive rather a poor uptake from the scientific community, as scientists prefer to teach themselves from, for example, generic textbooks, colleagues, the Internet, and so on [17] [6]. Additionally, as described by Kelly [18], the chasm that divides the different approaches to software development between the communities of scientific end-user developers and software development professionals only serves to cause further discrepancies in the overall communication between the two groups. Methodology I intend to investigate case studies of scientific end-user software development in which various software engineering techniques and methods were used in covering the following: - The transition of turning purely academic (Limited Context) scientific software packages into commercial ones; - The transition of turning purely academic (Limited Context) scientific software packages into open source (Extended Context) ones; - The development of scientific software which directly involves software development professionals (Extended Context). Since this PhD research project is exploratory in nature, qualitative research methods would seem to be the most appropriate. Moreover, studies in information systems are highly context-dependent and interpretative [19], something which requires making use of methods that allow researchers to investigate issues in depth. I will use interviews and participant observation as the main methods of data collection. The interviews will be conducted with both scientific end-user developers and software development professionals who are directly involved, together with scientists, in scientific software development teams. The former will constitute the majority of the respondent group whilst interviews with software development professionals will aim to provide additional information about the application of methods and techniques for supporting scientific end-user development. Ideally the interviews will be combined with participant observation enabling me to obtain a fuller picture of the process and to perceive any issues related to scientific end-user development. Two things will be crucial in the sampling of the case studies: being able to obtain maximum variation within the sample, but also the ability to include convenient sampling (e.g. contacting respondents, access to the fieldwork etc.), something which will doubtless have an impact on the final construction of the set of case studies. References [1] B. A. Myers, M. M. Burnett, S. Wiedenbeck, A. J. Ko, and M. B. Rosson, "End user software engineering: CHI: 2009 special interest group meeting," in Proceedings of the 27th international conference extended abstracts on Human factors in computing systems Boston, MA, USA: ACM, 2009. [2] H. Lieberman, Paternò, F., Wulf, V., "End user development," Dordrecht, The Netherlands: Springer, 2006. Page 80 of 125
- 4. 2010 CRC PhD Student Conference [3] M. F. Costabile, P. Mussio, L. P. Provenza, and A. Piccinno, "End users as unwitting software developers," in Proceedings of the 4th international workshop on End-user software engineering Leipzig, Germany: ACM, 2008. [4] J. Segal and S. Clarke, "Point/Counterpoint: Software Engineers Don't Know Everything about End-User Programming," Software, IEEE, vol. 26, pp. 54-57, 2009. [5] J. Segal, "Software Development Cultures and Cooperation Problems: A field Study of the Early Stages of Development of Software for a Scientific Community," Computer Supported Cooperative Work (CSCW), vol. 18, pp. 581-606, 2009. [6] R. Sanders and D. Kelly, "Dealing with risk in scientific software development," Software, IEEE, pp. 21-28, 2008. [7] V. R. Basili, D. Cruzes, J. C. Carver, L. M. Hochstein, J. K. Hollingsworth, M. V. Zelkowitz, and F. Shull, "Understanding the high-performance-computing community: A software engineer's perspective," Software, IEEE, vol. 25, pp. 29-36, 2008. [8] C. Rickett, S. Choi, C. Rasmussen, and M. Sottile, "Rapid prototyping frameworks for developing scientific applications: A case study," The Journal of Supercomputing, vol. 36, pp. 123-134, 2006. [9] G. Wilson, "Those Who Will Not Learn From History," Computing in Science and Engineering, vol. 10, p. 5, 2008. [10] J. Segal, "Models of scientific software development," in Workshop on Software Engineering in Computational Science and Engineering, Leipzig, Germany, 2008 [11] S. Thew, A. Sutcliffe, R. Procter, O. de Bruijn, J. McNaught, C. C. Venters, and I. Buchan, "Requirements Engineering for E-science: Experiences in Epidemiology," Software, IEEE, vol. 26, pp. 80-87, 2009. [12] D. Hook and D. Kelly, "Testing for trustworthiness in scientific software," in Proceedings of the 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering: IEEE Computer Society, 2009. [13] S. Easterbrook and T. Johns, "Engineering the Software for Understanding Climate Change," Computing in Science and Engineering, vol. 26, 2009. [14] "Reporting Back - Open Middleware Infrastructure Institute Collaboration Workshops 2010," http://www.omii.ac.uk/wiki/CW10ReportingBack, 2010. [15] M. Vigder, "End-user software development in a scientific organization," in Proceedings of the 2009 ICSE Workshop on Software Engineering Foundations for End User Programming: IEEE Computer Society, 2009. [16] "Software Carpentry - an intensive introduction to basic software development practices for scientists and engineers," http://software-carpentry.org/. [17] G. Wilson, "How Do Scientists Really Use Computers?," American Scientist, vol. 97, pp. 360-362, 2009. [18] D. Kelly, "A software chasm: Software engineering and scientific computing," Software, IEEE, p. 120, 2007. [19] H. K. Klein and M. D. Myers, "A set of principles for conducting and evaluating interpretive field studies in information systems," MIS Quarterly, vol. 23, p. 67(2), 1999. Page 81 of 125