Managing Inquiry-based Learning: Learning from experience

C.D. Thomson, A. Corbett, M. Holcombe University of Sheffield, Department of Computer Science and the Institute of Work Psychology This work was supported by an EPSRC grant: EP/D031516 – the Sheffield Software Engineering Observatory.

Teaching Overview 20 years of experience teaching three modules. [Parker 1999 A & B, Holcombe 1998, Thomson 2007 A] The introduction of standardised technology over the last 7 years. Based in the computer science department. In each module the student’s develop a piece of software based on requirements that they capture. In each successive module the students are given more autonomy and harder technical challenges. A high percentage of the marks are awarded on the process and evidence of it.

Software Hut module In the second year, teams of 4-6 students, 15 hours per week, over 12 weeks. 3-4 teams compete to deliver a product to a external customer. Problems: Students would produce great software but not record how they did it. Students did not plan their work formally, occasionally this meant that a team would not deliver.

Week Date Deliverable Items 1 14-15/2 THURSDAY – initial briefing and lecture on XP, FRIDAY – meeting clients 2 21-22/2 THURSDAY – lecture on Unit tests, FRIDAY – Management meeting 3 28-29/2 THURSDAY – lecture on System testing and modelling, FRIDAY - Release 1: Demo to client and manager. Place code and binary in release directory . 4 26/2 THURSDAY – lecture on how to package and deliver software, FRIDAY – Management and client meeting – deliver requirements document 5 13-14/3 THURSDAY – lecture on debugging, refactoring, library development, Place code and binary in release directory. FRIDAY – Management and client meeting Easter 16/3-7/4 6 10-11/4 THURSDAY – lecture and XP review, FRIDAY – Management and client meeting 7 17-18/4 FRIDAY - Release 3: Demo to client and manager. Place code and binary in release directory. 8 24-25/4 FRIDAY – Management meeting 9 1-2/5 FRIDAY - Release 4: Demo to client and manager. Place code and binary in release directory. 10 8-9/5 FRIDAY – Extensive systems testing and Demonstration to client. 11 15-16/5 MONDAY – Final release to client. Please hand in a CD and printed documentation to be set to the client, via reception. WEDNESDAY – QA exercise email to Mike and the other team. 12 22-23/5 MONDAY – Final release and QA hand in, to be checked by the managers, place in team directory. (NB you may correct any errors found by the client or after the QA exercise). FRIDAY – Poster presentations in the Lewin Lab and Personal Evaluations via reception (posters also from Crossover and Genesys teams – external visitors will be present..

Software Hut mark scheme The following will be assessed for the team as a whole: 5 A set of weekly meetings, entered into the management tool (automatically copied to wiki). 5 One or more well documented code libraries. 15 XP Compliance as measured by details on the Wiki: Story cards; Pair working/programming (via tasks on story card pages); Automated and Manual tests; Small releases. The following will be assessed for each team member: 20 XP Compliance as above, but for individual entries: Tasks on story card pages; Time sheets. 5 Personal: Personal Evaluation, in the final week of the semester (5).

Timeline Crossover Software Hut Genesys 2004 2002 2003 2001 2005 2006 2007 2008 Current management tool introduced Linked wiki First wiki Web management tool Formalised templates Test server

Legacy system Students were asked to record two aspects of their process: Their meetings (in the form of timesheets, actions, agendas and minutes) And their time spent on a timesheet. Our first step in 2003 was to formalise these (in software hut) so that each team used the same template, thus making them easier to compare. The quality was very variable, often a document would be present but little information was contained in it. Meetings were often not minuted in the second half of the project.

Typical minutes Team: Group LL (XP) Date/Time: 07/03/03 09:06---10:55(9:00 am) 11:15---11:35(with client) Agenda: Discuss the structure of the software Chair: S1 Present: Every team member Apologies: None Details of Discussion: 9:00 am meeting: 1. Produced some story cards; 2. Made the Power Point, which will be shown to the client, to simulate the software that will be produced; 3. Do the research for the life cycles that we are going to show in the software With Client: A few points from the client: Give us CD for the relevant information; Show students the opportunities of this path; Do not put so many complex stuff into the system, because that will scare the students away (consider the difficulties’ level); Action Points: Have a meeting with client on 18/03 10:00---12:00; Finish story cards and test sets which are needed for the requirements documents by next Friday; Do the estimation for the project; The client will be free every Monday and Wednesday(off normal working hours) Get the CD from Dr. M George Next Meeting: 07/03/03 Mapping Building LT03 with client

Another example Team: LL (XP) Date/Time: 14/02/03 13:00---13:53 Agenda: Discussed Software Hut preferences and Roles Chair: S1 Present: Every team member Apologies: None Details of Discussion: Discussed Software Hut preferences; Discussed Software Hut Roles; Looked at what each person had to accomplish, and decide each member’s job. Action Points: 1 Print the stuff for the archivist; 2 Decide the role for each member: S1 is chair, S2 is leader of the team, S3 is the secretary and S4 the archivist; 3 Look into what XP is; 4 Wait to find out which client we are working with. Next Meeting: 17/02/03 Lewin Lab

Directory structure In order to capture the process followed we depended on taking an image of the teams’ directory structure weekly. In some cases this worked well with teams updating their files and arranging them into predefined locations. Other teams ignored this advice, and appeared to work only sporadically or at the end of the project.

Formalising process capture To address these issues we set out to: Make clear exactly what was required. Place a grade bearing requirement on the students to provide evidence that they followed a process. Provide tools that would guide the students to produce evidence.

Wiki mk 1 With the Genesys students we introduced a wiki (mediaWiki) where they could document their projects. We hoped this would help to provide continuity from one year to the next. We found that it required a committed student to ensure the wiki was updated and was uniform across the various projects. Often the wiki was well maintained at the start of the project but not at the end – when the documentation is most needed.

Code management systems Teams of developers working on software often use tools to manage the files they create. These can be used to record the work of an individual. Popular tools include: CVS Subversion We found that using such a system could get in the way of the main teaching aims with the students in years 1 and 2. However the fourth year students find them easier to use perhaps because they are more familiar with the task.

Problems with code management systems for evidence We found that students did not fully understand how to use these systems, which resulted in their frustration and time not spent on the core problem. Two specific classes of problem are worth mentioning [Thomson 2008]: The non-use of the system (0.16 Errors Per File). This resulted in limited information about the roles of the team members and their working practice. Direct manipulation of the repository. Data corruption due to files deposited without using the CVS tool. Data corruption due the re-initialisation of the repository (0.03 EPF).

Web management tool This tool was written in PHP and accessed via a web browser [Thomson 2007 B]. The aim was to implement the formalised documents so that the students would have to fill in the required fields. However the editing was rudimentary, particularly in the first year of use. This was refined in the second year.

Figure 2: Software Hut 2003-4 usage. Figure 1: Software Hut 2004-5 usage.

Managing Inquiry-based Learning: Learning from experience

Web tool feedback In the first version of this tool the students had to manually associate the different forms of data, they found this difficult to do. In the second run we automated this process, but they still did not associate items, other than meetings and agendas. The students commented that it was default to enter the text, as often boxes were the wrong size and laid out awkwardly.

Hackystat Hackystat is a tool (Uni. Of Hawaii) that automatically records process information from software engineering tools. [Johnson 2007] We set this up but the students were not keen on using it as it felt like it was snooping too much. The information collected could be queried by the students using a web interface, but it was not visually appealing.

Management tool The server is implemented in PHP/MySQL. The client is implemented in Java and can run on most computers. The Java interface allows a richer experience for the client, which is easier to use. We can customise the view to allow different student groups to see different functionality. It is possible to add entries directly into teams diaries to remind them of important deadlines.

Useage Data Figure 3. Software Hut 2007-8 Figure 3. Software Hut 2006-7 Week Week Number of hits Number of hits

Tool minutes example 1) Present 1.1) Members 53:;78:;59:;57: 1.2) Chair 59: 1.3) Secretary 53: 2) Team problem Only remaining work is on the research supervision changes the client identified on last Friday's client meeting. Other work is testing, which we are far ahead on already. 3) Team problem Documentation must be completed by Monday 12th May. User Guide (already started), Maintenance Document and Installation Guide - start on these as soon as possible this week.

Tool minutes example 1) Present Work is likely to be slow this week, as all members of the group also need to dedicate a large portion of time towards the HCI assignment, however progress should be made in a number of area. This week we will also focus on getting all of the past information into the team directory. 1.1) Members 32:;64:;34:;554: 2) Story The contact form is to be reimplemented. Chris will provide code for the javascript/css based popup which will be used, which the form will be added to. 2.1) Identifier or creator c: Client 2.2) Story card 2026: Contact Fo... 3) Story This needs to be implemented. 3.1) Identifier or creator 32: 3.2) Story card 3004: Newsletter... 4) Story News is to be changed to be an adaptation of pages rather than it's own specific entity. Mat will make the Page code more robust to allow this to be done. 4.1) Identifier or creator 64: 4.2) Story card 3011: Add News ;3012: Edit News ;3013: Remove New... ;3014: Show News ... ;3015: Show News ... 5) Story The client has decided they would like an upload area and file manager within the partner's zone and admin area. Indi will attempt to address this during the week by using php_file_tree() however this is unlikely to be completed for the next meeting. 5.1) Identifier or creator c: Client 5.2) Story card 3008: Partner Zo... 6) Story Overall, while progress has been slow in area, We feel that we will be able to complete the project with time to spare, allowing us to rework the design after presenting a working model should the client choose so, and provide substantial documentation. 6.1) Identifier or creator 32

Management tool feedback In the first iteration the tool was built into Eclipse, this meant that it took a while to load. We now provide a standalone version. The minute editor was initially a little clumsy. We implemented a word processor like editor. The Genesys students did not see the relationship between the tool and their wiki. Sometimes the tool was slow to run. We were able to optimise the server component which helped a lot. Some students wanted a web based tool.

Test server This was an application that the students submitted their tests to (one of the deliverables). It ran the tests written by the students and returned the results to them. It also made a log of the tests, and when they were made as evidence for testing. Submission was via FTP.

Test server feedback The system proved to be difficult to use, as the students were struggling with the test methodology that we had proposed (Although they had apparently used in previous years!) The server showed that typically teams did not manage to write tests that would run before the final weeks of the project. The students also reported that they found the FTP procedure hard to use. The tool was enhanced this year and we will try to use it again next year.

Wiki mk 2 Sometimes the students found it hard to decide what to put on the standard wiki. On this occasion we used mediaWiki again but added a plugin, which integrates into the management tool. Students can now click a button in the tool and the details of that screen are added to the wiki. Crucially the wiki allows students to annotate the required parts with their own information.

Wiki useage (modify) Week (2007-8) Number of modifications

Wiki mk 2 feedback If was found useful to add extra information, and record things in their own way. Still unclear about how to get started on the wiki. We provided an example wiki, but it maybe worth while pre-populating this info into each teams wiki individually as an outline template.

Conclusions Students are looking for both guidance and freedom to run their projects. Tool support should provide both, guidance through structured forms, freedom through the ability to annotate or provide additional information. Students also want to be rewarded for their work so mark schemes must be realistic in rewarding the time spent ‘running’ a project. By providing this support we have seen the student

The future The idea of networked working, especially when software developers are not in the same place is becoming important. As is the idea of having contact throughout the customers organisation. We are considering extending the tool to support these features.

References Johnson, P. M. (2007). Automated software process and product measurement with Hackystat. Dr. Dobbs Journal . Holcombe, M. and Stratton, A., Fincher, S., Griffiths, G., (eds) (1998). “Projects in the computing curriculum”, Proceedings of the Project98 workshop, Sheffield, Springer. Parker, H.E.D. and Holcombe, W.M.L. (1999 A) “Campus based industrial software projects: risks and rewards”. Manuscript. Parker, H.E.D. and Holcombe, W.M.L. (1999 B) “Keeping our clients happy: myths and management issues in ‘client led; student software projects”, computer science education, 9 (3), pp 230-241,. Thomson, C. and Holcombe, M. (2007 A). 20 years of teaching and 7 years of research: Research when you teach. In 3rd South-East European Workshop on Formal Methods , Thessaloniki, Greece. South-East European Research Centre. Thomson, C. D. (2007 B). Defining and Describing Change Events in Software Development Projects . PhD thesis, Department of Computer Science, University of Sheffield. Thomson, C. and Holcombe, M. (2008). “Correctness of Data Mined from CVS”, In proceedings of 5th Working Conference on Mining Software Repositories, Leipzig, Germany, May 10-11, ACM,

Managing Inquiry-based Learning: Learning from experience

More Related Content

Similar to Managing Inquiry-based Learning: Learning from experience (20)

More from cilass.slideshare (20)

Recently uploaded (20)

Managing Inquiry-based Learning: Learning from experience