Jisc smudie project report 2

Smudie Project Report

Synthesis of Messages
This report covers the outcomes of the second phase of interviews carried out as part of the JISC
Smudie Project. The focus was on the Faculty Management Information Officers and on the Assistant
Deans with responsibility for Quality in the Faculties. Further interviews were carried out with IS and
Registry staff, as well as with two of the Faculty Secretaries, with the Examinations and Assessment
Manager and with the Head of Library and Learning Services.

1. Management Information Officers

The role of the MIO is to oversee the completion and accuracy of student records managed by the
Faculties. They are central to the adoption of QLS V4 by the Faculties and the direct entry of student
assessment information by academic staff. In principle they have no direct input of primary
information, but check that the students are registered on the correct modules and that assessment
outcomes are entered and up to date. Having said that, where changes in marks, module transfers
and student withdrawals are to be made, only the MIOs are authorised to make those changes.

Periodic checks are made (twice or three times per year) of validity of module registration,
assessment outcome entries, student transfers and withdrawals. These are done through the FQS
system which generates programme matrices and are carried out directly with the year tutors.

The management of the system is geared towards the exam boards at the end of the year when the
QLS generates comprehensive spreadsheets of cohort outcomes. The outputs from the system
include comment sheets that record the exam board actions.

Issues:

Academic staff are not all engaging with the QLS V4 records system for a number of reasons:
o there is reluctance/resistance to change to a new system;
o there remains a proportion of staff who lack confidence/competence in the use of
technology;
o the task is seen as administrative as opposed to academic and regarded as a waste
of their time/expertise;
o aspects of the system user interface are regarded as non-intuitive/tedious;
o certain course management information cannot easily be extracted from the system
and staff keep their own local records;
o Part-time staff do not use the system regularly and forget processes and procedures
and need to be re-trained.
The MIO management of the student record, where modules and grades need to be edited,
is non-user friendly, time consuming and frustrating;
Where resit marks are to be entered or students are transferring from one set of modules to
another, this can be only be done by individually deleting the previous records;
The system times out when not in continuous use and this can be very frustrating;
The different ways of reporting marks and grades need reconciling.

Overall comment about MIOs: the system, as a bridge between registry, quality and the faculties is
seen by all stakeholders as a successful arrangement that adds significant confidence to the validity
of quality processes, whilst assisting academic staff in their reporting duties.

The regular MIO group meetings are very beneficial for those lesser experienced than the others.
The group shares experiences and identifies the differences in procedures applied within different
faculties.

A key activity for MIOs is working with academic staff to ensure that the QLS V4 student information
is complete and accurate. This includes checking that students are enrolled on the correct modules,
that their assessment records are up to date and that staff have sufficient familiarity with the system
to reliably input data.

2. Assistant Deans (Quality)

The ADQs are responsible for all aspects of academic quality management in the Faculties. This
includes arrangements for new course proposals and validations, exam arrangements, exam boards,
external examiners and the management of all associated information. It also includes responsibility
for managing relationships with external agencies including QAA, Estyn, HEFCW and the University of
Wales. The ADQs chair their Faculty Academic Quality and Standards Committee, attend the other
FAQSCs, and report to the main AQSC.

Overall the perception is that the online student information management system is a significant
improvement and that its effectiveness will improve as academic staff become more familiar with
its use. There is also a general agreement that the role of the MIOs in the system is central to the
effective maintenance of data accuracy and consistency.

However, the discussions with the ADQs revealed a number of data management issues that need to
be considered. These issues relate to four different aspects of the system:

1. The ease of use of the online student records system;
2. The staff development needs of academic staff responsible for managing student
information on the system;
3. The assumptions in the system structure based on a standard academic year and student
cohort;
4. The fact that, currently, there are four different components of the student information
management system which have yet to be fully integrated:
a. QLS V4
b. FQS
c. ACMS
d. Student Attendance Recording

ADQs do not input data to the student information system in that role, only as teachers in their
subject specialisms. They do, however, retrieve data from the system for exam boards, annual
course monitoring and similar quality assurance activities. The specific issues identified were:

The identification of all part time students as year 9 on FQS – this prevents tracking of part
time student progress;

The Agresso system is set up for a standard academic year which means that any course that
does not follow that pattern requires students to re-enrol if their course straddles the
normal entry and exit points. This is reported as particularly frustrating for overseas
students;
Students on courses that share modules with other courses are difficult to track. Currently, it
is difficult to obtain an individual student record if they are in multiple groups;
Attendance recording is manually input and very time consuming. The CelCat application
was recommended by one ADQ and it is known that options are currently being discussed;
The FQS system, developed by SMU data services and in use for over 10 years, is regarded as
much easier to use than QLS and has been retained as a result. The matrix printouts are
used for periodic checking of student module choice and accuracy of data;
The course coding system is regarded as logical, but a barrier if academic staff (particularly
part-time staff) do not know the codes. This could be linked to the mode of attendance
issue, by including this information in the course code.

An interesting issue was raised regarding disability access to the Agresso system: that the on-screen
colours used were problematic for users with impaired vision and that the company should be
alerted to the issue.

This series of discussions was the first with academic staff at the University and demonstrated the
context difference between them and non-academic support staff. Quite understandably, academic
staff are focussed on the support of learners and the quality of the learning experience in their
particular curriculum area. They recognise the need for the accurate recording of student
information and their role and responsibilities in that process. However it is seen as a ‘necessary
chore’ and separate to their primary role of teaching. For that reason they are looking for a system
that is simple, intuitive and time efficient.

3. Information Systems

Each member of the IS staff has a particular role in the development and maintenance of the
student information management system. The roles are based on experience, technical expertise
and institutional need in terms of, for example, database support, systems development, user
support etc. This is not at all unusual in organisations, but there is a risk of critical capacity loss if
individuals are the sole repository of expertise in a particular systems function. This is no doubt
being managed through appropriate duplication/documentation and, where appropriate, succession
planning.

The IS team works directly with academic, registry and other staff with responsibility for using the
student information management systems. This ranges from basic systems training and user support
for activities such as student enrolment; through to workbench development to improve the user
interfaces with the QLS application. There are several areas where further improvements in usability,
efficiency and effectiveness of the systems have been identified. All of these are known by the IS
team and the staff are working towards solutions within the constraints of team capacity.

This includes working with Unit 4 to develop and improve the Agresso QLS and CMS applications.
There are several aspects of the systems that have potential for improvement, particularly with
regard to intuitive and efficient usability. The company is working with SMU, as well as other

institutional users, on a programme of continuous improvement; but this does give the impression
that the application is in permanent beta mode and that the institutions are de facto test beds. No
doubt this symbiotic relationship is reflected in the license fee negotiations.

The overall strategy for information systems management centres on user responsibility. Where
data is required by the system from a user then it is the user that enters the data. The students self-
enrol, the academic staff enter the assessment results, the programme leader completes the CMS
course specification. The fact that the system is entirely online makes the process particularly
efficient and accessible. The effect is to eliminate the inefficiencies and cost of duplicated data
recording systems. The risk is that, with so many users having an input to the system, data reliability
will be compromised.

Such a risk is addressed with appropriate checks and balances. The MIO arrangements are central to
this, particularly when it comes to ensuring that students are enrolled on the correct modules and
that assessment results have been entered by staff. The on-demand support from the IS team for all
system users is also key to successful operation. This support is available to academic, registry and
other staff as required and also to students, particularly when they self-enrol.

The original Faculty Query System (FQS) that was used before the adoption of the Agresso QLS, is
expected to be abandoned once the new system is fully functional and stable. This has not happened
because there is certain FQS functionality that is valued by staff and cannot yet be delivered by QLS.
An example is the cohort matrices provided by FQS that are used periodically to check that students
are enrolled on the correct modules and that all information is complete. There are also varying
degrees of use by staff of local record keeping, both paper based and digital, mainly for personal
assurance that a backup exists should there be a systems problem or a query about data validity.
Such local records are also used for QLS held student data that is regarded as difficult to access.

The main issues and areas for development currently being addressed by the IS team include:

Workbench development for the retrieval of information from QLS including the use of the
Crystal reporting system. Current workbenches include interaction with UCAS, SLC, HESA and
UKBA;

Development and testing of the Agresso Curriculum Mangement System (ACMS) for
curriculum design and validation. The definitive documents delivered would interface with
QLS;

Planning for an improved student attendance monitoring and recording system, possibly using
the CelCat application. The most efficient and reliable method of recording attendance data is
the present focus.

Planning for the online enrolment of part-time and international students

Other tasks involving student information management include the provision of exam registers, the
setting up of assessment parameters and the merging of student information when duplicate
records have been entered on the system.

4. Faculty Secretaries

The Faculty secretaries manage the Faculty office(s) and typically have administrative assistance.
They do not have a direct data input role to the student record system but do retrieve information
from it in their daily activities.

Interaction with students, and hence involvement in the management of student information
includes:

Receiving assignments submitted by students, issuing of receipts, logging of submissions
(typically on a local spreadsheet record) and the distribution of assignments to the
appropriate tutors;
Receiving student application forms from Registry, distribution in paper form to the
programme leaders and emailing students to confirm receipt of their application;
Management of the Faculty student file system with paper records for each student that
contain all relevant documents including letters, medical records, disciplinary records,
withdrawal, transfer and suspension of studies forms etc;
Sending of referral letters to students when (1) the need is indicated by the QLS V4 system
and (2) after checking with the course tutor. Record of letters sent are typically kept on local
spreadsheet;
Assistance in student attendance monitoring, typically by receiving monitoring forms from
lecturers and entering on a master attendance register. Registers are to be handed in to
Faculty Secretaries at the end of the academic year and kept for a minimum of two years;
Assistance in Exam Board preparation in collaboration with the Faculty MIO. This includes
the preparation of results to be presented to the Board and the recording of meeting notes
(one Faculty secretary also provides meeting minutes);
Responding to student enquiries about such things as open days and UCAS applications, and
the provision of general advice and guidance;

The general issues relating to the student information management system identified by the Faculty
secretaries are similar to those raised by the other stakeholders:

The tedious multiple window operation of the V4 system when accessing data, and the fact
that it times out when left for a few minutes;
Changes in FQS (which is generally regarded as a useful application) that have altered its
functionality. These have included the misleading use of the ‘year 9’ identifier for part-time
courses and the fact that it no longer remembers where the user was when they last
accessed the system;

Suggestions that would improve the way Faculty secretaries manage student information included
the establishment of a centralised student attendance monitoring/recording system and the auto-
population of student addresses on letters to be sent (though I suspect this can already be done).

5. Library and Examination Management

The library system at SMU uses the Capita/Talis application and student core data is included when
the students enrol and receive a library account. The system records all details of library borrowings
and usage and communicates with students by email about overdue loans and reservations.

The Examinations and Assessment Manager accesses the student information system to identify
students with special needs in order to make provision for those needs. A local spreadsheet is
created which contains all the information needed to cater for the special needs students currently
enrolled (around 160 students in 2011-2012). The Examinations Manager liaises with student
services who determine the kind of support needed and appropriate provision is made. All students
with special needs are then informed of the arrangements for their particular requirements.

The Examinations and Assessment Manager coordinates exam timetabling and organises the
transfer of exam board outcomes to the University of Wales for the issuing of awards.

6. Summary, Conclusions and Planning

This phase of the project involved evaluating the roles of staff who have a significant impact on the
student information management system at SMU, but who do not have primary responsibility for
entering core student data. The roles of the MIOs, the ADQs and the IS staff are mainly to ensure
that the recording system is fit for purpose; that the data being entered is accurate and complete;
that access to information meets the needs of the university systems; and that staff and students
who are responsible for entering primary data are adequately supported.

A key message from this exercise, which supports the conclusion drawn in phase one, is that the
strategy of developing an online student information management system where primary data is
entered by the academic staff and students who have ownership of that data, is entirely logical and
achievable when supported in this way.

Having said that, the system is still in development both in terms of its implementation at SMU and
as a fully functional application from Unit4. This report has identified a number of areas for
improvement that would make the system more usable, efficient and effective.

It should be said that the SMU IS team are aware of all the issues and are working with staff at the
University and at Unit4 to create solutions. However, it is hoped that the report will help in
prioritising the actions to be taken in moving to fully functional and integrated student information
system.

The main issues for consideration include:

The usability of the QLS V4 application. There has been a consistent message from all users
that navigation and data editing in the system is frustratingly inefficient. All users have also
commented negatively on the time-out feature.

Recommendation: discussions should continue with Unit4 about system usability with
specific reference to these user comments. A more detailed analysis of the editing
functionality should be carried out and improvement proposals documented. A time-out
option that allows a direct return to the previous window would seem an obvious
development.

The continued use of the FQS application. The stated reason for this is that it is perceived to
provide a service that QLS V4 does not; particularly the periodic production of cohort
matrices for quality checking.

Recommendation: a detailed analysis be carried out of the ability for QLS V4 to deliver the
same services as FQS (which I expect has already been carried out). Based on the outcomes,
one of two options can be considered; (1) the QLS V4 functionality be adopted/developed to
deliver the same services, or (2) FQS be developed with an interface to QLS (workbench?) so
that it becomes an integrated sub-system using the same data.

A further recommendation is that key system users are included in the development
planning communications loop so they have an opportunity to comment on the impact of
any changes on their interaction with the system.

The alignment of the system with the standard academic year. If SMU plans to become an
agile responsive HE institution with flexible attendance patterns, then the student
information management system needs to reflect that. The present system is reported to be
aligned to the traditional academic year for full-time students.

Recommendation: the analysis of QLS V4 suggested above should include a consideration of
its capacity to support a flexible non-traditional curriculum structure. It seems unlikely that a
major vendor would not be able to deliver such flexibility.

The student attendance monitoring system. It is known that this has been a matter of
debate for some time. Different Faculties (and even programmes) manage it in different
ways and a common system that is time efficient would clearly be of benefit.

Recommendation: That the discussions already underway come to a speedy conclusion and
that they include consultation with the lecturers and students who will be responsible for its
implementation.

Phase three of the Smudie project will continue with the final stakeholder consultations: course
admissions tutors, lecturers and the students themselves. Where necessary, repeat discussions with
previously interviewed stakeholders will be undertaken to clarify aspects of the information
gathered.

At the end of the phase the project will have been in operation for 6 months and will be delivering
an ‘as is’ model of the student information management system, together with a consolidated set of
development recommendations based on this information.

This draft report will be presented at the Smudie project steering group meeting at Swansea
Metropolitan University on Friday 30th March 2012.

Tony Toole
March 2012

Jisc smudie project report 2

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Jisc smudie project report 2 (20)

More from Tony Toole (20)

Recently uploaded (20)

Jisc smudie project report 2