Bradford background

1 Our initial impetus
The University started to revise its learning teaching and assessment (LTA) strategy to include e-learning as a major component in 2004. This revision was supported by a series of consultation events, including a special workshop on e-learning and an investigation into levels of student satisfaction with the University VLE (Blackboard). Levels of activity on the VLE have grown considerably year on year and levels of staffing and technical support have been increased. Every one of the University’s academic Schools has now embedded Blackboard as an essential aspect of course and module provision. The VLE is now seen as ‘mission-critical’ and is high up our risk assessment list. In 2006, we felt that we should more formally review the expansion of e-learning and the opportunity to benchmark seemed a good way forward as it offered both consultancy support and the exchange of ideas and experience with other HE institutions.

2 How to do it?
We decided against a very formal steering group structure as the existing committee structure of the University gave sufficient avenues to explore and agree the criteria. A small core group will drive the project and this group will expand during its life span. The papers and outputs from this group will be fed into the Electronic and Distance Learning (EDL) working group (which then sends agreed papers on to our Learning and Teaching Sub-committee – LTSC – and thence to the Academic Policy Committee – APC), the E-Strategy board (which is responsible for managing the University’s e- strategy and component projects) and the informal meetings of the Associate Deans Learning and Teaching (which allows for an informed discussion of issues before papers reach the formal committee stage). This means that the papers produced by the core group are thoroughly reviewed both from the learning and teaching aspects and from the technical and structural aspects.

3 Changing roles
Our project management was initially undertaken by Carol Higgison from TQEG . She has now become involved in a further project for JISC and so Professor Peter Hartley (Head of TQEG) has taken over.

4 Start-up meeting

The benchmarking start-up meeting was attended by Peter Hartley, Nigel Robertson, and John Purvis, Professor of Pharmaceutical Education. We felt that this meeting gave us very useful background in context but were a bit disappointed that there was only limited time during the event to start our discussions with our other partners.

5 Initial work on criteria
We borrowed the list of initial criteria which Glamorgan had used and distributed this to the three groups mentioned above as a way of starting debate within the University.

6 First partner meeting
Our first partner meeting at Glamorgan gave us the opportunity to discuss some of our issues with our partner institutions and hear of their experiences plus the input from Strathclyde. For us, it reinforced the need to pay particular attention to the generation of appropriate criteria.

7 First meeting at Bradford with consultant
Our first formal meeting here at Bradford with Professor Paul Bacsich included a meeting with the PVC Learning and Teaching and the PVC E-Systems. This was a useful discussion as it both clarified the nature of the exercise and confirmed the University’s commitment.

8 And more criteria
Once we received the more refined list of criteria supplied by Glamorgan (and thanks again for their support and encouragement), we discussed these with the Associate Deans and made them available to the other two working groups. Two particular areas emerged from discussion which did not seem to be fully covered in the initial list of criteria: the evaluation of impact of e-learning, and the need to define an appropriate blend of online and face-to-face contact in many of our courses and programs.

9 Further consultancy support
Further interaction with Paul has included a telephone conference to confirm
progress and future dates and his e-mail which included a very useful review of e- benchmarks. We used this e-mail to prepare version 3 of the benchmarking which has now been discussed by the EDL group. A revised set of criteria based on this discussion will be published through this blog by December 18.

Getting to the start

The eLearning Benchmarking project was announced by JISC and the HEA in the second half of 2005. The project was designed in 3 stages – an initial pilot with 12 institutions and then a Phase 1 and Phase 2 with significantly increased numbers of participants. One of the main aims of the pilot was to identify and narrow down the potential benchmarking methodologies. The current Phase 1 institutions are now using one of four methodologies to benchmark their eLearning provision. At the same time they are attempting to define the methodologies in terms of eLearning in HE & FE. In the final Phase 2, institutions will use the results from the two earlier stages to assist in refining the methodologies within these terms through application to their local context.

The University of Bradford submitted an application to participate in the project and was accepted for Phase 1. We attended a Town Hall meeting where an overview of the Phase was given along with details of the available methodologies. Following this, we had a workshop meeting to identify which of the four possible methodolgies would be most appropriate to use in our context. We had further deliberations and discussions with stakeholders and made the decision to use the MIT90 methodology. On the surface, this may be seen as one of the more difficult methodologies to apply. Unlike the other main methodologies available, MIT90 doesn’t come with a set of criteria against which an institution can measure itself. Instead, it offers a framework which is used to provide a structure to the processes, resources and influences of, on, and within the body being examined. Besides being used as a tool to examine existing provisions, MIT90 can in fact be used to design provision. Wills (2006) gives a current account of this type of use in Australia.

A key reason for choosing MIT90 to use at UoBradford was the fit with our balanced scorecard system. We already use a balanced scorecard to measure many functions and major projects at the university. eLearning (in its broadest terms) is a significant component of several major strategies and MIT90 gives us the opportunity to align our benchmark outcomes with existing measurements on the scorecard.

September 2006 saw us attending the HEA ‘kick-off’ meeting in Manchester along with the other Phase 1 institutions. Although many useful things were covered at the day long meeting, most seemed to cover background and context. Along with some other MIT90 users, we felt that we never really got to the meat of the subject during the day. Consequently, the MIT90 cohort arranged to hold a further meeting at the University of Glamorgan.

We met in Glamorgan in October with our three partner institutions and the consultants retained by the HEA for a productive and supportive event. Michael Coen from Strathclyde University also attended to give an account of their use of MIT90 in the initial pilot stage of the project. Since then we have been redrafting, refining and disseminating internally our initial criteria candidates. Later posts here will cover this process and its outcomes.