24.04.07

Outcomes of the HEFCE consultation on developing the TQI/NSS

1. Introduction

In November 2006, following on from HEFCEs update on Teaching Quality Information (TQI) and the National Student Survey (NSS) in July 2006, HEFCE encouraged responses to a consultation exercise on the development of TQI/NSS.

The 1994 Group Office asked member institutions for their responses to the consultation exercise, which were then submitted as a collective group response.

This information note will analyse and compare the responses collated from 1994 Group members and the outcomes produced by HEFCE of the consultation on developing the TQI/NSS. The full report can be downloaded via the HEFCE website - Annex A: Outcomes of the consultation on developing the TQI/NSS [doc 87k]

1.1 Consultation responses

HEFCE received a total of 118 responses to the consultation:

2. Outcomes of the Consultation

2.1 Question 1 Do you agree that the data should be presented at level 3 in the subject hierarchy, wherever possible?

(i) The 1994 Group agreed with this statement. Members consider that a more detailed JACS level will enable more meaningful information to be available to the audience.

(ii) 89% of the 118 responses HEFCE received supported the proposal. It was recognised that data at level 3 would provide more specific information, closer to course level.

2.2 Question 2 Do you agree that the threshold for publishing NSS results should be lowered to a 40% response rate, and 20 students responding?

(i) The 1994 Group's response is echoed by the majority of respondents to the HEFCE consultation. 76% of the consultation respondents supported the proposal to reduce the threshold for publishing NSS results to a 40% response rate, and 20% students responding (from the current minimum of 50% and 30 % respectively).

(ii) A number of respondents suggested a 50% response rate and 20 students threshold, on the basis that it would still require the majority of students to have informed the data and therefore would prevent a minority view prevailing.

(iii) The steering group have considered the options and are seeking further advice from a group of statisticians. The result will be published shortly.

(iv) A sizeable number of respondents suggested that providing the sample size and response rates for the NSS would aid users with interpreting data on the site. Many also suggested that if the Higher Education Statistics Agency (HESA) data were reported in percentages, the cohort size would provide important information for users.

(v) The steering group agreed with these suggestions, and subject to user testing, will implement the changes.

2.3 Question 3 Do you agree with the principle that data should be aggregated across years and/or subjects, where it is unavailable at level 3 using one year's data? Are there any options for aggregating data that should not be offered to uses?

(i) The 1994 Group took the view that whilst we would like all subjects to be represented, there was a concern that aggregation might produce distorted information. We believe that if this proposal were to be implemented, it is essential that the methodology of aggregation to be made transparent to users.

(ii) On the question of aggregation across subjects, HEFCE received mixed comments. Only 33% agreed, subject to user testing, that data should be aggregated across subjects rather than years. A further 29% agreed that data should be aggregated across subjects and years.

(iii) The steering group took into account these views and decided that, as there was no clear overall preference, various options will be developed and tested with potential users.

(iv) On the question of aggregation of data across years, there was concern amongst some respondents that aggregation over a long period of time might conceal changes in the quality of the provision. It was suggested that a maximum of two years' worth of data should be aggregated.

(v) The steering Group agreed with this concern, and have decided that where used, aggregation of data should occur over a maximum of two years.

2.4 Question 4 Do you have any comments about the suitability of the proposed items for a summary of key data?

(i) Despite raising a few concerns, the 1994 Group generally agreed with the proposed items for a summary of key data.

(ii) HEFCE received support from 64% of respondents towards this proposal. Many felt it would be useful to provide a distribution of responses to the question of overall satisfaction with the course. This view is supported by the steering group, who intend to develop for implementation during the summer of 2007.

2.5 Question 5 Do you have any comments about the proposed student profile data?

(i) The 1994 Group expressed full support of the proposed student profile data. Subject to user testing, 61% of respondents shared the support expressed by the 1994 Group.

(ii) A number of respondents to the consultation suggested that further data about the ethnicity of a student, the proportion of students with disabilities, the percentages of students from low participation areas and from the numbers of student from state school, should be included.

(iii) The steering group is consulting with the Equality Challenge Unit, having expressed concern as to how this information could enhance student choice.

2.6 Question 6 Do you have any views on whether the NSS or DLHE surveys should, in principle, be extended to include directly funded higher education (HE) students in further education colleges (FECs)?

(i) The 1994 Group had no additional view on this proposal. Of those who did respond to the consultation, 64% (76 respondents, of which 25 were from FECs) were in support. Many felt it would aid potential students in their choice of where and what to study by giving them a clearer picture of HE in FECs.

(ii) The steering group shared a similar view and have agreed to keep the sector informed of further developments.

2.7 Question 7 Do you have any comments on the options to present franchised students with the FEC's data, and also make it available with the HEI's data?

(i) The 1994 Group did not add any further comments. The response to the proposal was complex. 17% (20 respondents, 9 of which were from FECs) supported option 'a', whilst 31% (37 respondents, of which 4 were from FECs) agreed with option 'b'.

(ii) In light of the views sought through the consultation and from the steering group, HEFCE has decided to delay the implementation of presenting data on HE in FECs and franchised students until 2008. HEFCE believes the proposal requires further consideration to ensure it resembles and is as robust as the HESA data form HEIs.

(ii) The issue of how other collaborative arrangements, such as the Peninsula Medical School can be represented on the TQI website was raised through the consultation and throughout the sector. Presently, students make their applications directly to these partnerships, but are not represented on the site. HEFCE is to consider how this issue can be addressed.

2.8 Question 8 Do you have any comments on the proposed approach for institutions to provide commentaries on the data?

(i) The 1994 Group did not comment. 53% of respondents from the HEFCE consultation (63 respondents, 20 of whom were from FECs) agreed that FECs should be able to provide commentaries on the data about their HE provision, to be included on the TQI site.

(ii) Due to the complexities of collaborative arrangements, the steering group recommended that further work is conducted in order to develop a protocol for consultation with partner institutions on the content of commentaries on data about HE in FECs.

3. Conclusion

Overall it appears that the majority of respondents to the HEFCE consultation shared similar views to those of the 1994 Group. Many of the concerns raised by the 1994 Group in response to the proposals appear to have been shared by respondents and in response addressed by HEFCE. This demonstrates the important role the 1994 Group has in shaping and developing areas of Government Higher Education policy.

Rachel Craine
1994 Group
27/04/07


back to news archive index