Are we asking for student feedback too late?

Policy changes mean that universities are having to take a more robust approach to course and modular evaluation – with the majority of institutions capturing student feedback at the end of semester, and many through standardised systems which allow for effective analysis, benchmarking and continuous improvement.

However, as we move into the world of TEF which will assess universities on measures such as student retention and student satisfaction, the discussion I am increasingly having with institutions – in the UK and internationally – is whether ‘just’ seeking student feedback at the end of a module is enough going forward.

Students rarely see impact with end-of-semester evaluations

The issue with end-of-semester modular evaluation is that students rarely see the impact of that feedback themselves (any benefit is usually reserved for the next cohort of students) and for lecturers it doesn’t enable quick and effective analysis of student concerns. This is so important for implementing change – and in turn helping address other pressure points facing universities e.g. National Student Survey scores (increasing student satisfaction) and degree completion rates (reducing student drop-out).

There are two core issues in play here. One: how do you enable prompt action to address students’ issues and concerns prior to traditional end-of-semester evaluations (and before they complete their NSS scores or communicate negative views and feelings via online forums)? Two: how do you engage ‘quieter’ students who may not be comfortable asking questions or sharing concerns – especially publicly – and therefore may be unhappy or thinking of dropping out of university altogether?

At present, the vast majority of universities run modular evaluation at the end of a semester which captures all the student experience through a module, and however a student is feeling at that time is reflected in the survey and ultimately (because their concerns have not been dealt with) in the NSS. Some institutions have moved to mid-term evaluation, but the data from that only gives insight on half the student experience and the organisation misses out on the later insight. So my question is: are we asking for student feedback too late in the cycle?

Of course, there are various ways of gathering feedback during a module. Students can give their feedback in-class or in a one-to-one when asked to do so – but in my experience many don’t, because it is not anonymous. There are staff-student liaison meetings where course reps have the opportunity to feed back issues and concerns – but in my experience wider student engagement with course reps is often poor. And then there are VLE chat rooms. But there is not one ‘anonymous’ place, one fully integrated into a VLE and mobile friendly, that enables lecturers to ask students questions and follow up on the responses, which can then be reviewed as part of their end of term summary.

Formative evaluations offer a way to communicate before end-of-semester

This is where our own Bluepulse platform comes. Under the principle of formative evaluation, lecturers have a way to communicate with students to gain feedback prior to end-of-semester evaluation. They can ask questions at any time, and students can give feedback at any time. It enables lecturers to build teaching strategies that receive a welcome response from students and, because this is done in collaboration with the students, increases participation and engagement. Anecdotally, institutions using Bluepulse report greater uptake of end-of-semester evaluation and more positive feedback.

The reality is that universities need to consider formative evaluation (which gives lecturers the opportunity to seek feedback through bespoke, non-standard questions, during a module) and end-of-semester evaluation (which provides standardisation on questions enabling comparisons across the institution) – they are complementary services. No one university, in my experience, has mastered this yet – and this is a missed opportunity to improve crucial measures on student satisfaction and retention.

Culturally, there is a question mark over whether UK Higher Education is ready for formative evaluation. End-of-semester evaluation is a standard process – if a university is not doing it they are behind – and indeed many are playing catch up. So this next step may seem a bridge too far. Part of the issue is that senior management teams generally don’t yet know how to require people to use formative evaluation, but it can be for projects to support first-year student retention, help new teachers to self-evaluate, and be a general tool for teaching and management. This should be the direction of travel.

Why? Because alongside the TEF discussion, the 2017 NSS is asking students about the opportunity to give feedback and how their feedback is acted on. And we can help with both challenges. Through our Bluepulse service we engage in feedback and the evaluation of teaching during a module. And our Blue product is used for end of term and mid-term evaluations and provides a huge amount of quantitative, qualitative and demographic data.

John Atherton is the General Manager, Europe at Explorance

In practice University of Aberdeen

Martin Baker, Senior Lecturer (Scholarship) in the University of Aberdeen’s School of Biological Sciences, outlines his own experiences of formative feedback on assessments and formative evaluation of the modular experience.

How long have you been implementing formative approaches to modular feedback?

“I’ve been interested in formative feedback and, in particular, closing the feedback loop for a few years and had a paper on ‘iterative feedback’ published in 2014. For the last two years I have invited informal feedback from students in response to my own feedback on coursework assessments. Sometimes, students respond with really perceptive comments. Mostly, in this informal, optional arrangement students have not really followed up. But, starting recently on one course (module), I have embedded student feedback into assessments. Students have an opportunity to gain extra credit on their work based on the feedback they send back to me. This ‘iterative feedback’ is an additional grading criterion, and has its own mark. My rule is that this grade can have either a positive effect, or (if only perfunctory) a neutral effect, on the previous, provisional grade. In other words, students have nothing to lose from providing this feedback, and they have an incentive for providing it.”

What are the benefits of formative approaches?

“Formative feedback is important because students cannot learn in isolation. Weaker students usually need to know what they can do better, and what to focus on in their learning. Stronger students often need to have the quality of their work validated and confirmed. Probably all students need encouragement and guidance both during and after submission of coursework. Formative evaluation can provide a sort of roadmap for the development and consolidation of students’ understanding. If you want to use the roadmap analogy, formative feedback can provide way-markers. Formative feedback can also provide a reference point, so that students gain a better understanding of how their work compares with what is possible or expected at their stage of academic development. In addition, closing the feedback loop like we do at the University of Aberdeen gives me a chance to see how (or even whether) my comments are being understood. From this, I conclude that many academics are likely to be interested in raising the quality and utility of their feedback.”

How can this help universities with issues around student satisfaction and retention?

“For students who are feeling unengaged or disconnected with their university experience, there is a risk that their voice is not heard. During critical periods of transition (including entry into a university), there is a chance vulnerable students feel ignored. This could easily have a negative effect on their satisfaction and, in acute cases, even retention. Listening to students can often be really rewarding for academics, and is a good idea anyway. Hearing from students whose voices are quiet or muted could be critically important for those students. But even high-achieving and confident students need to feel they are being heard by academics, as well as support staff. It is probably no exaggeration to conclude that formative feedback to and from students can help to raise student satisfaction and retention. We should use formative feedback as a respectful and dynamic dialogue between ourselves and our students. For all involved, it is not unreasonable to suggest formative can enhance a sense of purpose and belonging within the academic community.”

As a sector, do we generally ask for student feedback too late in the cycle?

“Feedback from students can often be more useful and dynamic when it is formative. If we know about the student experience while the course is still running we can do any mid-course corrections. Otherwise, we are unlikely to have a full understanding of students’ experience of a course. This is probably the case even for a well-established course, since cohorts respond differently every academic year. Also, among each cohort of increasingly diverse students, there are likely to be some individuals who are looking for opportunities to provide formative feedback. But there is a risk that we might not hear from students who are particularly reticent or respectful. So, yes, end-of-course surveys and the NSS may be too late if they are the only or primary way in which we capture the student voice. There is still an important role for such summative feedback, but it is more likely to benefit the next cohort rather than the one that provide it. So timing, relevance and incentives are all issues that we should all consider.”

What has been the impact of implementing ongoing modular evaluation on your course/in your institution more widely?

“At the University of Aberdeen, academics and students, in partnership, spend a lot of time considering how we use feedback and whether there are ways in which we can improve it. The impact of good feedback to students can be seen in good feedback from students. We get student feedback in satisfaction surveys (including the NSS), end-of-course reports, staff-student meetings, iterative feedback, and informal contact with students. We know when feedback is working, because we establish a dialogue with students. And we know what/how they are thinking. For me, that is why Explorance’s Bluepulse has so much potential in helping students and staff understand each other.”