Transforming Student Feedback Analysis at the University of Manchester: Leveraging Explorance MLY for Scalable Qualitative Insights

Challenge – Analysing Qualitative Feedback at Scale

The University of Manchester, as one of the biggest universities in the UK, struggled to resource qualitative analysis of student feedback. It did not have the platform to undertake such analysis of free-text comments in its surveys.

“Survey administration is a huge challenge given the size of the University, a single campus institution spanning three faculties and nine schools, and our student numbers,” explained Teaching and Learning Officer (Student Voice and Feedback) Daniel Bayes. “For example, we have over 50,000 comments from a single semester-end course-unit survey project, drawn from 138,000 individual survey instances, so one of the big things for us is the scale of what we get in, and what we can actually do with that.

Previously, the only way the University of Manchester conducted qualitative analysis of student feedback was through its strategic planning office, which handled comments from the National Student Survey (NSS). This process involved using Power BI to categorize comments. However, due to the University’s devolved structure, individual subject areas were then required to perform manual analysis on their own from that point forward.

Solution – Closing the Feedback Loop and Enhancing Institutional Insight

In the 2023-24 academic year, the University invested in Explorance MLY as the Russell Group institution stepped up its work to capture and respond to student voice.

“We have a combination of institutional questions and school-level questions, so each of the nine schools can ask up to five questions and some of those schools do ask five open comment questions, so we do have a lot of free-text responses,” Daniel said. It was the first way the University could break those down by category.

“Our ability to look at themes and common topics, both across the whole data set and to filter down just to one school’s question set, is really important because the context of that school may be very unique. We can then deep dive into institutional interests and research questions. The alerts feature is also really useful for us, again with the size of the institution and scale of our surveys, as it gives us an indication of where to start our response. This does not mean we necessarily capture everything that we should be looking at, but the value of MLY is it then suggests what should be reviewed to begin with.

“The recommendations MLY produces are also really good for academic staff, as sometimes you get comment analysis to them that is not necessarily presented in a way that they can easily action. For MLY to be saying ‘you should do more or less with X or Y, or continue with Z’, even if it is only the starting point for the conversation that staff want to have around it, it puts it into a format and a language model that is very clear. Finally, through the ease of filtered sharing, we have the ability to put all the data in but then just filter by school and provide that data for the school to work with. This saves me weeks of work.”

Additionally, through a separate consultancy support agreement, the University has utilized Explorance experts to undertake comparative data analysis and going forward intends to use this support to measure the institution’s strategic goals.

Outcome – Transformative Insights for Continuous Improvement

The University is successfully analyzing thousands of free-text comments through MLY.

“From where we were, to where we are now, has been a complete overhaul,” Daniel revealed. “Even with the number of comments we put in, 50,000 in one go, I am still baffled by how quickly MLY does the analysis once you have got it into the system to then being able to look at it within a matter of hours.” For the first time, the University can compare and contrast insights from different data sources with MLY, and leverage its recommendations to frame discussions in planning and review meetings based on the feedback received.

Another major impact for the University is “how fast we can start getting analysis to people, even if it is top-level”, according to Janine Holdway, Head of Teaching and Learning Delivery.

“MLY is supporting continuous improvement through the speed at which we can look at things, which means we can take action a lot quicker,” she said. “Rather than feedback coming at the end of the semester, that then goes to a panel in January between semesters and then we think about it for the following academic year; you can do a mid-semester survey, send the analysis for review to the module lead the next week, and if they want to action that immediately they can.”