Streamlined Surveys at UNSW Sydney Help Increase Response Rates for More Robust Data and Analysis

UNSW Sydney wants to establish itself as Australia’s global university. And thanks in large part to their new strategic plan, they’re making big strides. Since the plan was first developed and implemented in 2015, they have moved up significantly in world rankings, among other notable achievements. The ARTU ranking, which aggregates results from the Times Higher Education World University Rankings (THE), Academic Ranking of World Universities (ARWU) and QS World University Rankings, recently listed them in the top 50, four years ahead of their target goal of a top 50 ranking by 2025.

Integrating The Student Voice

Integrating the student experience throughout the strategy was a priority in the 2025 Strategy Update. This was followed by the release of the Education & Student Experience strategy called “Listening, Challenging, Supporting”. The new strategy places ‘Acting on Insight’ as the first strategic theme and ‘taking surveys seriously’ as the first initiative stating that ‘Listening to students and staff, and acting on their feedback, are essential prerequisites for the realization of a fulfilling university experience’. To drive forward these strategic priorities, they needed to ensure the right tools were in place to capture the student voice.

“The Education & Student Experience strategy is focused on students and their experiences, and it pushes us to look at the university from the students’ perspective,” said Keelin George, Head, Insights & Engagement in the Office of the Pro Vice-Chancellor (Education & Student Experience) at UNSW. “In order to be able to act on insights from student experience survey outcomes and use the data to inform strategic decisions, it’s important to achieve high response rates.”

To maximize response rates, the university undertook a number of initiatives, including refining processes for both internal and external communications, as well as making some technological changes to improve the student customer experience for surveys.

Blue Experience Management Platform

UNSW has been using Blue online Course Evaluations since 2017, to replace the paper forms they formerly used to gather student feedback. The recent drive to improve the student survey experience became a perfect opportunity to check in with Explorance and see how they could improve it, with the goal of raising response rates.

Strategies For Raising Response Rates

To begin with, the faculties put a significant effort into creating communications materials and informing students of the value of the student surveys, letting them know that their responses were indeed read and what happens with them. The central survey team and faculty staff also sent personalized communications to course authorities, using the Blue dashboard to monitor response rates, and target large courses with low response rates.

The survey management team also reviewed and streamlined survey processes. In 2017, the first year of using Blue, students were asked to complete one survey for each course, and a separate teaching survey for each member of staff that taught them for each lecture, tutorial, lab, studio, or seminar they attended as part of the course. This increased the number of survey forms students received, adversely impacting response rates. “Students could receive anywhere from six to eight links to surveys every semester,” said Paul Keitley, Manager, Student Survey Systems for the Pro Vice-Chancellor, Education & Student Experience Portfolio.

To address this problem, the university added a list of all the teaching staff associated with a course to each course and teaching evaluation form. The student would then self-select staff that taught them. This helped improve response rates, although the solution was sometimes unreliable. Although photos of teaching staff were displayed to help students select the correct staff member, students would still sometimes make an incorrect selection. Keitley would then have to manually fix these errors and re-issue reports.

Streamlining Surveys With The Right Technology

The survey management team met with Explorance to discuss how best to tackle these issues. In 2019, Explorance implemented a feature whereby each course evaluation form would be pre-populated with teaching staff assigned to classes students were enrolled in. This reduced the number of errors in survey feedback and improved the quality of the data received. “So far, it’s been a great solution for us,” said Keitley. “Students no longer need to go through a long list to select their teachers.”

In 2021, the university also integrated Blue with their Moodle Learning Management System (LMS), so that in addition to receiving emails, students would receive a pop-up reminder whenever they logged into Moodle. Only outstanding forms appear in the pop-up reminders.

Raising Response Rates

All these efforts contributed a significant lift to response rates on course evaluations. When UNSW first introduced a combined form for students in 2019, which contained the links to each students’ tutorials, they went from 43.7 to 45.9% in response rates.

By 2021, after they implemented the pop-up reminders in Moodle, the university truly became a leader in the field, achieving over 50% in their response rates. “Response rates went to 50.4% in Term 1 and 51.6% in Term 2,” says Keitley. “We’re seeing a good five percentage points higher in response rates with the new changes in Blue.”

“Response rates went to 50.4 percent in Term 1 and 51.6% in Term 2,”

Paul Keitley, Manager, Student Survey Systems for the Pro Vice-Chancellor, Education & Student Experience Portfolio.

This is a remarkable achievement on its own, but doubly so given the university is in the middle of reponding to a global pandemic.

Conducting in-depth Analysis

The Student Surveys Team manages gathering feedback for a large university – over 59,000 students – with an astonishingly lean staff. Keitley oversees the project management, IT, day-to-day communications with faculties and schools, and report generation on Blue for the entire university, while George oversees the data analysis, strategic development and implementation.

Among the Blue tools that assist Keitley in managing his work is Blue Text Analytics, which he uses to conduct qualitative data analysis. He screens the data regularly for any comments that require an urgent response, such as mentions of self-harm, depression, aggression, etc. For the final screening, he checks for inappropriate comments, such as comments on appearance, and conducts the necessary edits before sending them on to faculty and staff for their review.

Nothing Compares to Blue

As a publicly funded institution, UNSW Sydney is accountable to the public for its expenditures. They have a responsibility to ensure they are spending their IT budgets wisely, and Blue falls under this scrutiny. “We are obligated to conduct periodic reviews of our systems to ensure we aren’t overspending on costs,” says Keitley.

“We recently conducted another review of all the course evaluation systems available, and we’ve found again that there is nothing else out there that can do everything that Blue does. You can find some that do bits of one and bits of another, but then you’d have to mash them together, and do all the work to make sure they work together. In our view, there really is no better course evaluation platform out there,” he concludes.

In our view, there really is no better course evaluation platform out there

Paul Keitley, Manager, Student Survey Systems for the Pro Vice-Chancellor, Education & Student Experience Portfolio.