Automating Course Evaluation and Facilitating Data Integration at the University of St.Gallen (HSG)

From manually-intensive process to complete automation

The University of St.Gallen (HSG) is an elite teaching and research university that specialises in management, economics, law, social sciences, international affairs and computer science.

The HSG offers over 800 undergraduate and postgraduate level courses every semester and, via its over 30 institutes and research institutes, a complete executive education portfolio. The institution holds “triple crown” accreditation (AACSB, AMBA and Equis) and consistently ranks as one of the world’s leading business schools.

Following a successful pilot in 2018, the University chose Explorance to manage its course evaluation and survey processes institution-wide. The Blue People Insight Platform particularly impressed with its ability to automate data collection, analysis and report distribution. “We are responsible for setting up evaluations, and our decentralised process gives lecturers the platform to introduce them at the appropriate time for their individual courses,” said Quality Development Expert, Kristin Huber.

“Previously we used paper-based surveys. The required data was all collected through spreadsheets, which was a manual, labour intensive process. Blue has provided complete automation of data running from our student information system, Un.IT, which is integrated with our learning management system, Canvas. Initially there were lots of instructor questions coming back, but over time these have become less, so the amount of follow-up we have had to do has decreased. The number of voluntary evaluations has risen every semester – this is a sign of increased buy-in.”

Increasing speed of reports to instructors and programme managers

Instructors see evaluation reports within 24 hours of five or more student responses being received. Previously programme managers only received data at the end of every semester once the data had been collected and processed manually. However, with results being used for course improvement, instructor feedback and overall academic development, the speedy availability of reports for programme managers is critical.

Kristin explained: “Evaluations are one criterion for whether a course gets offered again or not, and in what format. For example, if a course has a low score for online implementation in a hybrid course, the University will work with instructors to improve that aspect. The evaluation results are also used to inform whether contract lecturers are employed to run a course again and full professors are re-elected. If programme managers do not get the data soon enough to inform decisions the whole process is slowed right down.”

But with strong data protection rules in Switzerland, and transparency of distribution being a key aspect of the University’s own quality development guidelines, every instructor must be informed regarding who can view their course results.

New institutional hierarchy for insight-sharing

“We are moving from a situation where we had lots of evaluations; the programme managers would never see the results in real time, and even if there was a problem that was reported, they could not really investigate student feedback,” Kristin revealed. “This was a problem for us because programme managers would then ask us to generate lots of reports manually. So, considering employee turnover, we are creating an institutional hierarchy based not on the person’s name but on their title and whereby only programme managers see the results.”

Separately, through leveraging the full data created, instructor reports are available at the same time for those above in the hierarchy, meaning trends can be identified quickly if needed. “Through Blue instructors have a lot of power to determine the evaluation period themselves as long as it ends before the end of the lecture period and before exams start,” said Specialist Data Management & IT Processes, Joris de Vries.

“The broader policy is for instructors to discuss the feedback with students. As a result, students are becoming really engaged with course quality issues. Overall, we have created a high feedback culture, which is beneficial for staff professional development and promotion applications, and is seen positively in our community.”

“Capacity to develop new feedback and evaluation”

In addition to the standard course evaluations, the University offers qualitative analyses every semester for smaller courses that may not meet the overall institutional threshold of five student responses.

Joris explained that the HSG is also looking at moving to a module evaluation system, and closing the feedback loop further through Blue. “The relationship with Explorance has been incredibly beneficial,” he said. “The exchange is something we really value. Talking to, and learning, from each other – the Blue community is an open culture and that is a good style.”

Kristin added: “The electronic, and automated approach, has allowed us the time to build up other areas in the department. We now have the capacity to develop new feedback and evaluation tools in addition to course evaluations.”