University of Johannesburg
Johannesburg, South Africa
Module and lecturer evaluations did not contribute to the University’s overall strategy to improve student success through evidence-based decision making. A new online, semi-automated system was needed to support this change in institutional thinking.
"Module and lecturer evaluations did not contribute to the University’s overall strategy to improve student success through evidence-based decision making"
- Online system that is fully integrated with Blackboard
- Ability to handle high volumes of evaluations
- Provision of meaningful data for the University
- Automated evidence-driven reports
A change in strategic and operational thinking
With a student population of over 50,000, of which more than 3,000 are international students from 80 countries, the University of Johannesburg is the third biggest residential university in South Africa. Since 2018, the University has been on a course evaluation journey, from paper-based to Blue-based evaluation that is semi-automated and fully integrated with Blackboard.
Driving the University’s response is Riaan Loots, a Senior Manager in the University’s Centre for Academic Technologies (CAT), set up to provide a richer learning experience for students and fulfil the institution’s vision of 21st century skilled academia through the promotion of innovative technologies.
“The old system of course evaluation was ad hoc, paper- based and cumbersome,” explained Riaan, who has been working at the University since 2003 and is also a University of Johannesburg alumnus. “The bubble sheet scanners we used were reaching the end of their life and, with a steady increase in evaluations, operationally we had to do something different. A change in thinking was also underway. Evaluation reports had been primarily used for academic professional development and promotion purposes – the University’s management team had no insights from this valuable data source. Module and lecturer evaluations did not contribute to the University’s overall strategy to improve student success through evidence-based decision making.”
Expert system for high volumes of evaluations
A decision was taken that all first-year undergraduate modules and all priority modules had to be evaluated. Therefore, a new approach was needed that was efficient at handling high volumes of evaluations – 4,000 in all – while also providing meaningful data to the University as a whole.
“After investigating Blue, we were really happy with the scope of the offering and appointed Explorance as our course evaluation partner,” Riaan said. “With the Centre for Academic Staff Development (CASD) we began work in 2018 with a small group of volunteers in the University’s College of Business and Economics and ultimately piloted the online system over a two- week period. We were particularly pleased with the integration with Blackboard – the anonymous and easy-to-use interface, enabling strategies for increased responses, improved reporting, automated feedback and data analytics on the feedback received.”
The University has developed two categories of evaluation: taught courses – undergraduate and postgraduate (module evaluations and teaching evaluations) – and supervision programmes (different sets of questions), and full implementation is underway.
“Last year was the first time we ran all undergraduate evaluations online – featuring 1,043 modules and 1,388 academics, with over 40,000 students surveyed, and more than 1,000 reports being released. The biggest success has undoubtedly been the automated evidence- driven reporting for individual lecturers, aggregated reports for the Department, School, Faculty or College, aggregated reports with relevance to student successes and staff development, and an aggregated report on all evaluations for the Deputy Vice-Chancellor: Academic, to whom CAT/CASD colleagues and I ultimately report to.”
Blue an enabler to “move everything online”
Moving into 2020, Riaan highlighted a number of goals as the course evaluation journey continues. “Closing the loop, and feeding back to both students and academics, is one area for development alongside some more operational targets,” he said. “However, going back to the rationale that module and lecturer evaluations should form part of our overall strategy
to improve student success through evidence-based decision making, we are definitely beginning to do that. While some evaluation is still paper-based, with Blue we now have the ability to move everything online at any time, and hopefully that will be the case by the end of this calendar year. Blue is definitely the most complete supporting tool I have seen, from its data handling to ability to capture multiple evaluations.”
Blue•Case study•Course evaluations•Higher education•