Blog
Driving Student Success with Course Evaluations at the University of Chicago
Written by Explorance.
The University of Chicago (UChicago) faced an interesting challenge a few years ago. While course evaluations had been a cornerstone of their institution, they were behind on the trends. So much so that an article in the student newspaper was published questioning the school’s current state of teaching evaluations.
If the students were talking about it to the point of an article feeling warranted by the editors of the school publication, perhaps it was time to rethink how they were run at the university. This was the backdrop that led the Dean of the university to ask Scott Campbell, assistant Vice President for Academic Services and University Registrar, to find a new course evaluation solution.
Scott was tasked with a daunting challenge since the University of Chicago is a large institution with numerous departments that largely operate independently. The goal was to find a solution to unite all faculties under one system to make course evaluations less tedious and improve data sources overall. The university selected Explorance Blue to power its course evaluation process and deliver meaningful insights to drive teaching effectiveness and student learning.
Scott recently gave a very insightful talk explaining his experience; here are the highlights:
Course Evaluations at the University of Chicago, Before
Scott observed the course evaluation process institution-wide and noticed that almost every faculty had a different way of doing things. Certain schools, like the Booth School of Business, still used paper evaluations, while other faculties used homegrown solutions built by the IT department of the school.
Because of the various requirements of every department, it quickly became obvious that the only option would be a vendor-based solution that could support their exhaustive list of requirements. The elements on this list are rather common, but they are often daunting for most institutions:
- Ability to set response thresholds before evaluations are released
- Student anonymity
- Integration with the Canvas Learning Management System (LMS) and their student portal
- Mobile app for responding and management
- Evaluation of complex courses with various student setups
- Ability to function as the system for all the academics across the university
At the time of the evaluation process, faculties were using a homegrown application to automate the evaluations. However, certain departments weren’t using it; others had different applications altogether, leading to fragmented data that the school couldn’t use efficiently.
This posed a problem since the University of Chicago prides itself on a rather unique process: Offering course evaluation results at the time of enrolment. That way, students can see feedback and decide if the particular course is what they are looking for. Without unified data, the process of unifying student feedback data became quite tedious to gather and distill.
Additionally, it was crucial for the University of Chicago to have a flexible system that allowed evaluations to focus on the course and not the instructor. Course evaluations at the university are fully optional; an instructor can opt out of them at any time.
The real purpose of the new technological platform the University of Chicago was researching was to improve student engagement and response rates on course evaluations. They knew it wasn’t as high as they wanted it to be, but they didn’t have an accurate oversight of how bad it was with fragmented data.
After carefully reviewing several solutions, the University of Chicago selected Explorance Blue as its new course evaluation software. Since Blue’s implementation, every school except the School of Medicine has since come on board after seeing the capabilities of Blue.
The Evolution of Course Evaluations
Explorance Blue was first launched across all undergraduate programs for the University of Chicago, and the rollout was a massive success. They touted a 40-45% response rate, and while the number is lower than they’d like for now, they can move forward feeling empowered by the slew of features Blue offers to improve response rates.
Seeing these amazing results, the Provost’s Office reached out to Scott about expanding the program to their graduate offerings. It’s important to note that UChicago is known as a Ph.D. powerhouse, with some of the highest numbers of Ph.D. students across the nation. Getting good feedback on their education and potential job opportunities was a crucial aspect of these programs.
Ultimately, the implementation process for the graduate schools centered around a committee of the faculties and a student representative who settled on ten quantitative and six qualitative questions.
While the questions were similar to the ones seen in undergraduate programs, graduate programs added a question about teaching assistants and their effectiveness in classes. Additionally, the student representative pushed to include the following question: Please comment on how respected, included and valued you felt in the course.
This highlights that while implementing a platform like Blue often aims to centralize data, it is crucial to remain flexible to every departmental need. This notion is at the cornerstone of Explorance’s design philosophy.
Course Evaluations at the University of Chicago Now
So, what does the finalized course evaluation program look like at the University of Chicago? After a few months with the program, a few adjustments have been made to integrate the program fully into their institution:
Using the right words
The University made an interesting branding decision by changing the term teaching evaluation to course feedback. The fact that students might not be properly trained to evaluate the teaching of their instructor is often a touchy subject at educational institutions. However, referring to it as course feedback resets expectations for everyone involved.
Flexibility and engagement
As previously mentioned, instructors can opt out of the course evaluations at any time, no questions asked. Additionally, the University now allows them to add up to 4 free response questions in order to personalize the evaluation to the specific needs and context of the class.
In order to increase participation and engagement, students who provide course feedback gain access to their final grades three weeks earlier. All students who haven’t shared feedback get their grades a week after the end of the course feedback process.
All results are released on a secure university URL. Students can review evaluations during the registration process for all their classes; historical data is also available upon request.
Giving a voice to students
The University of Chicago prides itself on being a champion of freedom of expression. For that reason, they have chosen not to censor any answer by students on qualitative questions, as long as it is focused on the class content and not the individual.
Another important aspect is their pledge of anonymity. For the university, this goes further than simple confidentiality. Only the project admins can access the unique identifiers in case of disturbing or dangerous remarks. Otherwise, students are welcome to speak and give their opinions on their classes.
Closing remarks
One final crucial requirement at the University of Chicago was the ability to analyze qualitative student feedback captured on course feedback. Traditionally, transforming student comments into data-driven insights was a lengthy process that required additional resources.
Since the University of Chicago takes such a powerful stance on allowing students to voice their opinions freely, reading and analyzing these free-formed responses became a crucial necessity for any vendor that they choose to work with. Luckily, Explorance has developed a solution to circumvent this issue.
Explorance MLY allows universities to harness the power of machine learning to quickly read, analyze, and categorize their qualitative feedback data. Get started with your free Comment Analysis Report:
Blue•Course evaluations•Educational experience•Higher education•People insight solutions•
