University of Sharjah Achieves 90%+ Response Rates with Integrated Course Evaluations

The Deanship of Quality Assurance, Institutional Effectiveness and Accreditation at the University of Sharjah (UofS) has developed a highly successful method for attaining course evaluation response rates that are consistently over 90%. Previously, those kinds of numbers were achievable only by preventing students from seeing their grades until they completed a survey form. UoS has developed a new method that achieves similar results but with more moderate incentives.

University of Sharjah is a relatively young university, established in University City in Sharjah, United Arab Emirates in 1997. From the beginning, the university has fostered a culture of feedback within the community, developing processes that elicit feedback from multiple sources for improving the quality of education and delivering an excellent student experience.

“Getting feedback from multiple sources has many benefits. First, it allows for better formative decisions for teaching improvement,” says Dr. Esam Agamy, Dean of Quality Assurance, Institutional Effectiveness and Accreditation at UofS. “It also furnishes important feedback for summative decisions and for merit-based pay and contract renewals. Multiple source feedback provides more accurate, reliable, fair, and equitable decisions than one based on just a single source of feedback.”

Meeting the challenges of feedback gathering

But obtaining feedback from a large population – UofS hosts over 15,000 students at 5 campuses located all over UAE – isn’t easy. The university used to conduct surveys on paper, which consumed enormous amounts of resources and time.

In 2011, UofS switched to online forms provided by Blue, a world-class online course evaluation and survey system designed for the academic environment. “We looked at Blue because it has an established reputation in Australia and the US, and when we compared it to other comparable systems, we found it more effective and easier to use,” Agamy says.

The shift from paper to online provided many benefits. “We used to print and distribute over 130,000 pages,” says Hiba Jadallah, Data Analyst and the Blue administrator at Quality Assurance, Institutional Effectiveness and Accreditation at UofS. “We are now able to capture and analyse much more data than previously, and all without consuming so much paper.”

Shifting to an online mindset

While the savings in time and paper provided considerable benefits, the shift to online also introduced a new shortcoming: lower response rates. They started with a highly respectable 51 percent, but this figure declined steadily over the years, hovering between 22 and 36 percent, depending on the various incentive programs they ran.

The team consulted with Explorance, the maker of Blue, and put their heads together to try a new approach. Blue has a feature that lets administrators block grades until students complete a course evaluation form and which has a track record of producing very high response rates. The quality assurance team wanted an incentive that was less restrictive, however, so they took this concept and modified it.

“Instead of blocking grades, we chose to block course materials until the student completes the course evaluation or required survey,” says Agamy. “This provided a high incentive to students for completing the form, but without feeling like they are being penalized for failing to do so.”

Course blocking the right incentive

Course blocking takes place on a course by course basis. Course materials available on the Blackboard® Learning Management System (LMS) are made unavailable to the student until he or she completes the evaluation. This lets students complete surveys one at a time, on their own schedule, instead of blocking all grades or materials until all evaluations are completed.

The team designed an appropriate timeline for the process that has had excellent results:

• Day 1-10: Course evaluations are available for completion, accessible from the student portal as well as by email
• Day 11-28: Course materials blocked on Blackboard until the course evaluation is completed
• Day 28: Course evaluations end and all course materials are unblocked
• Exam period begins

From a technology perspective, the implementation was a little complex. The university had to integrate Blue with the Student Information System (SIS), which relies on Banner® by Ellucian, to determine which courses to block for which students. They also needed to integrate with their Blackboard LMS, in order to block course materials themselves on a course-by-course basis.

“We are very happy with the support we received from Explorance to help us make this system work,” says Jadallah. “They worked hard to ensure we could go-live on the expected dates and to respond quickly to our concerns whenever we had any.”

Getting positive response all-around

The deanship used a similar strategy to raise response rates for the important Senior Exit Survey, which provides vital data for UofS’s reporting and accreditation. To increase response rates, they made it a part of the exit clearance process. “Our response rates were at 38 percent in 2013-14. Today, in 2017-18, they are at 75 percent,” says Jadallah.

To ensure continued high response rates, and to further the culture of feedback within the university, the quality assurance team maintains constant communication with the academic community, informing them of the results of surveys and the steps they’ve taken as a result of the surveys.

“For example, we noticed a trend in complaints about certain facilities. We routed these to the appropriate operations departments, who then provide feedback on what steps they’ve taken to address various issues, which we communicate back to faculty and the student body,” says Agamy.

More data for analysis and development

Online capabilities also provide richer data for analysis, instructor development, and important communications. “Instructors and department heads can now do comparative analysis, looking at different semesters or years,” says Agamy. “We are right now in the middle of implementing a dashboard, so instructors can conduct their own personal analysis on the data.”

Overall, the deanship is very pleased with their experience with Blue, and the many benefits they’ve seen from their implementation and the processes they developed around it. “We run a wide range of surveys; with Blue, we are able to run them properly, on time, with good response rates, and good reports that can be delivered to all the necessary stakeholders,” says Agamy. “This really has been an excellent experience.”

“Blue allows us to be very productive,” Jadallah concludes. “We are able to produce quite a lot with a relatively small group of people.”