Assumption College Goes Digital with Blue

Like many other primarily undergraduate institutions, Assumption College was still using paper-based course evaluations to capture student feedback on their instructors and courses.

The paper-based process was slow and expensive, as can be expected. “We distributed forms, even pencils, then collected and scanned the forms, typed up the comments, and then collected them again for reporting,” says Landy Johnson, Director of Grant Development and Coordinator of Undergraduate Research at Assumption College.

“The scanner was old and prone to jamming. Storage took up massive floor space, and faculty didn’t get reports until halfway through the following semester.”

But faculty and administrators were reluctant to move to online evaluation forms just yet.

“We were concerned about several things: cost and security especially. Probably our greatest concern was around response validity — many faculty questioned whether students could give useful or even reliable feedback outside of a classroom setting,” says Landy.

The college appointed a committee to evaluate the best course of action, which started out split on the decision to keep course evaluations on paper or go online. After a year of research, assessment, and discussion, they achieved full consensus for going online, with a list of key criteria to be met.

Blue course evaluation the top of the class

After careful consideration of the options available, Assumption College decided on Blue course evaluation software from Explorance to meet their list of needs.

“Blue offered a lot of the security features we were looking for: We could keep our data in-house as well as integrate with our LDAP authentication system, which provided the highest security possible,” says Landy.

Blue had other features that were also appealing. As a feedback management system that was designed to integrate easily with in-house systems, Blue could be quickly integrated with the college’s Datatel system.

“We wanted to be able to make evaluation forms available on our Datatel school portal sometime in the future,” Landy says.

In-house management taking less time than with outsourced service

Once Blue was implemented, Institutional Research found that managing course evaluations took less time than when an outsourced provider did it — more than half the time.

“We used to have to extract data from Banner using queries, and then clean up all the data,” she says.

“Now that the surveys are integrated with Banner, it’s so much easier to pull the data we need. It takes us about half the time it used to.”

Smooth implementation

Assumption purchased Blue course evaluations in August, began implementation in late September, and launched the pilot project with about half the departments in November.

“Importing our college registration information into Blue and setting up the automatic email was very straightforward,” she says.

The only real difficulties they ran into during implementation were in the database.

“We quickly figured out that a lack of standardization in naming conventions could cause problems,” says Landy.

“It was good incentive to straighten up our database and for people to notify us when they changed their names. For example, we had professors who were in the Registrar’s database with their married name, but were in the course database with their maiden name. We also found that some names, like de la Cruz, had the de or the la capitalized in one part of our system, and lower case in another part.”

The first semester went so well, they decided to roll out to all departments for the second semester.

Saving money, trees, and floor space

Assumption has now completely done away with the paper and manual labor resources required to manage course evaluations, though oversight from the Provost’s Office is still required.

“We were using 100,000 pages of paper every year, and estimated labor costs were about $11,000 a year. Plus, we had to store all our completed evaluations: that’s 36,000 pages just for one semester of the undergraduate program.”

“Now when we need to research any professor’s evaluations, they’ll be very easy to pull up and print out as we need them.”

“Blue can also now be used on mobile devices. With Wi-Fi in every classroom, faculty will have the option to conduct course evaluations in-class. This could work well especially for our Honors students who all have iPads.”

 

Getting a response

Administrators were pleased by the response rate results from the pilot, and are refining their processes as they continue to learn what works best.

“Our first round of course evaluations had an overall response rate of 66.31%,” she says. “We were expecting it to be lower than what we had with paper, so this wasn’t a surprise.”

“For the second semester, it went down to 57.9%. We’re seeing varying response rates from different departments: some have 71% and 72%, which is great — the same as we had with paper. But some departments with less incentive to get the forms completed are as low as 27%, which brought the overall average down.”

“We’re looking at ways we can improve response rates, such as integrating with our Blackboard system or increased outreach,” she says.

“We think compliance will increase as time goes by and students become acculturated to the new process. Over the summer we saw significant improvement with the response rate for our Continuing Education program, thanks to the awarding of Bookstore gift cards and improved communication with professors.”

“Blue can also now be used on mobile devices. With Wi-Fi in every classroom, faculty will have the option to conduct course evaluations in-class. This could work well especially for our Honors students who all have iPads.”

Information that makes a real difference

Faculty have been giving excellent feedback on the new reports they’ve been receiving, which are now all available online.

“A few professors called to say they were thrilled to get the reports so quickly,” she says. “Some even changed their planning for spring courses based on their reports from the fall semester.”

The college also has other useful reports which were previously unavailable to them.

“We now have a departmental report that summarizes results for all the courses in the department. The Provost now reviews individual department reports as well as the college-wide report.”

Getting full access

Getting a fully accessible online evaluation form was an added advantage for the college.

“We’re attentive to the needs of our students with disabilities, so we were very happy to learn that Blue evaluation met Section 508 and WCAG accessibility standards,” she says.

“This was doubly important for us as we have federal grants that specifically require us to recruit students with disabilities for our rehabilitation services degree program.”

Landy is also happy to report that they’ve been getting excellent customer service from Explorance.

“We’ve been very happy with the customer service,” she says. “When we call or email, we get a response right away. We have always felt we can reach them whenever we need to.”

With the right solution and the right communications, Assumption College has gotten out of their paper chase for course evaluations, and done it right.