Del Mar College Gets a Clear View on Course Evaluation Data with Blue

For Dr. Larry Lee, it wasn’t just that paper course evaluation forms were inefficient. They were holding them back.

“If we want to realize continuous improvement at an institutional level, we need to have data that we can rely on to make informed decisions,” said Lee.

Larry Lee, PhD, is Dean of Business, Professional and Technology Education at Del Mar College (DMC), a large community college in Corpus Christi, Texas. A former Professor of Computer Science, Lee holds a doctorate in Curriculum and Instruction.

For Lee, improving the quality of data collection for course evaluations was an imperative.

“Paper-based course evaluation forms are very costly, in terms of both physical and human resources,” he said. “And the quality of the data is insufficient. It’s too costly and time-consuming to get the data tabulated and analyzed at a level we’d like.”

Elearning task force provides platform for change

Lee heads up an eLearning task force, which was mandated, among many other projects, with finding a new solution for their course evaluations. The task force is made up of two Academic Deans, the CIO, the eLearning Coordinator, three institutional committee chairs (Chair’s Council, Faculty Council, Technology Committee), three faculty members, the Assistant Dean for Continuing Education, the Dual Credit Director, and the DMC Instructional Designer.

The task force used a systematic approach in their acquisition of a new course evaluation system. They began by analyzing their needs, defining the requirements, then identifying potential solutions.

“The ability to integrate easily with our current Canvas and Ellucian Colleague systems was a critical requirement,” said Lee. “Integration makes processes more efficient. It also provides more opportunities to analyze and understand data by combining results with demographic data.”

Selecting the best possible through shared governance  process

Lee started researching products in November 2012, then conducted assessments throughout the 2013 winter semester with the task force, evaluating solutions against their established requirements.

“We have a shared governance process here at DMC,” said Lee. “To introduce effective, long-term change, we include input from all stakeholders when making decisions that are going to affect the whole college.”

The task force narrowed the shortlist down to three products: two online course evaluation software products and one scanned paper form product.

By May 2013, the committee selected Blue from Explorance as the best solution for their requirements.

“We chose Blue for several reasons,” said Lee. “In particular, we liked the flexibility, the analytics features, and the ability to customize it the way we needed it, particularly at the department level.”

“We’ll get a bigger return on our investment once we can start using Blue to conduct surveys in addition to our course evaluations.”

Larry Lee, PhD, is Dean of Business, Professional and Technology Education at Del Mar College

The group also liked that Blue could be extended beyond course evaluations to other applications, such as surveys.

“We’ll get a bigger return on our investment once we can start using Blue to conduct surveys in addition to our course evaluations.”

Del Mar College opted to have their system hosted, which provides several benefits, including a projected reduction in long-term costs.

“If we ran Blue ourselves, it would have been another server to maintain and another person to train and specialize,” he said. “With Explorance hosting Blue, it reduces our long-term costs and allows our in-house IT resources to better focus on their tasks.”

Professional services ensure a smooth rollout

Implementation of the project began in the 2013 summer semester. Explorance

provided professional services for implementation and training, including initial consultation, project set-up, and monitoring and reporting.

They ran a small pilot in August 2013 in six classes, to keep close watch on the project and obtain feedback for making improvements.

“We wanted to see what the students would see, what the instructors would see, and the response rates,” he said. “We were impressed. It all went smoothly, and any technical issues that came up were resolved very quickly.”

To maximize response rates, DMC implemented several programs that Explorance professional services recommended and that they tailored to fit their policies and procedures.

Email invitations, for example, sent directly from the Provost , encouraged students to complete the evaluations online, with a link to the form contained in the body of the message. Two automated email reminders were sent from academic deans if students didn’t complete their forms.

Available from anywhere, including the canvas portal

DMC also used the Blue Portal Integrator (BPI) to integrate Blue course evaluation forms into the college’s Canvas portal. Explorance professional services provided BPI training to show how they could add blocks to the portal, create email triggers, and adjust areas of view, among others.

“Once students login to the portal, they receive a message that notifies them of their outstanding evaluations,” he said. “It was important to get response rates as high as possible, and making these forms available from the Canvas portal was an essential component.”

The evaluation forms are also compatible with any operating system and any type of device.

“Students today do almost everything on their mobile devices,” he said. “We need to be where they are, so we were glad to see the forms rendered nicely on mobile phones and tablets.”

After the initial pilot was completed, Lee met with the participating instructors to discuss results and ways they could improve. One area of need was around communications. Instructors noticed that students often did not check their DMC email inboxes, so weren’t receiving the notifications.

“Several instructors took the initiative to provide in-class time to complete the forms, if they had computers available in class or in a nearby lab,” said Lee. “We have also developed a new text message alert that will go out to students in addition to their email address of choice. This will be launched for Fall 2014.”

Flexibility to handle unique school policies

In Fall 2013, DMC rolled out the online course evaluation forms to the entire college. As part of the professional services package, Explorance ran the course evaluations for them, ensuring emails went out to the appropriate places and generating the reports afterward.

DMC’s evaluation protocol is unlike most other colleges. While untenured faculty are required to have all their classes evaluated by their students, tenured faculty are only required to have two courses per academic year evaluated.

“Blue’s flexibility was able to handle this schedule,” he said. “We just had to go into the faculty table and select the appropriate instructors and class sections ourselves, instead of generating evaluation forms for all classes.”

“In Canvas, we can hide the sections that weren’t being evaluated, so students saw only those classes that required them to complete a form.”

The hard work paid off, and the college had satisfactory response rates. “We are happy with our response rates,” Larry said. “Nonetheless, we’re always looking for ways we can do better.”

Vastly superior qualitative data

Since all went well for the Fall semester, DMC decided to run the course evaluations themselves in the Spring. They received remote Autonomy Training from Explorance professional services, which showed them how to manage projects, reports, and other administrative functions. DMC then launched the  survey.

“We ran the survey for one week less than for the fall semester, and saw an improved response rate,” he said. “We are really pleased with the results.”

DMC is also very pleased with the quality of data they are getting.

“Right now, we have the best picture we’ve ever had,” he said. “The comments are substantially better. Once we eliminate the outliers, we have a much more accurate picture of our students’ perspectives.”

DMC also appreciates the speed with which they are getting the results.

“Reports are now taking a matter of days, as opposed to weeks or even months,” said Lee. “And they are much more robust. We can ask multiple questions, such as regarding the facilities.”

More detail, broader distribution

This key information is now being distributed to administrative levels, in a variety of formats that give administrators a better understanding of instructors’ interaction with students.

“The Department Chair, the Dean, and the Provost can see results for everyone in their division all in one report, which is great,” he said. “These are in PDF form, which is much more manageable. They can easily access them from their email or the Canvas portal.”

The numbers provide a general overview of student response, while comments allow both instructors and their supervisors to dig deeper.

“The reports show how a class compares to the division and to the college as a whole. You can see general trends in the data, where teaching might be strong, or require some peer support, or where facilities are lacking,” he said. “You can also look at the individual questions to see how an instructor is performing in certain categories, such as instruction.”

DMC is also benefiting from the tabulated comments, which appear in one report for viewers to read and understand.

“The graphs give you a quick picture. Then you can scroll through the comments, to clarify what you’re seeing in the numbers.”

Clear skies ahead

In under a year, Del Mar College has gone from paper processes to online course evaluations, relying on strong decision-making from shared governance practices and the guiding objective of supporting improvement at the college. Explorance is glad to play an important supporting role in their success.

“Blue is a very powerful product that allows us to do more in-depth surveying,” Lee concluded. “It’s going to be a powerful tool for us to get the information we need to help us change our processes and work towards improvement.”