Responding to Faculty Concerns about Online Course Evaluations Part 2

In a previous post, we responded to some common faculty concerns regarding online course evaluations. One of the main concerns is that response rates will drop when evaluations are moved online.  This is of critical importance to faculty as student participation in the evaluation process contributes to course development and teaching improvement.

Research does indicate that there is a small decrease when assessments are first transitioned online. However, response rates increase over time and can equal, or surpass, paper response rates depending on institutional support, promotion and incentives. There are several factors that affect student response rates including awareness, confidentiality and faculty influence that need to be taken into consideration to increase participation.

Today’s post will discuss some further faculty concerns regarding moving course evaluations online. In particular we will examine evaluation scores, type of results, qualitative data and response rate bias between online and paper evaluations.

1. Evaluation scores:

   Some faculty members are concerned that using online evaluations will lower evaluation scores. Studies have indicated that even when there are lower response rates, the average evaluation scores are not affected. Researched by DeCristoforo, Layne, & McGinty (1999) found that the type of delivery method (paper or online) did not have any significant affect on mean evaluation scores. Studies by Thorpe (2002) and Avery, Bell, Bryant & Kang (2006) echo this finding showing that paper and online evaluations do not have any substantial difference in evaluation scores. Avery et. al. purported that transitioning to an online system, from a paper-based method, was unlikely to affect evaluation scores.

2. Type of results:

   Another faculty concern is that online results will differ greatly from the data obtained in paper-based evaluations. Research has been conducted on this topic with the outcome being that the results between paper and online evaluations are very similar. A study performed by Donovan, Mader & Shinsky (2006) showed that quantitative results are similar between online and paper evaluations. Donovan et. al. also indicated that both modes of delivery will yield the same proportion of positive to negative comments.

3. Qualitative data:

   As we highlighted in a previous post, some faculty members are concerned that online assessments will yield less qualitative data. Studies that have compared the results obtained from both formats find the opposite to be true. Online evaluations actually have more comments and qualitative data. Donovan, Mader & Shinsky (2006) found that online assessments yield more open-ended comments that contain more qualitative detail than paper-based evaluations. Johnson’s research showed that online evaluations have six times the amount of written comments than paper assessments (Johnson (2002)).

4. Response rate bias:

   Some faculty members may be concerned about response and non-response rate bias in online evaluations. There are several factors (demographics, academic standing, etc.) that can influence response rates. However, those factors appear to have the same influence on both online and paper evaluations. Thorpe (2002) found that students responding to in-class assessments and online evaluations were not significantly different. The concern that dissatisfied students will participate in online evaluations to a higher degree is unfounded. Research indicates that unhappy students are not more likely to participate in online evaluations than they would in paper assessments.