4 Challenges in Evaluating Massive Open Online Courses (MOOCs)

Whether you consider massive open online courses (MOOCs) the future of higher education or a fad, the fact remains that they are prevalent. As the ‘massive’ part of the name indicates, MOOCs are courses with unlimited enrollment. These courses are generally provided for free, are open to everyone and usually are not-for-credit. MOOCs are currently being offered by over 100 well known universities with enrollment ranging from a few thousand to over 180,000.

MOOCs are beneficial as they provide an accessible way for people to gain new knowledge and skills. However, the nature of MOOCs makes evaluating their quality and effectiveness difficult. Today’s post will outline four challenges in evaluating MOOCs including the lack of established criteria and low completion rates.

1. No established criteria:

   MOOCs are a relatively recent phenomena that have no established evaluation criteria. These courses vary greatly and do not adhere to a standardized, accepted definition or structure. Generally MOOCs do not have set learning objectives that apply to all participants. Each person  can set their own goals and objectives, or none at all. The lack of validated assessment criteria and the variety of learning objectives makes these courses difficult to evaluate. Traditional course evaluation methods do not easily apply to MOOCs as they are based on classroom teaching.

2. Low completion rates:

   Thousands of participants enroll in MOOCs with only a small number completing their courses. Recent evidence shows that only 7-10% of enrolled participants actually finish their course. Using completion rates as an indicator of success for MOOCs is not a valuable measure. These rates do not take into account the variety of participant learning experiences. Some students gain the knowledge they require from particular content but do not complete the course. Other participants passively learn or engage only in discussions, but, again they do not complete the course.

3. Varied instructor involvement:

   Another challenge in evaluating MOOCs is that the role of the instructor varies. In some courses the instructor is merely a facilitator, in others the teacher is the principle subject matter expert. The large number of participants also poses a problem for the formation of a instructor-student relationship.  The sheer volume of students in MOOCs makes individual student attention impossible. The outcome of this is that students can feel disconnected and unengaged in the course. The role of the instructor, along with the variety of student experiences, makes evaluating teaching effectiveness extremely difficult.

4. Accessibility:

   One of the benefits of MOOCs are that they are available online for everyone. However, this also brings up the issue of accessibility. As some MOOCs rely heavily on multimedia content accessibility has to be taken into consideration. Videos, presentations, audio lectures, social media discussions, etc. need to be accessible to everyone. Videos, for example, should include captioning to ensure that all students can engage with the content. Accessibility and differing student information-literacy skills also pose a challenge in evaluating MOOCs. If participants cannot access, understand or engage with course content evaluating learning will be difficult.