The 8 Step Program Evaluation Process – Part 1

Training Experience Management (TEM) is a comprehensive evaluative cycle that allows you to achieve continuous improvement in the teaching and learning process. In today’s post, we apply the TEM framework to program evaluations to ensure recurring feedback for improvement. This post will include the first four steps: establishing benchmarks, identifying stakeholders, describing the program and designing the evaluation.

1. Step 1 – Establish benchmarks:

   It is important that you develop a set of benchmarks. These benchmarks will allow you to evaluate the different aspects of your program in relation to best practices and industry standards. Benchmarks are powerful tools for change when they are aligned with the institution’s mission and strategic objectives. Program benchmarks can be based on initial student training requirements, government accreditation, comparative analyses (other universities) and employer needs. This comprehensive set of benchmarks then becomes the baseline from which the institution will monitor its progress.

2. Step 2 – Identify stakeholders:

   As teaching and learning affects a variety of stakeholders, it is crucial to involve them in the evaluative process. TEM moves beyond simply administering program evaluations to students and includes gathering feedback from a wide array of stakeholder groups. Program stakeholders can be separated into three main categories: those served by the program, those involved in operations and those who will use the evaluation data. All of these audiences must be engaged in the process so that a comprehensive view of the program is established. Including multi-source feedback in your assessment will provide you with useful and diverse feedback for continual program improvement.

3. Step 3 – Describe the program:

   The next step in the process is to describe the program that will be evaluated. The program description plays a very important role as it becomes the reference for all future program decisions. This description should include the program’s goal, objectives, learning outcomes, structure and how it fits into the overall educational environment. The summary should also include a list of resources and activities that are needed to accomplish the program’s goal. It is important that the program description be agreed upon by all stakeholders. If there is no clear and accepted description the evaluation runs of risk of producing inaccurate and questionable data.

4. Step 4 – Design the evaluation:

   At this point in the process, you need to determine the purpose of the evaluation and its design. The purpose of the evaluation can be to measure student learning, find ways to improve the program or to determine the program’s usefulness. Evaluations can focus on a variety of areas such as processes, resources, activities and outcomes. You need to also take into consideration how the information will be used and what are the best methods for collecting feedback in your institution. Establishing the purpose and method will help you in selecting your survey questions. Evaluation questions can be based on: the institution’s mission, the program’s goals & objectives, previous needs assessments, strategic plans, accreditation criteria, etc.