The 8 Step Program Evaluation Process – Part 2

In our last evaluation post, we examined the first four steps of an effective program evaluation process. This included establishing benchmarks, identifying stakeholders, describing the program and designing the evaluation. However, the program evaluation process is far from complete. Now it’s time to move into the actionable phase of the process. Today’s post will cover the next four steps: gathering feedback, analyzing results, making improvements and continuous monitoring.

5. Step 5 –  Gather feedback:

   In this stage of the process, you need to determine the type of data you want to collect and the appropriate evaluation method. The data you obtain may be qualitative or quantitative, depending on your needs. The type of data you compile will depend on the questions you designed and on your target audience. Depending on the audience, you may want to use qualitative feedback methods such as interviews, focus groups or observation. For a different audience you may want to collect quantitative data regarding your program. You can achieve this by methods such as surveys and questionnaires. Regardless of the method, you’ll want to take great care to ensure the information is valid and reliable.

6. Step 6 – Analyze results:

   Now that you’ve administered and gathered the feedback it’s time to analyze the results. You can look for trends, gaps, strengths and areas for improvement in the data to get a picture of how the program is performing. When analyzing you need to interpret the significance of the results and consider different ways to compare the data. At this time, you’ll want to go back to your benchmarks and program description and compare the information. This will allow you to understand if the program is meeting its objectives, goals and learning outcomes. As part of the analysis phase, recommendations should be provided that are consistent with the findings. These recommended actions or decisions will form the basis for any program alterations.

7. Step  7 – Make improvements:

   At this stage, you will want to share the evaluation report and recommendations with relevant stakeholders. These decision makers will play an integral role in whether program improvements or alterations are implemented. The recommendations, along with stakeholder input, will drive which actions get implemented and when. Program changes can range from small improvements to the curriculum to strategic decisions such as ending the program. Regardless of the actions, you will want to ensure all aspects are considered such as impact, policy implications and budget. Keep in mind that follow-up may be necessary to remind stakeholders of the agreed-upon actions to ensure the improvements get executed.

8. Step 8 – Continuous monitoring:

   We’ve labelled this ‘step 8’; however, monitoring should be a continuous cycle that happens throughout the evaluation process. Improvements that are made to the program should be monitored and compared to the initial benchmarks. This frequent monitoring will ensure that your program provides a consistent high return on expectations (ROE). ROE is defined in the Training Experience Management framework as a holistic measure of the benefits that are realized from any program or initiative, both qualitative and quantitative. ROE encapsulates what program success looks like to all stakeholders involved in the process.