Key Metrics Series: Measuring Performance Improvement Due to Training

Companies spend billions of dollars every year to improve the effectiveness and efficiency of their employees. One key path to push this performance improvement is via training programs. As a result, the learning function continues to be one of the most vital roles within the business.

The very essence of training is not to increase knowledge and skills but to go further and cause a change in behavior that has a positive impact on the business. Learning and Development (L&D) works painstakingly hard to ensure their programs are moving the needle in this regard. Much time and energy are invested to quantify that moving of the needle.

Sometimes key programs are evaluated closely, and impact studies are conducted to uncover if the training influences business outcomes. These impact studies are fantastic ways to uncover learner performance change, but they are costly, time-consuming, and require large amounts of complete data to ensure the findings are statistically useful. Unfortunately, this approach is just not scalable for all training programs to reap the rewards of a thorough analytical study.

There is another way to measure performance improvement as a result of training. It has been validated across industries and course types and is being collected by hundreds of organizations across thousands of training programs as I write this. The cost-efficient and scalable approach to measuring a performance improvement metric can be added easily to any learning evaluation strategy. According to works of Dr. Jack Phillips of the ROI Institute, it just takes three ingredients:

Estimation

• Estimation is commonly used in measuring employee perceptions. This is true for training as well. At this stage, the learner is estimating the job performance change in general, including training and all other factors.

Isolation

• Learning functions are deeply interested in the effect training had on performance improvement. Therefore, the next step is to isolate the estimated increase in performance to just the training event. In this part of the process, the learner estimates how much the training has or will influence job performance, relative to the other factors. The learner also estimates how frequently they expect to use the training on the job.

Adjustment

• Bias adjustments are a way of life in the social sciences. Learning evaluation should be no exception. In training, adjustment is made for two main reasons: conservatism and bias. Conservatism is essential when building a complex metric because being conservative in assumptions allows for integrity to be woven into the metric. Self-reported bias by learners is typically inflated. When dealing with bias, it is recommended to either directly ask learners their confidence in estimates or to use a relevant proxy. In the absence of direct confidence estimates, Explorance utilizes an adjustment factor of 35%, based on historical research.

Taken together, the principles of estimation, isolation, and adjustment form powerful reference points for tabulating a systematic, replicable, and comparable performance improvement metric: Estimated Performance Improvement (EPI). This is precisely why the foundation of Phillips ROI Methodology is centered around these principles.

EPI may be gathered immediately after training to provide a leading indicator of performance improvement, and through follow up surveys of learners and managers after learners have had an opportunity to apply the training on the job. When combined with the cost of training and learner salary at the program level, the projected impact can be monetized to calculate ROI.

When gathered consistently across programs, Estimated Performance Improvement can be used as a relative indicator of the impact each program is delivering, even without program-specific cost and participant salary data. Programs with a low EPI measure may deserve additional scrutiny to ensure that they are delivering the expected impact on the business.

Measuring performance improvement via the ROI methodology allows organizations to understand better how their training programs impact the jobs of their learners and, as a result, drive business results. Logically, you cannot manage what you do not measure. Organizations need to establish the right performance measures for all key investments, and employee growth and development is one of the most important investments any company will make.

Explorance pioneered the scalable use of Estimated Performance Improvement in the Metrics That Matter evaluation methodology, as one of three leading indicators that L&D leaders can use at scale to better understand and manage their L&D Portfolios.

Logically, you cannot manage what you do not measure. Organizations need to establish the right performance measures for all key investments, and employee growth and development is one of the most important investments any company will make.

Read part 2 of the Key Metrics Series: Why Your Learning Measurement Strategy Should Include the Ultimate Question.

Download the white paper “Human Capital ROI Scorecard – A Comparison of Results from Three Impact Methods”.