Medium 9781605098241

Predictive Evaluation

Views: 1478
Ratings: (0)
At last, an answer to the question that has bedeviled trainers for decades. Predictive evaluation enables you to effectively and accurately forecast training's value to your company, measure against these predictions, establish indicators to track your progress, make midcourse corrections, and report the results in a language that business executives respond to and understand.
Dave Basarab explains how to begin by identifying the specific goals and beliefs you want to instill in participants. The next step is to determine exactly what these will look like when put into action. Finally you develop quantifiable measures of how employees' adopting the target beliefs and goals will impact the business. A key strength of this process is that it is profoundly collaborative—supervisors and employees work together to establish standards for success each step of the way. A how-to guide filled with worksheets, examples, and other tools, Predictive Evaluation ensures that, rather than being regarded as an expense and an act of faith, training will be seen as an investment with a concrete payoff.

List price: $39.95

Your Price: $29.96

You Save: 25%

Remix
Remove
 

7 Chapters

Format Buy Remix

Contents

PDF

 

Chapter 1: The Predictive Evaluation Approach

PDF

Chapter 1

The Predictive

Evaluation Approach

ASTD Study Shows Training Evaluation

Efforts Could Be Enhanced

“The Value of Evaluation: Making Training Evaluations More Effective,” an ASTD Research Study in 2009, shows that companies struggle with evaluating whether their programs meet the business needs of their organizations and whether they are meaningful to employees and business leaders.

It also points out that the Kirkpatrick model is the most utilized evaluation approach followed by Brinkerhoff’s success case method. Jack Phillips ROI

(return on investment) model is also being employed. I was not surprised when in the report they stated that “The least likely metric is actual business outcomes, which nearly a quarter of respondents said they do not measure at all.”1

You may wonder why they don’t measure business outcomes. The report states that the following are most common barriers:

1. It is too difficult to isolate training’s impact on results versus the impact of other factors.

2. Evaluation is not standardized enough to compare well across functions.

 

Chapter 2: Predicting Training’s Value

PDF

Chapter 2

Predicting Training’s Value

Business forecasting is indeed big business, with companies investing tremendous resources—in systems, time, and employees—aimed at bringing useful projections into the planning process. Business forecasting is a process used to estimate or predict future patterns using business data. Some examples of business forecasting include estimating quarterly sales, product demand, customer lifetime value and churn potential, inventory and supply-chain reorder timing, workforce attrition, and Web site traffic and predicting exposure to fraud and risk. Basically business forecasting is used to try to predict the future.

When we predict training’s value, we are using a form of business forecasting—estimating what training will deliver in terms of increased organizational and business results. These predictions come in the form of participant Intentions, Adoption, resulting Impact on the business, and the costs to deliver the value. These predictions enable decision makers to make highly informed proactive decisions. As with any business forecasting method, predicting training’s value has benefits and drawbacks that need to be weighed before proceeding.

 

Chapter 3: Intention Evaluation

PDF

Chapter 3

Intention Evaluation

An Intention Evaluation (IE) addresses the following question: Are participant goals and beliefs upon course completion aligned with desired goals? The steps to conduct a successful IE are the following:

Step 1: Establish the Success Gate for Intention Goals

Step 2: Establish the Success Gate for Beliefs

Step 3: Capture Goals and Beliefs

Step 4: Compare Goals against Standard

Step 5: Calculate Intention Goals and Beliefs Scores

Step 6: Analyze and React

Step 7: Create Intention Dashboard

Step 1: Establish the Success Gate for

Intention Goals

An Intention Goals Success Gate is the percentage of goals written by participants that are judged as matching the goals created when you predicted training’s values (see Chapter 2). For example, if an Intention Goals Success Gate is set at 90 percent and fifty goals are authored by participants, the 90 percent success gate requires that forty-five of those goals be judged as acceptable (as matching the predicted Intention Goals).

 

Chapter 4: Adoption Evaluation

PDF

Chapter 4

Adoption Evaluation

An Adoption Evaluation addresses the following question: How much of the training has been implemented on the job and successfully integrated into the participant’s work behavior?

An Adoption Evaluation measures participant goal completion rate against the defined Adoption Rate created when you predicted training’s value

(Chapter 2). Data are collected on the success that participants have had in transferring their goals to the workplace, results are analyzed and compared to the Adoption Rate, and an Adoption Dashboard is produced to report findings. If needed, corrective actions are put in place to improve adoption. The steps to conduct a successful Adoption Evaluation are as follows:

Step 1: Review the Adoption Rate Success Gate and Decide When to

Collect Data

Step 2: Develop Adoption Survey

Step 3: Collect Adoption Data

Step 4: Create Adoption Dashboard

Step 5: Analyze and React

Step 1: Review the Adoption Rate Success Gate and Decide When to Collect Data

The Adoption Rate Success Gate is the percentage of participants who have successfully implemented their training goals; it is a measure of transference from the course to the workplace. The Adoption Rate for each behavior (Success Gate) was defined in the Predict Training’s Value phase and is an element in the Impact Matrix (Chapter 2). It is the percentage of participants that we want to perform the work (Adoptive Behaviors) and the prediction of successful transfer from training to the workplace. This value becomes the metric used to evaluate the program.

 

Chapter 5: Impact Evaluation

PDF

Chapter 5

Impact Evaluation

Impact Evaluation is identifying the direct impact on and value to the business that can be traced to training. It assesses in quantifiable terms the value of the training by assessing what Adoptive Behaviors have made a measurable difference. Impact evaluation goes beyond assessing the degree to which participants are using what was learned; it provides a reliable and valid measure of the results of the training to the organization.

When participants report a positive impact from training, this approach allows you to articulate how and why training is impacting the business, with a view to leveraging this information to enhance the organizational impact of future deliveries. Conversely, when little or no impact is found, this evaluation method uncovers why that is so and what can be done to remedy that undesirable outcome.

Impact evaluation seeks value to the business. This is done by collecting data on actual business results that participants attribute to successful adoption of their Intention Goal(s). In their Adoption Evaluation survey, participants told us what they did, and we classified them as Successful Adoption or Unsuccessful Adoption. From the successful participants, you now collect additional impact data via three methods: (1) completion of an Impact Survey, (2) interviewing participants, and (3) examination of company records to confirm findings. The steps to conducting a successful Impact Evaluation are the following:

 

Chapter 6: How to Get Started—Implementing Predictive Evaluation

PDF

Chapter 6

How to Get Started—

Implementing Predictive

Evaluation

So you’ve decided that you’re going to do a PE, and, yikes, it is your first one!

First, answer these questions:

• Will PE be used in deciding whether or not to move forward with training?

• Why is the evaluation being done; that is, what decisions does the company want to make during the evaluation?

• Who are the key stakeholders who should receive evaluation reports?

What do they want from the evaluation?

• What elements of a PE will provide the data to aid in the decisions that you and/or stakeholders want to make?

• When is the information needed?

• What resources are available to conduct the evaluation? Can a Steering

Committee be formed to aid with the evaluation? Will internal resources be used or will you employ external evaluation resources?

• Is funding available? What projects will be delayed or not done if PE is implemented?

• What existing evaluation practices are in place? Can they be leveraged and used in the PE? Will PE replace some of these practices?

 

Details

Print Book
E-Books
Chapters

Format name
PDF
Encrypted
No
Sku
B000000023373
Isbn
9781605098579
File size
867 KB
Printing
Allowed
Copying
Allowed
Read aloud
Allowed
Format name
PDF
Encrypted
No
Printing
Allowed
Copying
Allowed
Read aloud
Allowed
Sku
In metadata
Isbn
In metadata
File size
In metadata