40 Chapters
Medium 9781576753484

3. The Four Levels: An Overview

Kirkpatrick, Donald Berrett-Koehler Publishers PDF

Chapter 3

The Four Levels: An Overview

he four levels represent a sequence of ways to evaluate programs.

Each level is important and has an impact on the next level. As you move from one level to the next, the process becomes more difficult and time-consuming, but it also provides more valuable information. None of the levels should be bypassed simply to get to the level that the trainer considers the most important.These are the four levels:


Level 1—Reaction

Level 2—Learning

Level 3—Behavior

Level 4—Results


As the word reaction implies, evaluation on this level measures how those who participate in the program react to it. I call it a measure of customer satisfaction. For many years, I conducted seminars, institutes, and conferences at the University of Wisconsin Management

Institute. Organizations paid a fee to send their people to these public programs. It is obvious that the reaction of participants was a measure of customer satisfaction. It is also obvious that reaction had to be favorable if we were to stay in business and attract new customers as well as get present customers to return to future programs.

See All Chapters
Medium 9781576753484

10. Using Balanced Scorecards to Transfer Learning to Behavior

Kirkpatrick, Donald Berrett-Koehler Publishers PDF

Chapter 10

Using Balanced Scorecards to

Transfer Learning to Behavior

James D. Kirkpatrick, Ph.D.

I have asked my son, Jim, to write this chapter because of his knowledge and use of the Balanced Scorecard. In my description of moving from Learning to Behavior, I have concentrated on the motivation of the learners and the encouragement of their supervisors. I have described the various types of supervisors, including those who prevent or discourage the transfer. I have urged the trainers to work with the supervisors to help them become “encouraging” instead of “preventing” or “discouraging” bosses. I realize that this is not enough to be sure that the transfer takes place—hence this chapter on the Balanced Scorecard.

Don Kirkpatrick

believe that transferring learning to behavior is one of training’s biggest challenges. My father agrees—so much so that we recently wrote a book called Transferring Learning to Behavior: Using the Four

Levels to Improve Performance.The University of Toyota (UOT), under the leadership of Russ Mundi and Chuck O’Keefe, also believes it to be true. Transferring Learning to Behavior contains “ten best” practice case studies, one of which is from Toyota and in which Russ outlines a corporate challenge to improve a critical element of customer satisfaction. Based on customer feedback, the UOT designed a ten-step program to do just that.The program is designed to ensure that training participants actually apply (level 3) what they learned (level 2) during training.

See All Chapters
Medium 9781576753255

9. Manufacturing Organizations

Kirkpatrick, Donald Berrett-Koehler Publishers PDF

Manufacturing Organizations


solution from a performance improvement perspective, rather than training solution perspective.

Preliminary Background and Research

The TFS conducted three main activities:

• A group of four associates spent eighteen months researching top-performing dealerships. In some cases they spent a full day observing and interviewing all job functions in Toyota sales departments. They observed and interviewed Dealer Principals,

General Managers, Financial Services Managers, and Sales


Important Note: All dealerships are franchises, which means that managers can choose whether or not participate in any improvement program.

• Researchers mystery-shopped public auto retailers that were just coming onto the market and advertised a new of way of doing business.

• A major American captive automobile financing corporation allowed the researchers to visit their training facility for three days.

During that time, they were allowed to speak to the facilitators and directors of training. They also observed actual training sessions, which were conducted in a unique learning environment.

See All Chapters
Medium 9781576753255

3. Strategy and Leadership

Kirkpatrick, Donald Berrett-Koehler Publishers PDF


Foundations for Success

nies are driven by tradition and budget. The practice of trying to be all things to all people is very tempting and pervasive. In response to this, decision-makers are jumping at impressive vendor and consultant packages that promise higher sales, better customer or employee retention, executional excellence, or faster turn-around times.


Since I have broached the topic of vendors and consultants, I would like to offer some words of advice. I have worked for and with a number of professional consultants, and have been around enough to have seen the spectrum. Let’s start with the downside. Watch out for overpromise, under-deliver. This is how that scenario typically unveils. A

firm somehow gets your attention—either from a mailing, phone call, website, e-mail, or personal contact—inquiring about your training needs. You agree to meet with them (“but I only have an hour”) and they show-and-tell you with an impressive presentation of what their products or services can do to enhance your training effectiveness. If you are impressed enough and find their offerings potentially beneficial, you set up another meeting with higher-ranking participants from both sides. So far, so good. To make a long story short, you agree to make the investment and begin the implementation process. There are several considerations you would have been wise to discuss before making that decision.

See All Chapters
Medium 9781576753484

9. Managing Change

Kirkpatrick, Donald Berrett-Koehler Publishers PDF

Chapter 9

Managing Change

here is one important ingredient that is basic to all evaluation approaches. There must be a realization that managing change is that ingredient. It starts with the determination of what changes are needed. We call it “determining needs.” We need to determine what knowledge, skills, and/or attitudes are needed to achieve the desired behavior and results. This means that training and development professionals must know the concepts, principles and techniques required for “managing” change. I have put “managing” in quotes because it has a twofold meaning. It not only means to decide on the changes to be made but also to get the acceptance of those involved in the change.

This chapter is written not only for training and development professionals but also for line managers. It is important to emphasize that the training and development professionals can control the determining of needs and the learning content. But it is also important to emphasize that changing behavior is under the control of the manager whose subordinates were trained.Therefore, these concepts, principles, and techniques are equally important to trainers and managers.

See All Chapters

See All Chapters