Event Date: 07/29/2015 (2:00 pm EDT - 3:15 pm EDT)
SARAH SHAFER: Welcome to today’s webinar, Training that Delivers Results: Instructional Design that Aligns With Business Goals hosted by HRDQU and presented by Dick Handshaw. Today’s webinar will last around one hour and 15 minutes if you have any questions, you can always type them into the questions box. We will be answering questions as they come and live at the end of the presentation or as a follow-up by email if we do happen to run out of time. My name is Sara Schafer and I’ll be moderating today’s webinar.
Dick Handshaw, founder and chairman at Handshaw Inc., is a consultant, speaker, author and champion for innovation and quality and performance improvement and instructional design. He is a pioneer in the field with 35 years of experience as a learning and performance improvement professional. Dick has served as a consultant for many organizations to help them establish a result oriented training practice. He founded his own training consulting firm in 1985. He and his staff developed the Handshaw Instructional Model over nearly 30 years of practice. Welcome, and thank you for joining us today.
DICK HANDSHAW: Good afternoon everybody. This is Dick Handshaw. I really want to thank you and HRDQ for putting this on today. I’m going to start out by finding out who I’m talking with for a minute. So, I’m going to launch this first poll. I want to find out how many instructional designers we have, how many performance consultants, and how many are both. I’m really interested especially in the both. So if you go ahead and complete that survey now for me. We will wait just a couple of seconds until everybody finishes. About half of you have voted so far. Let’s see, so far the instructional designer and both are running about neck and neck and only about 16% are just performance consultants. A few more seconds while we finished the voting. Okay, so, I’m going to assume that 44% are instructional designers, 41% of you say you are both, and 14% say you are performance consultants.
I want to give you a little idea of how this presentation came about. What you see on the screen now is the Handshaw instructional design model. I just called it that because I’ve been using it for 35 years and I figure, okay, I can call it that. What I also did was I decided that the only thing wrong with instructional design models is that they didn’t have performance consulting in them. People have told me for about 10 years, oh you should write a book. And I said I have nothing to write a book about that there aren’t 400 books about instructional design already, until it occurred to me that this is something people really haven’t talked about much. And so I want to give you a little bit of a rationale of why I thought it was important to include performance consulting in an instructional design model.
Back in the mid-90s I happened to meet in Pittsburgh, Pa., Jim and Dana Robinson, who wrote now three books on performance consulting along with about five other business books. What I learned when I met them was that I was probably designing a lot of instruction for my clients that may or may not work, may or may not be a waste of money, and it wasn’t really my fault because I was doing what they asked me to do. But it wasn’t really a good business strategy to be taking money for things I didn’t know were going to work or not. So, I learned performance consulting. I actually hired Jim Robinson to come and do a workshop for my company in 1996 and now we do those workshops for other people. But the reason I thought it was important to include that in our instructional design model is it’s part of the analysis as you can see I’m talking about an addie-model here and it’s really important that we identify what our client’s business goal is. I know your clients don’t usually ask you to do this. I know what they ask you for, they call you up in a say I need a two-hour e-learning program on something. Or I need a one day workshop on something. I know I still get those requests all the time, too, but, the point is, we are a much better value added partner if we know what they are trying to accomplish from a business perspective and if we become responsible to help them do that. So, I’m going to talk a little bit about performance consulting in the beginning here. Then I’m going to talk more about instructional design. Since this is an addie-model, you know there isn’t one addie-model, almost all of them out there are addie-models. Anything that has analysis, design, development, implement, evaluate and it which almost all of them do is an addie-model even if they say they are not one.
So, we are going to focus first on the first three steps and I want to talk also about some of the criticisms of instructional design models. People say well, you know, it takes too long, I don’t have that much time to do all that, my clients don’t want me to do all that. So, we developed what we call our cost versus risk rule, and the only thing wrong with instructional design models is that people try to use them the same way every time. And that doesn’t necessarily fit. So, to give you an idea of how to save time with your instructional design model and still do what’s important try the cost versus risk rule. Weigh the cost of doing a step versus the risk of not doing a step. I’ve got an exercise at the end here where you will get to practice that, but I’m going to talk about that for each step we go through is how much is this going to cost you, and what would be the risk that would make you not want to skip the step? What would be a situation where I don’t really need to do an audience analysis this time, it’s the same audience, I’ve worked with them before, and I’m going to drag out my old audience analysis. So, again, the whole thing I’m trying to do here is I’m trying to save you time, because I know everybody’s pressed for that. And I’m trying to make your learning more accountable to the bottom line results for your client. So, in looking at the ways to get into performance consulting, there’s two ways: proactive and reactive. Reactive is the one we are all used to. Somebody calls up and says I need a training program on such and such. The other way you’re probably not used to and that is proactive meetings with your client probably once a quarter might be all you can do, maybe every other month, but the idea is just call your client and say do you have 30 or 40 minutes to meet with me? I just want to see how your business is going. And I know as learning people we think we are not really supposed to do that. I have been a business owner for 30 years now and if somebody on the street that I didn’t know walked up to me and said hey, do you have 10 minutes? I just want to see how your business is doing, I would probably talk to them. Business owners, people who are really in charge of a business, cannot resist the opportunity to talk about their business. So, it’s not as hard as you might think. Give it a try.
Why use performance consulting? Well, I had this, Tom LaBonte day is actually a consultant who introduced me to Jim and Dana Robinson years ago. I went to visit him at his new job in Pittsburgh where he had 125 direct reports, all training people. And Tom started the meeting off by telling them that they’re all doing a good job. The only thing wrong with training was that they were developing too much training, which confused the heck out of everybody. What Tom meant was is that sometimes your developing training to solve a problem that can’t be solved by training. So, my suggestion is the answer to your busy schedule and how to be able to get everything done is don’t do just training for every training request that comes your way. I’m also not suggesting that you say no to those training requests. I’m asking rather that what you do is refrain that request and turn it around. So performance consulting then, I have a definition of it here, it’s a process that produces business results. As training people you may not be asked to do that but the performance consultants in the group know that you are asked to do this and I’m sure you do that fairly often. And, the idea is that you want to maximize the performance of people and organizations. Now we are used to looking at the performance of people. The thing we are not used to doing in our profession is looking at the performance of organizations. So what I would suggest to you is that almost all business problems require multiple solutions. There is hardly anything out there that I have ever come across and in 35 years of doing this work that is going to be solved by me just doing the training program. I don’t think I’m that good and I just don’t think it ever happens. So, we are also going to focus on the organizations as well as the people, and that’s what makes this a little bit different.
Right now, I’m going to launch a poll again because I want to find out how many of you have access to the true client? The true client is the person who kind of owns the business and you may feel ambivalent about whether you have access to senior leaders who are in charge of businesses especially if you are in a large organization. So what I would like to find out right now is how many of you feel like you do have access to people at that level, senior managers, people who are in charge of the business and how many of you say I really can’t get access to those people? Okay were turning in about 60% yes and 40% no right now out of 75% voting. We will wait just a little bit more. Okay so we’re still looking about 60-40, that’s good.
So here’s what the true client is. They are the owner of that business, they are responsible for the performance problem. They know the current business goal. If you are talking to someone ask what the business goal is and they don’t know, you’re probably not talking to the true client. What you should do in that situation and say would it be a good idea to find that out? Can we talk to the person who knows what the business goal is? They know what internal and external barriers there are in their organization. They know strongest and weakest performers, which is really important because getting to some of the strongest performers is how you find out what kind of performance is going to solve the problem. Too often we are asked by somebody get people to do this performance and this is going to solve my problem, and you develop training for people to get to that performance. Nothing changes, and then somebody says well that training program did not work. It’s not really your fault. If you can’t get to the best practice that leads to the business result all the training in the world is not going to help. So you need make sure you’re doing the right thing. Looking at the best performers is a great way to do that. Your true client can also answer questions as you conduct your proactive and reactive meetings. Most importantly they can make decisions that affect the business results. So I see that 40% of you out there say that I do not have access to this person. My advice then is to get with some senior manager, your supervisor talk about the need for performance consulting, there are books out there, there are workshops out there, but most importantly you need to get that person to agree with you that maybe you need to be talking to the people who can make these different things happen and get your supervisor or manager to help you do that. It’s one of the hardest thing to do, but it is hard to be responsible for true change and get results out of your training if you can’t do that. So one of my first big pieces of advice if you want to get results from your training is make sure you are talking to the true client.
So back to the instructional design model. One of the tools that we use as a performance consultant is a gaps map, a needs or gaps map. It’s a fairly simple process. Here’s what a gaps map looks like the four boxes on the top are where we identify gaps. The three boxes on the bottom is where we identify causes for the gaps. So the gaps are fairly obvious. We just want to look on the left side at what should the business be doing? That’s that business goal we talked about. What are the current business results? And there should be a gap in the middle. That’s what we want to fix. So, then we look on the performance side. There is a performance should and a performance is, same thing, what performance would get the desired business goal? That arrow that points from business should to performance should is saying that this is very important here. It’s saying that that business goal should be the driver for the performance. So, if we are going in and taking the training request, and trying to train to a certain performance, we are not linking it to the business goal, we are just taking a shot. We are taking a chance that it’s going to fix things. Very important that you identify that business goal. So, you can find a gap between what performance should be happening and what performance is happening. Then you want to ask questions about causes. Here’s the thing that so many of our clients come to us with all kinds of solutions build me training for this, build me training for that, build me some kind of communications tool. And they haven’t really looked at the causes for their problems yet. So we categorized causes in three areas here. There are things that happen outside the organization things that happen inside the organization, and we say some of those our client can control and some of those our client cannot control. And then there are factors internal to the individual. Now that last box internal to the individual that’s where we usually live. It is skills and knowledge or maybe just things they like to do because people don’t, the reason people don’t do a certain performance is not always because they don’t know how, maybe it’s because they feel they are not being paid, there are some other reason they don’t want to. So, very important to identify these causes. Once you begin to identify causes and do this with your client, you may see some lights go on not maybe this is not worth wasting training time on. Maybe this is an entirely different cause.
So, again, there is a lot to performance consulting. It might take some of you quite a while to figure that out and learn to do that. It probably takes a couple of years to transition your organization to practicing performance consulting. But I think if you truly want to get results from your training, spend some time in that arena first before you begin to design instruction.
So, with that said, there are couple of things you have to have lined up in order for you to have performance consulting work for you. You have to have access to that true client. You have to have the time to conduct analysis, and, you know, on time, sometimes this is just two or three days; at the most it could be a couple of weeks. We have to get our clients to agree that it is important enough to have time to do that. And I also want to say that not every project requires performance consulting. That cost versus risk rule again and when I wrap up I’m going to have a little exercise where we can practice that. So, you also have to have the influence to suggest those nonlearning solutions. Those things that you find out that are wrong in the organization probably 80% of performance has nothing to do with training needs. It has to do with other things like management, setting expectations, rewards, coaching, bad systems or information. There are so many other things that get in the way of performance it’s hard for us as training people to try to get a training program when, believe me 80% of performance issues have nothing to do with training solutions. And finally, I think it’s good to have a comfort level with ambiguity because as you start doing more performance consulting with your clients, things might get a little bit murkier before they get clear, but if you follow the should is cause logic, what should things be, what are they, identify the gap and then find the causes for the gap. That’s going to lead you to better decisions about your solution.
So, let’s suppose now that we have identified a need for training. Maybe there are other solutions in this whole problem solving as well. But we know there is at least a training need so now we’re going to continue more with traditional instructional design here. And I’m going to launch another poll here. I want to find out how many of you are doing task analysis at this point. How many of you do a task analysis with your projects? This is my favorite question to ask people. Yes, the no’s are slightly winning here. We have about 70% of the vote in. And we are 60-40, no. A few more votes coming in. And that’s about it. So, about 60% of you are saying that you don’t do a task analysis with all of your projects or most of your projects. So, for those 60% of you, I have a question for you. How can you design training if you don’t know what people are supposed to do when you’re finished with them? And I know what most people say in this circumstance if I was looking at you face-to-face you would all tell me that you really just don’t have time to do that for your client doesn’t want to take time to do that. The one thing I think out of this entire process today that will save you time is to begin doing task analysis with your projects.
So, I’m going to show you what I mean by task analysis. It’s not something that should take an awful lot of time and, in fact, I’ll give you a really nifty high-tech tool to do that with. The high-tech approach that we use is sticky notes. We take something like 3 x 3 sticky notes or 4 x 5 sticky notes and we write a task on each sticky note. Each task should have a verb and an object. And that is the beginning of so many things that you’re going to do in the instructional design process and them we will organize the sticky notes on a whiteboard or on a window or wall and we’re going to organize them into being either a procedural analysis which I’m showing you here. So many things that we do are procedures. First I do this, then I do this, then I do this, then I do this and I have achieved the instructional goal. Sometimes there are knowledge tasks required. This one is strictly knowledge. It would be something like reading a paragraph or writing a paragraph where it’s a hierarchical thing and that I have to be according to this drawing, I have to be able to master steps one, two, and three before I can master four, I have to master four or know five before I can do six and if I can do four and six I can master the instructional goal so, if you think about it almost everything in life is a combination of these two. We are doing steps across the top but there might be things we have to know in order to do those steps. And so that’s how we come up with this task analysis diagram. It is really a good idea if you can down with either in person or on the phone your subject matter experts and do a task analysis. In fact, I had a client who, for years, was having difficulty managing the relationship with their subject matter experts and I suspect a lot of you do as well. They said when they started doing task analysis with their client all of a sudden there INAUDIBLE were so much easier to deal with because they let them in on it and they let them help design that process in a flowchart task that everybody could see and understand. So by doing this task analysis, you can do it on a whiteboard or do it on flipchart paper and then keep them. You can put it in flowchart software you want to. Pass it around let other people look at it, sign off on it. Here is the biggest thing, one of the most time-consuming things for us when we started in business was doing things over again instead of doing them right the first time. So, once you have identified the task analysis and get most of the people on the project team to agree, yeah that’s how we do that, you will spend so much less time doing things over, your projects will take less time. This is the big cost savings. It shouldn’t take you a long time. In most cases it should take you 6 or 7% of your total project time to build the task analysis. It will pay off hugely. If there is any one step that I would recommend you take away today, if you’re not doing with the 60% of you who are doing it give it a try. It really does work.
So, here is a case study were my staff had a project that had 4500 hours of development work. It was a pretty big project. It took over a year. Six and a half percent of that was allocated to task analysis. Okay so was 270 hours, but it was still only six and a half percent of the total budget. We correctly identified the scope and kept the project on budget for the entire time. No INAUDIBLE at all in a 4500 hour project. On the one single thing that we can attribute that to is a good task analysis. Here is an example of one I think you can pick up pretty easily, putting a golf ball. See can see across the top there are sequential steps: plan the stroke, assume the stance, and these are things, all you have to do is watch a golfer step up there and do those things, and write them down and say is this what I just saw you do? And the golfer will say yes, you’ve got your procedural task analysis done.
The harder part is the knowledge part, so, let’s look at plan the stroke. In plan the stroke, you’ve got to combine direction and speed in order to get a plan. And there are things you would do under predicting direction, to figure out what the direction is. And there are things you would do under the speed. To figure out speed is, you have to ask questions well how do you do this, how do you do that? You have to ask those kinds of questions if you are going to get answers to the knowledge component of what it takes to plan that stroke. And that’s all there is to doing the task analysis. It takes a little practice but it’s not that hard, and believe me, once you start doing a project with task analysis you will never change.
What I would like you to do right now is ask anybody out there who would like to make a comment. Is there someone who has either had a good or bad experience with task analysis? I would really like you to tell us about that right now.
Okay, so, I see somebody out there who has been doing task analysis for about two or three years now and doesn’t say how they learned that, but says that what they notice is pretty much what I was saying is that before they started doing task analysis they would have hiccups in their content and they would have to constantly stop and make changes and make changes, and make changes. And they couldn’t get finished with training programs. The task analysis really seems to have helped in this case.
On this next line is a sample task analysis that we did for a recruiting class. And this guy, an interesting guy, he was one of the best recruiters in the country, an HR recruiter. He taught workshops, but he really didn’t know how he did what he did. Once we worked with him on a task analysis, and this was a really extensive one, it probably went around the room one and a half times, but this is what it looked like. And we were able to write down and identify for him what was it that made him one of the country’s top recruiters. Before he had gone through that process with us he really couldn’t sit down and tell you that, and he found it really difficult to teach people because he really couldn’t identify it. So, once again, the task analysis helps you identify what it is that people have to do when you are finished with them. And it shouldn’t take that long.
We are going to move on to the design phase now. The main thing I want to talk about in the design phase is performance objectives. And I am going to launch another poll here. I’d like to know how many of you are currently using performance objectives to help you write testing instruments. I know a lot of people out there write performance objectives, that’s what I want to know. So, if you use them to design tests, give me a yes. And if you write them, but you don’t necessarily use them to design your testing instruments, that would be a no. And what I’m seeing now is about half the votes in, it’s another 60-40 split. About 60% of you say yes, I use them to design testing instruments. And about 40% say no. So what I’m going to say to this is you know, I used to get really tired of writing performance objectives because I didn’t know what I was supposed to do with them. And I kind of figured it a waste of time. But, the performance objectives, and there’s a specific part of them that will really help you design your testing instruments. And I see that we’ve got about 70% of the vote in yet we’re still looking at about 60% yes and 40% no.
So, go back to the slides now. And I’m going to show you a performance objective, and this is Robert Gagne’s five-part format which I’ve been using for a long time. I find that there’s a good reason to do five parts instead of three. So the situation is simply the stimulus you had for why should you perform the task that you are going to do. The task from our task analysis says recommend the best product mix for customer. So you can see we’ve got a verb we have an object, recommend what? Best product mix. And so the situation is, okay, we have a customer. We want the student to be able to recommend what the best product mix is. Here’s the magic: How? By asking probing questions, presenting benefits, and handling objections. And we’re going to say the constraint is that has to be done with every customer. So if I look at that and I want to design a testing instrument, and I look at that and I go, well, I can’t do that with the multiple-choice question. Somebody has to demonstrate to me that they can ask probing questions, present benefits, and handle objections. So somehow that begs for a role-playing situation, and I know that I’m going to use a rubric or checklist, I like to just call them checklists, to measure that with. That person only has to do three things. They have to ask probing questions, they present benefits, and they handle objections.
Now there are a lot of ways you could define the correct way to recommend best product mix. But when we did our task analysis, we found out that those were the three subtasks under recommending a product mix they fit easily into the action portion and it makes it a very measurable way to find out if somebody has achieved this performance objective.
I am going to look at another quick example here, This time we’re going to turn a customer objection into a sales opportunity. And, we’ve got an angry customer this time with an objection. They must have just talked to the company that upgraded my Internet line, because we upgraded it yesterday and we were out all morning. So, we got angry customer, learner will be able to do what: Convert? What are they going to convert? They are going to convert the objection into a sales opportunity. Again, here’s the magic part, the action: How? By restating the customer objection, offering alternatives, identifying a solution that satisfies the customer need. Once again there could be lots of different ways, different companies, different organizations may define. They have different processes for dealing with an angry customer, but the nice thing about this performance objective is we know exactly how we’re supposed to do it here and once again all we have to do is look at this action portion, and we know exactly what kind of testing instrument we have to design and, we know how to measure it.
So, I am very big on making sure that if I’m going to do anything in my instructional design process, I’m not doing it just to put it on the shelf, I’m doing it because I’m going to use it for something. I find there are lots of other uses for performance objectives. Yes, some of us show them to our learners. In most cases I find learners don’t even read them. I think they’re very useful if you are designing instructor-led training for your instructors to know exactly what is expected of learners when they’re finished with them. It’s great for that. But, the very best way is if I’m trying to design level II measurement, that is I’m measuring how much people learned. I can look at my performance objective when it’s written in this way and I know exactly how to do it.
So, we have been talking about instructional design stuff for a while. We’re kind of going back now to more of a performance consulting step and called a blueprint meeting. This is kind of something we added to our instructional design model. We review this model annually, and every time we develop some good ideas during the year we add them to it. This is something that probably has been added in the last 15 years. We have a blueprint meeting which looks like this. We try to get everybody on the project team, we want the project sponsor, we want the design team, we want the subject matter expert, sometimes we want some people who represent the learning audience. We want everyone who is on that team to be in a meeting. We used to be able to do these face-to-face, but now they are mostly virtual. It doesn’t matter. The first thing we are going to do is to discuss the business goal. You know, somebody has told us what the business goal is and we want to make sure that we are always cognizant to keep the business goal foremost in our minds. So we will start out restating the business goal. Sometimes somebody in that meeting will say no that’s not the real business goal. Here is the real business goal. Good to know. Always start out a blueprint meeting with the business goals. These meetings will take anywhere from between an hour to two hours, but they are well worth the time spent.
Then we define the training goals that are going to be linked to that business goal. We might even take the opportunity to say there are other solutions. Maybe one of the things we found is we needed a new count plan, maybe one of the things we found is we needed a different staffing model, but now we’re going to go on and talk about training. So we will use those performance objectives that we wrote to show the team these are the key objectives that we are trying to get people to be able to do. Then we will go right next to our measurement strategies and this is how we are going to measure and know how they got there. Really good thing to bring measurement up early on in the meeting and early on in your process so that everybody agrees in the need to measure. You can’t tack measurement on at the end of a project. That never works. We always make sure we have agreement for our measurement strategy in the blueprint meeting, and remember we have not designed anything yet. All we have done is we have analyzed the task, maybe the audience, whatever other analysis we are going to do, we write performance objectives, and then we make sure that those are the objectives and here is how we are going to measure them and then we will tell people how we are going to teach people.
Finally we get to the instructional strategy and the media. We do not mention it’s going to be an e-learning program or it’s going to be an instructor-led program, or how many hours going to last. We don’t do any of that we know what it is that we have to accomplish. I know your clients always want to start with well, I think we need it a 2-hour e-learning program. When they do just say sure, okay. Put it on the shelf, don’t think about it until your design process has taken you to whatever is the best solution. We also discuss the course outline. People have to see some sort of content in that meeting so we try to keep it high-level at this point. And we are always looking for people to push back or add information. We just don’t present things in this blueprint meeting. It’s very much give-and-take. The idea of the meeting. What we want to have happen at the end as we want to have a collaborative idea of what is going to happen what everybody’s going to do, so we are are not continually second-guessing and revisiting our process. So, sometimes we will show samples of learning materials at that blueprint meeting, at least what they are going to look like
If we have a pretty good idea of what the media are going to be, we are also going to ask our team to help us agree on what is the best content for a prototype. We always like to create a prototype of what exemplifies our instructional strategies before we begin and test it before we begin completing everything. So, again, it’s all about finding out where we are going, testing a little bit, and doing a little bit more before we do the entire thing. It’s all about not having to do things over again. And the last thing that happens in the meeting, is we agree on revisions and, again, remember the goal of the blueprint meeting is for everybody to have a consensus on what it is we’re going to do going forward.
So back to the instructional design model. Here is one of my favorite parts here. This is formative evaluation. Once we had the blueprint meeting, we have built that prototype that we have talked about, and then we conduct a learner trial. And I like to go back to my good friend, Thiagi, who has a wonderful definition of formative evaluation he says, formative is to improve and summative is to prove. So, all we are doing when we do formative evaluation is we’re evaluating how well the learning strategy works with our actual learners and there are always people on the team who want to second-guess this. There are always people who want to say I think this’ll work, I think that will work, and my answer to that is why guess when you can measure? It’s so easy to measure with a learner tryout, we get about six people we get people who are actual learners. We don’t want subject matter experts, we don’t want people who know the process, we want the people who we would be teaching. If it’s E-learning it’s really easy we get that sit down with 20 or 30 minute modules of an e-learning program and we go through it with them and ask them questions. We asked them to think out loud while they are going through the course. They will give us great information. In my book I say that learners are your best design consultants you never knew you had. It’s really interesting if you ask these people for their opinion of a prototype, they really will give you great information. They will tell you things that you could not figure out, and if you have questions about is this going to work better than that, just ask them and I promise you out of six or seven people you will see a trend develop. You don’t have to do this the 50 or 100 people in order to get trends. I’ve done it with as little as six or seven people, that’s all we ever use and we always get trends and learn a tremendous amount. A learner tryout takes about a half a day. If you think about it you’re doing 30 minutes with six or seven people, even if it’s an instructor led course we can teach 30, or 40, or 50 minute module of that course, the six or seven people, and get great feedback. So learners can be your best design consultants. Once we’ve done the learner tryout, we will, using the prototype, then we will build everything else based on what we learned about the prototype. And the last thing we do is, the third bullet here is to conduct a field test. You know, everybody has to give whatever they have designed to a final audience at some point. And we simply ask our clients to give us about 20 people that we can ask questions to after they have taken the training. We give them a little something to write things down on as they are going through it. And then we interview them afterward and you’ll find if there are typos or we might find some trends and some questions that don’t work in our testing. We might find some areas that we did not teach very well. That gives us an opportunity to make final revisions before it goes live.
Going back to the cost versus risk rule. If you have a fairly small audience, you probably don’t need to conduct a field test. But in our case, sometimes we are sending out a compliance course to 20,000 people, and you better believe I want to know how that course is going to work on 20 people before I send it out 20,000 people. So, remember the cost versus risk. I think in a lot of cases if you are looking at should I do a learner tryout, have you done this type of instructional strategy 30 times before? Maybe you don’t need to do one. It is a new strategy? A new creative thing that your team just came up with? I really want to try it out with six people and learn what I can before I build the rest of the course. So, it’s all about that cost versus risk in trying to keep from doing things over again.
I’m going to launch a poll to find out how many of you are currently using formative evaluation. So, do you currently use some form of formative evaluation like I just designed to improve your learning programs? Well, we got 60-40 thing going again. Okay, half of you have voted and I am seeing about 60% of you are saying yes, I use formative evaluation and about 40% of you are saying no, I really don’t. Is there anybody out there who wants to share with me an experience you had with using formative evaluation and what it has done for you? And while we are waiting for somebody to answer on that, I have an interesting question here that says what’s the difference between task analysis and objectives for the course? Okay, that’s a good question. What I want to point out here is that every step in an instructional design process, the output of every step is the input to the next one. So the task analysis becomes that input to the performance objective. You will notice in the task analysis I pointed out that every task needs to have a verb and object you’ll also notice in the performance objectives, that every performance objective first has a verb and then an object and then an action or how it is supposed to be done. So you can see that that task forms the root of that performance objective. So it’s very important to note how in instructional design, the output in one step always becomes the input to the next.
Okay, so somebody is bringing up Six Sigma. Six Sigma is certainly is a process that uses formative evaluation-type philosophy. I think that, I’m going to talk to you about a case study in a minute here, back, in terms of whether you should use formative evaluation or not it’s cost versus risk. You know how many times you’ve done a particular solution before and how well it works. Or if you are into territory where it doesn’t work, then I was would suggest that you find six sample learners, design a prototype that exemplifies your instructional strategy. Sit with those learners one at a time and ask them what they think and what they are doing as they go through that course. They become your best design consultants. So just to show you where we’ve used this and what it has done for us as a company, here is a revised curriculum project that we did for a bunch of front-line workers. They had some software that ran all their processes, we completely redesigned their training program for them because it wasn’t working for them. So it’s very important that we find out, since they wanted to redesign it, to make sure that it worked. We took a couple of prototypes and tested it on some sample learners first to see if it was working better than what we were redesigning. So we also had a couple of different opinions of different strategies to use in the redesign. And we had the usual situation where some people said I think this one will work better, some people said I think this one will work better, so we tested both strategies. And we found out there was a clear preference among the learners where one of the strategies worked better than the other one. We did not have to guess. We were able to identify a lot of functionality changes. There were a lot of things that we thought were good content decisions or good process decisions which were not. And people told us. Those changes would have rippled through all those subsequent modules in the course we found it in the module that we tested and that saved us about 80 hours of work, by fixing those things one time instead of in every subsequent module.
So, one of the things that we always test in formative evaluation is our implementation plan. With this particular client, they had a real problem with implementation. They had a particular group of instructors who are saying you know, we could do a better job with these scores if those people in instructional design would design a better course. It’s not us it’s the bad design. So our client was really nervous that what’s going to happen we put this out and instructors don’t like it? We said, invite the instructors to come watch the learner tryout. When we did that, it was so interesting to watch the instructors get excited and see how this new program works. They saw the redesign, they saw most of all the reaction, the positive reaction of the learners, the input from the learners about what could make it better, and the instructor group was the biggest fan of the new redesign and our client breathed a huge sigh of relief.
All a wonderful output doing the learner tryout which took us about two days. So, again, what’s the cost of it? Okay, this is that $4500 project? It was 150 hours but it was only 2.7% of our total budget. That 2.7% saved us from having the instructors not criticizing it because they got to have input and also the students, the learners, got to see the course and went out and told everybody how good it was. And we got a lot of great information from them on how to make the course better. So, it was a huge win for an investment of 2.7% of our total budget.
I’m going to show you an example of a fairly simple instructional exercise that was in an e-learning course. This was a company that sells aerial lift platforms. You can see the picture of the lift platform and it was showing you a two wheel steer, a four-wheel steer, and a crab steer. Okay, so if you look real carefully in that dark shadow were those black tires are, you can sort of see the wheels changing. And of course the picture changes size. The idea is you are supposed to be able to see the difference from a four-wheel steer to a two wheel steer to crab steer. What you’re looking at right there is a picture of the wheel steer. Okay so that’s one option for how to solve this problem. But now we’re going to go to an animation that we also thought might help. And in this case here is the two wheel steer and then here comes the four-wheel steer and then lastly, here is my favorite one, I wish my Miata could do this, the crab steer. So the client wanted to use the photo, I think mostly because they took the photographs. We took these two approaches and put them in front of six or seven learners and do you think anyone of those learners preferred the photo over the animation? Well, no. So, we didn’t really have to argue this with our client anymore. The client wants the learners go oh yeah the animation works better. End of story.
That’s a fairly quick powerful example of how conducting a learner tryout can keep you from having to deal with people. You just let the learners figure it out. Again, why guess when you can measure? So, all this so far has been done so that when we get to the production or the development phase of the project, we are not doing things over and over again. We have good testing instruments because we have good performance objectives. The performance objectives work because they came from a task analysis, and the task analysis works because it is linked to performance that’s required to achieve a business goal. So, this is where we as a company, and I am sure you do to spend most of your money actually making learning products. The idea is to do this as efficiently as possible and not have to do things over and over again.
And so here’s the field test which is the second part of the formative evaluation, the last thing we do before we go into implementation. And finally, in implementation I think I have seen more projects fail because of bad implementation than any other reason. Lots of times people develop great learning programs and they just kind of fail to do in implementation plan or they fail to see one through, they failed to follow up on that, and it’s not a very expensive step, but it certainly keeps, it’s an insurance policy to make sure that what you think is going to happen or what you had it designed to happen actually does happen. And then also in implementation is where the actual measurement begins to happen. I worked with Jack and Patty Phillips from the ROI Institute and I have their five levels of measurement and evaluation, very similar to Kirkpatrick, I like them both. The thing I like in this reaction, number one here, I think it’s really important to pay attention to do people, I don’t want to just know their reaction to it, but I want to find out do you think is going to work on the job? What’s the value of the program to you as you use this on the job? How will it help you? That’s the most important piece of reaction data. In the learning, we’ve already talked about how to design the learning measurements from the performance objectives and again, if your performance objectives are written off a task analysis you’re likely going to have the right things in their and you’re going to be able to develop a level II measurement instrument that work. And then number three is application. Do people actually use this on the job? That’s the kind of thing where you have to let the training program happen and you come back a few weeks or couple of months later and you see if you can observe people or managers tell you that they are using those skills on the job. It’s somewhat, it’s not always a real objective, sometimes it’s somewhat subjective but find out what you can about the actual application is really and, here’s a big one, the business impact. It’s a little bit hard to measure, but, you know, we started this whole conversation off with a business goal. We want to get results out of our training. So it really helps to see did that training work on the job and did it have an impact on the business? We knew with the business goal was to begin with so should be able to go out and measure whether we achieved it or not. And then the return on investment simply measures did we spend more on the training program than we did on the results we got from it that would be a negative return on investment. Or did we spend a little bit on the training program and we had a really big payback on the business side?
You know, people are sometimes scared to do this sometimes. People say well they don’t measure because they don’t want to find out. In those instances where you are asked to develop training, you think training is not the solution, it’s not going to solve the problem. Sometimes it’s a good idea to plot that negative return on investment so you can say well, what did we learn here? We learn that training was not a good investment of our money. Sometimes you want to show good things and sometimes you want to show bad things. So keep in mind that having a negative ROI is not necessarily a bad thing. Sometimes it proves a good point. And here is my favorite chart and then I’m going to turnovers that last exercise where you get involved like I said. Here’s the INAUDIBLE that Jack and Patty Phillips from the ROI Institute did. What we as training people measure and what executives want to see. So more than half of us measure the INAUDIBLE 22% of executives care. 32% said I’m appalled that only 32% of us, according to the study, measure our learning and again only 28% of executives care. Now the application we continue to drop only 11% of us do that. 61% of our executives want to see that people use those skills on the job. And then business impact, 8% of us measure that. 96% of executives are interested in the business impact of that training program that you just spent your time and the company’s money on and only 8% of us measure that. 4% of us do an ROI study, and even Jack and Patty say probably are not going to do an ROI study on more than five or 10% of your projects. 74% of executives are interested in that. So I think this is a very telling chart.
And now I’ve got that exercise that allows you to apply that cost versus risk rule that I talked about at the beginning of the program. In this case we have a scenario for you. It’s a software upgrade that our company is doing. And you are the training people and your clients and are upgrading their infrastructure, investing a lot of money the new software system just report frontline personnel, measure initiative for your company and you’ve been asked by several managers to develop training for new systems. Okay, new systems, going to need training, got it. The question for you all then: Do you need to reframe the training request through the use of reactive performance consulting? So remember that’s the reactive meeting where someone has said, hey I want training. Do you think you need to reframe that request or ask a bunch of questions to find out if training is really needed? And I’m just going to ask for comments here. I would like you to not only give me an answer to the question, but give me some idea of, I’m reading some of these questions here. Okay, well obviously somebody says I’m probably not going to go to a lot of trouble here to reframe the question. And, I would agree with that because we know that it is a brand-new system, we probably need training.
I’ve got another question for you though. Is it necessary to identify the business goal for this major initiative? What do you think about that? Again just write you are comments for me please.
Okay, the answers I’m getting here, I’ve got a couple of people say they probably think it is necessary to identify the business goal. In the comment I would make there is seen the first bullet they are spending several million dollars in new software. We can teach people how to use the software, but there is probably a reason the company invested several million in new software. There’s probably some business goal they wanted to achieve by that. And really, this is the case folks where training has a huge impact on where the people are going to achieve the business goal are not. So if we don’t know the business goal, we are just kind of shooting in the dark. Teaching people the best we can to use the software better that we know what they’re supposed to do with it from a business perspective and then we can reinforce that in the training program.
In the next question, same scenario. Would you measure the business impact or the ROI in this project? Would you take the time, remember cost versus risk, is this a project where I would want to ask my client to measure business impact or ROI? And, you know, you’re going to need help to do this. You need resources within other parts of the company other than training to get updated to see if you had a business impact. That’s not something you can do alone. Something you have to convince your client to do.
Okay, I’m seeing absolutely, that’s a good answer. Definitely yes. Always. I don’t know if I would agree with always. But definitely yes in this case where so much money was invested. I think if you try to do measurement always we get the project sometimes that are those checking a box training project where somebody says I don’t really care where the results are here I’ve just got to show some regulator that I did training. Probably not a good place to spend your measurement dollars.
And so now we have another scenario. Your client is introducing an entirely new sales process for 1,200 performers across the enterprise. Okay, so this is salespeople now. There is a lot of mistrust and ambivalence among the sales managers. Well, okay, salespeople. Hey, and I’m a salesperson myself. You’ve been asked to develop a series of instructional training programs for this new sales process. So, question for you: Would you really need to write performance objectives? And tell me if you would, why or why not? Just write your answers in here. Okay, I’m seeing a couple of people say yes, I need to use performance objectives because it’s really important that I measure people in this situation. New sales process, somebody’s going to be looking to see if you’ve got a sales increase. So you’re not going to get a sales increase so you can measure performance change and so that makes the measurement portion of this, the level II measurement portion of this, really important and you really need to have those performance objectives to be able to write better tests.
Next question, on the same sales process, other than task analysis what other types of analysis should you conduct? We didn’t really talk about this. In this presentation we only talked about task analysis. We talked about performance consulting as an analysis but are there any other types of analysis that you do out there that you think would be important in this particular situation?
Okay, change management. Somebody said, absolutely. You have a huge change here, so change management is important, but the type of analysis you would need for the change management is what we call either a moving environment analysis or a culture analysis. So, in order to make a change you have to find out with the current culture in the organization is and then identify the culture that you that want in order to make the change happen. This also goes along with implementation plan. You can teach people different processes but if their managers or the corporate culture doesn’t support that new process or new change it is not going to be successful. So, yes, I would say in this case where you have the second bullet mistrust and ambivalence, points towards a change management as one of our people pointed out, and that points toward analyzing the corporate culture. Managers have more to do with reinforcing good behaviors and set of training any other single thing. There have been a lot of research studies that prove that.
So, on the same sales process is it worth the effort to conduct a learner tryout? Why or why not? And just type your answers in again. Somebody says ideally you should do that. Maybe conduct an ROI. And then somebody says what if they prefer to bury their head in the sand? And I think that was referring to the ROI and the measurement on the last one. Okay and I think what I’m getting here is a consensus that yes, people think it is a good idea for a learner tryout. Chances are with this situation with the mistrust and the ambivalence among the sales managers, you need to kind of find out how your strategy is landing before you develop the rest of the course. You’re going to get great feedback. Salespeople are very upfront. They will tell you what they like and what they don’t like. They are the first audience I would turn to to conduct a learner tryout and find out if my strategy is working or not. They will tell you.
Now, this project has a tight schedule. You need to conduct a blueprint meeting. Remember that the meeting that we will do after the design to get consensus from the design team. It takes about two or three hours in some cases, do we have time in this tight schedule to conduct blueprint meeting? So tell me he would do that are not and why. Yes, and get input from sales managers on the process. Absolutely. So, I think the consensus here is yes, doing a blueprint meeting is a good idea especially here where there’s already a lot of mistrust and ambivalence. That’s a huge red flag to say there’s high risk that I’m going to have a lot of people not accepting whatever solution I with. I want to have them hash that out in a blueprint meeting before I develop my training program before I go through all the trouble to develop the training program. So this is a case whenever you have high risk like that with people probably having their own process or different processes, they may not trust the entirely new one and to get everybody together and make sure you agree on what is going to happen and where that project is going to go before you go through all the work is it really a good idea.
Going to open this up to questions now. I’m also going to mention that everything that has been in this is also in this book Training That Delivers Results. And, right now I would like to open it up to any preformed question that you have as we have about three minutes here.
SARAH: Okay, perfect. Thank you so much, Dick. We probably do have time for, will probably get a few questions in. But while we wait let me just share a little bit about how you can keep in touch. Dick’s contact information is on that slide right now so you can connect with him after the session. He has some great resources there as well. And then also you can connect with us. Please connect with us on all of our social media and also register for our weekly webinar Wednesdays at HRDQU.com.
It looks like our first question is coming from Pat. How do you convince clients to spend time and money on measurement?
DICK: That’s a tough one. I would say the thing that will get you the most success in that to start early. We start in the design phase with the testing plan then we take that testing plan to the blueprint meeting and we try to sell it at the blueprint meeting. We asked people if you have this business goal, it’s important to get this business goal. You do not need to measure what’s working, what’s not working, and usually get a consensus. So, the biggest piece of advice there is to start early.
SARAH: All right, great, thank you. Our next question is coming from Lisa. What are some good tips for influencing stakeholders to allow time for task analysis upfront?
DICK: I think in a lot of cases your stakeholders don’t necessarily need to make the decision on that. I think you and your manager need to make the decision on that. What is going to convince stakeholders is results. So, start with those champions, those clients, that you know you have an affinity for working with, and you are likely to be able to work with in a good way and show the results. Do the task analysis the subject matter experts will sing your praises. They will tell the other people who have to sign off on it how helpful it was. Task analysis is one of those things we hand clients and they look at it and they said wow, I’ve never seen the entire process in one piece of paper like this. This is great. I would suggest you don’t always ask for permission. Get it done and then let the results speak for themselves.
SARAH: All right great, let’s just do one more. This one is coming from Greg. Do you take learning styles into consideration and at what design point?
DICK: Good question, Greg. And I do have a fairly outspoken opinion on that. There is not much research that says that learning styles have any impact on learning. Yes, we all may have certain preferences and learning styles. But according to the preponderance of research, even if we’re given learning that is not in our sweet spot of learning style, research has shown that people learn better in their preference as opposed to a style that is not their preference. So, based on the research and my own observations, I haven’t seen that learning styles make a whole lot of difference. Now, the thing that does make a difference to me is what people will say in the learner tryout. Certain preferences there that might work and show evidence that they actually work I will go with, but I’m not, in my career I’ve learned that learning styles are not something I spend a lot of time and money looking at.
SARAH: All right, great questions and I wish we had more time, but, Dick would you just like to add any final thoughts before I go ahead and wrap up this session?
DICK: Well, Sara, thanks to you for one thing. Thanks to HRDQ for having me. I hope you always take at least one thing I have discussed today and give it a try. If you want more information on this you can go to www.dhandshaw website and click on the resources tab. There’s a bunch of free resources and instructions on how to do things there. But, just do me a favor and at least take just one thing from today and try it.
SARAH: All right, Dick, thank you again so much, and for the attendees, if we did not get to answer your question, you are going to receive those email responses probably mid next week from Dick. So we appreciate your time, and we hope you found today’s webinar informative. Thank you.
Most instructional design models don’t include performance consulting or enough emphasis on measurement to deliver results consistently. Training professionals resist the idea of being put into “order-taking” roles, but many have a hard time moving beyond this approach to find a more comprehensive, results-based approach. It can be challenging to include consulting and measurement on top of your already busy schedule of training development. What training professionals need is one streamlined process that helps weigh their priorities to determine what really must be done in order to achieve real business results.
Developed by one of the originators of computer-based training, Dick Handshaw’s results-oriented model is systematic yet flexible and works for both instructor–led training and eLearning. Figuring out how to include quantifiable results in your design process can be a challenging and confusing process. This session offers a better way to deliver those results; a way to connect learning solutions with strategic business goals.
Participants Will Learn
- Reframe a training request and analyze performance gaps
- Develop a useful task analysis that saves time and money
- Use performance objectives to develop measurement that works
- Build consensus with a strategic blueprint meeting
- Validate instructional strategies with learner tryouts and feedback
Who Should Attend
- Human Resources Professionals
- Organizational Development Professionals
Dick Handshaw, Founder and Chairman at Handshaw, Inc., is a consultant, speaker, author and champion for innovation and quality in performance improvement and instructional design. He is a pioneer in the field, with 35 years of experience as a learning and performance improvement professional. Dick has served as a consultant for many organizations to help them establish a results-oriented training practice. He founded his own training consulting firm in 1985. He and his staff developed the Handshaw Instructional Model over nearly 30 years of practice. Dick has presented at various international conferences such as Training Magazine, American for Talent Development (ATD) and International Society for Performance Improvement (ISPI).