About HRDQ-U Virtual Seminars
About HRDQ-U Webinars
Great leaders deliver equally great results, including their impact across the organization or sphere of influence. Without impact, leaders are ineffective. Achieving success in turbulent times requires leaders to deliver on five levels of outcomes, including impact and ROI. A courageous leader can deliver on all levels of outcomes in the face of many difficulties, challenges, and uncertainties in ambiguous environments.
This concept is straightforward yet very powerful. Success does not occur until impact is achieved. It suggests that the evaluation of leadership development should not stop at the learning level (to ensure the leader has learned the competencies) or even at the leader’s behavior level (to ensure the right behaviors are in place). Leadership development programs should be evaluated at the impact level.
This session will provide useful, practical tools to help you show the value of leadership development programs including how to measure the business value of your programs and how to calculate the ROI. This session also provides detailed case study examples that illustrate how to measure and evaluate different leadership programs and initiatives. Participants will see what data is measured, how it is measured, and how the data are used to improve the programs and influence more investment in leadership.
Jack J. Phillips, Ph.D., chairman of ROI Institute, Inc., is a world-renowned expert on accountability, measurement, and evaluation. He provides consulting services for Fortune 500 companies, nonprofit entities, and government and non-governmental organizations globally. He is the author or editor of more than 100 books, conducts workshops, and presents at conferences worldwide.
Jack has received several awards for his books and work. The American Society for Training and Development gave him its highest honor, Distinguished Contribution to Workplace Learning and Development. The International Society for Performance Improvement presented Jack with its highest award, the Thomas F. Gilbert Award, for his contribution to human performance technology. On three occasions, Meeting News named him one of the 25 Most Powerful People in the Meetings and Events Industry, based on his work on ROI. The Society for Human Resource Management presented him with an award for one of his books and honored a Phillips ROI study with its highest award for creativity. In 2019, Jack, along with his wife Patti P. Phillips, received the Distinguished Contributor Award by the Center for Talent Reporting for their contribution to the measurement and management of human capital. His work has been featured in the Wall Street Journal, BusinessWeek, and Fortune. He has been interviewed by several television programs, including CNN. Jack regularly consults with clients in manufacturing, service, and government organizations in 70 countries around the world. Connect with Jack on LinkedIn, Twitter, and at email@example.com.
HRDQ-U offers a curriculum of 80+ virtual seminars for training employees in soft skills. Covering topics from leadership to communication, conflict to change, communication to diversity. Enroll your learners in HRDQ-U Virtual Seminars and let them develop soft skills from their home or office.
Level 5 Leadership: Designing for Results
Hi, everyone, and welcome to today’s webinar, Level Five Leadership: Designing for Results, hosted by HRDQ-U, and presented by doctor Jack Phillips.
My name is Sarah, and I will moderate today’s webinar. The webinar will last around one hour. If you have any questions, please type them into the question area on your GoToWebinar control panel, and we’ll answer as many as we can during today’s session.
Today’s webinar is sponsored by HRDQ-U Virtual Seminars, HRDQ-U Virtual Seminars are engaging soft skills training classes, with real-time interaction and expert trainers, and roll your organization’s learners in an HRDQ-U virtual seminar and let them develop the performance skills that they need from their home or office. And on any device, from desktop to mobile, learn more at www.hrdqu.com/virtualseminars.
I’m excited to introduce our presenter today, doctor Jack Phillips’. Jack is a world-renowned expert on accountability, measurement, and evaluation. He is the Chairman of ROI Institute, which provides consulting services for Fortune 500 companies and major global organizations.
He is the author or editor of more than 100 books, including the Business Case for Learning, found at HRDQ store, and proving the value of soft skills. He conducts workshops and presents at conferences throughout the world. Thank you for joining us today, Jack.
So it’s good to be here, and welcome to our session.
Today, we’ve got some very specific objectives we want to meet.
The first one is to think about the levels of leadership.
We, we may surprise you with this, we may not.
But we look at leadership being valued at different levels, so we want to take you through that.
And then think about the chain of value that’s there for any leadership program we’ve ever conducted or any leadership experience that we actually have.
You’ve got some levels of outcomes there, so we’re gonna look at those. And then, the need to show the value of leadership.
Some of you track studies that are often published, one of those Coastal Trading Magazine.
It was released in May, the 2021 status of the industry.
They showed that leadership development funding is down part of its down because there’s up some funds gotten diverted into some pandemic related programs.
But also, it’s down according to them, because we’re struggling with showing the value of leadership development and they, and they asked some questions around the value of the impact of leadership development. They said, there’s showing very little improvement.
So, that’s an error we got to work in.
This is what we’re doing today, But the number four item is, hey, if you want results at the end, if you want to connect to the business at the end of the program, you gotta start with it and design for it, and that’s what we’ll be talking about here, and then think about your own next steps to bring more value to your programs.
So, here’s our contribution. Sarah mentioned one of those Bookshare.
They soft skills book just published by ATD. We’ve got another one planned with HRDQ. We’ll be talking about that during the session.
But this was also published by ATD as a book of case studies and how do you do it?
And then detail how do you do it from McGraw hill here and bring in this concept through this publisher Elsevier, almost two decades ago?
So it’s a long time we’ve been working in this area. Leadership is so critical to our organization. So powerful.
It gets undervalued. And that’s what we want to think about today as we look at our processes. We’ve got some tools for you.
Oh, we want to send you a chapter out of our book published by HRDQ Here.
Business Case for Learning.
It gets into the Designing for Outcomes Using Design Thinking Principles.
So we’ll send you that chapter. We’ve got a case study that’s in this book we want to send you, we’ll talk about it a little bit today. But we also have another case study we’d like to put in your ads.
It’s more in tune with the EMI process, diversity, equity, and Inclusion.
This is empathetic and inclusive leadership, so we’ll show you, its study evaluated all way to Roy for that program. Now, you don’t necessarily need a detailed book understand what we do.
Here’s a short version of it.
ROI methodology at 12 easy steps.
That’s like the CliffsNotes version in college, if you remember that, where you had the short version of the book, that you are supposed to read, but didn’t want to read.
And so we’ll give you that, it’s 32 pages.
And then we have this down available for you, downloaded. Now, if you can, if you, if you can easily download and print that model, it would be good, because you can go to follow along that process. If you cannot, maybe you put that one on the screen. I know both, you have multiple screens, many of you do.
If you can, plot that model, and let’s talk about that, it’s also the model I have behind me here.
And it’s also in this 12-step guide.
So, these are all resources to your course.
You’ll have the slides here at the end for you to refer back to, and look at again and again, where you like, not evening. What do you want, some more in a tank? So that’s what we have for you today.
And so let’s talk about getting some information from you.
First polling question here, what’s your role in leadership? Select all of those that apply.
Are you an internal provider or leadership development and external provider?
You manage that talent development function, maybe.
I teach leadership.
I’m a leader. I don’t do any of those roles, but I’m just the leader in the organization.
Hopefully we got some of those here.
So let’s see what we have.
And so, with the polling in place.
And Sarah is going to show me the results in just a minute.
Yeah, few more moments here to submit your answer. We have some votes streaming.
OK, great, and I am sharing those results now. Jack.
OK, so it’s not appearing on my Just two minutes, so if you can just tell me what you have here, yes. so, we have 49% say, I’m an internal provider of leadership development. 20% say, I’m an external provider of leadership development.
9% say I manage the talent development and 37% say I teach the leadership development. And 69% say I am the leader.
Alright, What a great combination.
So, A, with the right place with the right content. And that’s, this is great. So let’s think about levels of leadership for a minute.
So we’re gonna go back to a quote that’s attributed to are one of our presidents a long time ago, John Quincy Adams.
You allegedly made this quote.
If your actions inspire others to dream more, learn more, do more, and become more you’re a leader.
So that’s leadership development defined by John Quincy.
Now if we put some structure around it that you may be familiar with, you can see these words of reaction, that streamy more: learning courses, learning more, doing more, application, applying, what you’re learning, and then becoming more is the impact.
Now we’ve gone out of fifth level there and that it’s worth it.
John Quincy didn’t do that.
But I’m sure he thought about that.
You see, because is to be a leader, is it worth it from the leader perspective? That’s a good question. I’ve heard leaders this give up A leadership job says this too much.
Trouble in time for me to be a good leader.
Wasn’t worth it for them.
And I’ve seen others say, oh, this is so great, I love this, It’s worth it for them. But then the follower.
So look, think about that. For a minute, the person who’s following this leader, is it worth it for them?
Is it worth it for them to follow that leader?
See, So we’ve got the follower perspective and then we get the organization.
If I’m investing in a leader, is it worth it for us to invest in that later?
If I’m investing in leadership development, is it worth it? So they worth it.
Concept is Roy, will talk about that. We’ll call that level five.
Now, we’ll be talking about level five leadership and all due respect to Jim Collins, if you know of his work in his book, Good to Great.
You talked about level five leadership, Level five Leadership with His Mind, was it.
Yeah, at the top of the game in leadership.
So it’s a lot around the competencies.
We have something similar, but not quite that way, but we’ll still call it level five, and we’ll talk about that in a few minutes.
So if we took those, those logic flow from John Quincy, way back, and there’s a lot of people who’ve worked with that.
It’s a logic flow of data.
There’s a guy named …
Cat Soul, who first put it around the learning and development field.
And the personally Kirkpatrick actually publish those steps from quetzal.
It’s both popular in R Field, we’ve added to that in our methodology by adding a structure and process. And then the fifth level is Roy. Is it worth it?
So, with that little history, ever, the point is, any program that we implement, anything that we do, and its simplicity here, has this value chain.
See, it starts with who’s involved.
How long are they involved?
and, what’s the cost of it.
Those are usually the top three inputs that’s coming into the process.
So, for leadership development, that’s who’s there, and how long they’re there, and how much it’s costing, but then there’s some reaction, there’s certain reactions that we want we need for this to be successful.
We’ve gotta have the right reaction, so we’ll talk about that.
Important to us.
It’s a, it’s relevant to us, it’s something that we will implement, something that we We’ll recommend to others. Then, learning.
First, in the Leadership Development Program, there’s Acquisition of knowledge, and skills, and competencies.
Let’s see, we’ve got learning going on, and that’s important, but what’s more important, is the use of the learning.
Applying it, that’s often behavior change. Its actions taken use of the content, frequency of your success with use. And this is where we run into barriers and enablers in our process.
Then, there’s impact, that’s a consequence of the application.
It’s like, So What?
Those are the kinds of measures already in the system.
Productivity, quality, cost, time, retention, accidence, incidents, out of compliance, discrepancies, sales, revenue, part returns, all of those things.
And then we have soft ones.
The intangibles of teamwork and engagement and reputation and image. Those kinds of things. Those are all impacts.
Then we have they are off.
So we’re going to express the Roi using the two most common, rewind numbers on the planet.
A benefit cost ratio which comes out of governments, and the ROI which comes out of business, we showed you those definitions in a few minutes.
So let’s flip this around to leadership.
For those 69% of you who are leaders, where would you put yourself as a leader on this chart?
So let’s, let’s think about those leadership development, definite evaluation that we just talked about, reaction, learning, application impact and ROI.
So now let’s think about those as success levels.
So it could be a leader successful because we admire them.
So quality is that admirable and then focus the challenge to make that happen is to make it exciting for them.
And we ought to label this person as a charismatic breeder that’s reaction.
That they’re successful, we think.
because the charismatic well, we’d say that’s not the whole story, but those who are very sharp, they know a lot.
See, we’re concerned about learning from them.
So, intelligence is the quality. They know a lot about our organization. They know a lot about how to be a supervisor. How to be a manager and be a leader. They know about things that they should to make this work.
So, the focus there, the challenge, is to make it matter.
Make it matter to the people.
Bureau are following this leader, hey, we call those teachers, we seen them, you learn a lot from them.
We were like a sponge when that around them because we weren’t learned from this person.
So learning is the evaluation there.
So we say they’re successful if we have them.
Well, maybe not all, because they’ve got to do more.
Then just have knowledge and read, we acquire that knowledge.
A Bias for Action.
Tom Peters really pushed this concept in his book in Search of Excellence.
In the 19 eighties, hey, leaders or have a bias for action, they want to get things done. So, the challenge is to make it stick.
Make sure that people are routinely, systematically doing something that they should be doing to be successful here.
And the label, we often say these are influencers, they really make a difference, they influence people to take that action.
Application is the level of evaluation they’re applying, what they’ve learned, the people who ended up following the leader, or if we have leadership development, it’s, someone is using the leadership development skill, and now, that’s application, but we gotta go more.
You see, if we just have application without impact, we’re just busy.
We need action with a purpose. We need results.
A key here is to make it credible.
We’re showing the results directly from the leader.
Now, it’s gotta make sure that that’s coming because of the leader, and that’s something else. That’s the credibility. We call that an impactful leader.
And the focus here, of course, an evaluation’s impact.
But then you still got another one.
Level five, a value-add leader.
They create more value, then it cost.
It’s worth it for them, square foot for the followers, and it’s worth it for the organization, saying, we make it worthwhile. It’s worth it.
And call that a valuable leader.
That’s the ROI. That’s a level five.
So think of yourself as a leader, if you, how you stack up here, which one is your area where you shine most?
Do you think?
See what we’re suggesting here?
Is it, a great courageous?
Successful leader is the one that delivers the results in all five levels.
Re seen leaders who people love them, and one of her articles on this topic, I showed you the CEO of up HSBC, a large banking organization. Everybody loved him, but he couldn’t deliver results.
So it was great, great here.
Broke down, here.
So you’ve got, you’ve gotta have it all together.
So now we’re thinking about impact.
Can a leader create impact with the followers?
I think we probably would say.
well, yes. Let me show you a dramatic example of that.
We’ve heard of Jack Welch, maybe.
he passed away just before the pandemic in early March of last year.
When he led GE for 20 years, tremendous improvement market cap, so much so.
That Fortune magazine labeled him as an executive of the century.
None of the year, none of the decade of the century.
He had such a huge impact. Curious his.
Focus, great business leaders create a vision, articulate the vision and relentless drive to completion with that focus that dry.
It did this.
His replacement X is selected by Jack Jeffery. I’m out.
Look what he did he, he was in place almost 20 years, almost the same time as Jack.
But notice the market cap.
It basically lost half the value of that jacket created.
And here’s his famous quote.
Every job or decision is easy until you’re the one on the line package, looking for producers, does me.
To be fair, with Jeff is the book trying to explain why all this happened.
But the point is this is pretty clear that a leader made a difference here.
Now, you may or may not be a Jack Welch fan, but he did drive the results.
So a leader can be impactful. And we hope we’ve seen it.
We see these kinds of results.
Whatever GE invested in Jack, was a huge ROI for the company.
Level five is there, huge.
Got lot more out of him than they paid for, for his services, but let’s bring it down from the CEO, just bring it down to, I like to look at the first level. To me, that’s the, the, the most important level of leader in the organization.
The first level leader, I spent a lot of my career working with that level.
one of my books, in 19 80, it’s called The Improving Supervisors Effectiveness, it won the best book award, for sure, guess we looked right at that supervisor’s job and how important that is, how critical is in terms of selection and developing and preparing that person to do the job, given them the authority and the compensation to make it work. So we need those great leaders. They all have KPIs.
Every leader has a KPI, Key performance indicators. It may be an operating report, they get, it may be a scorecard or a dashboard, Or maybe just a list of KPIs.
They’re there, and they reflect, for the most part, almost all. If not all of those KPIs, or the team’s measures.
Yes, that’s what we wanna look at, the KPIs, those impact measures that are so important.
So let’s look at leadership development from these levels of these perspectives.
Is it worth it for the leader, the follower, other organization?
But said, so many of you in leadership development, 40%, and internal, About half that number. External providers of leadership development. Many of you teach leadership development. Let’s think about leadership development. Now, from the organizational perspective, we provide leadership development. So let’s look at that perspective, primarily here.
We have some challenges, and, you know, days.
Think about this list for a minute.
Think about, it’s just mostly true, are mostly false based on your experience.
These are statements: first one, most leadership development is wasted.
That means it’s not used.
They just don’t use leadership competencies that we gave them.
What do you think, mostly true? Most of the faults are not doing a poll here, but just think about, I want to tell you something we’ve been collecting.
This data was polling and with show of hands, a lot of polling in the last year and a half you can imagine.
And here’s the number we get on average Mostly true, 78%.
That number was going down until the pandemic and it went up.
And I think that’s because virtual learning is less likely to deliver the results.
And we can talk about that later. That’s not the purpose of this workshop, but that’s the reality of what the data shows doesn’t have to. But it does.
It’s just not designed for delivering application and impact.
Now, that’s what we’re showing you here. We will.
Let’s look at the next one for a minute.
Leadership development outcomes desired by executives and client organizations is E, not as rarely measure.
So what do you think most of the truth is, mostly false, the outcomes desired by executives, or is rarely measured. What do you think?
Mostly true. Mostly false.
Well, we’re going to show you the data.
This is, again, this is typically L&D managers tick in this poll, 89% said, That’s mostly true.
Think about that.
Oh, that’s disturbing, isn’t it?
We know what they want, we just don’t show them.
If these are executives who gives us the budget, that’s a little risky.
Let’s go to the next one. Most leadership providers do not have data to show that they make a difference.
Let’s define, make a difference.
Make a difference, is a combination of application and impact.
You need both of those together doing something with a consequence, if they do something is no consequence, that’s being busy.
We’ve all seen people who are free and busy, but they just don’t accomplish much. That’s what it looks like. You need both application and impact.
So when we asked that question, mostly true, mostly false, think about your response.
and your colleagues, 81% says, that’s true.
So, 81% is saying, we don’t have data, and we made a difference.
Isn’t that a little risky at budget time?
If you don’t have data that shows you to make a difference, which if you’re a top executive, would you want to keep invested in this?
Do you want to add to that investment?
Number four. Most executives see leadership development as a cost.
Not an investment.
What do you think?
I know what you like to say, but we anticipate all of you are going to say, mostly true.
And this number goes up to 93%.
You can see the problem here.
If they see it as a cost, what do they do with the costs?
Reduce it, eliminate it.
They curve it.
They pause it, they freeze it, they postpone it. We saw so much was frozen or postpone during the pandemic.
That’s just a nice way of saying we’re killing it.
So if it’s a cost, they’ll do that.
If it’s an investment, they may do more, and protect it, maybe.
So we’ve got to work on that.
It’s too high.
And then finally, most executives, seat hard skills as more valuable than soft skills.
What do you think? Mostly true? Mostly false.
Our data says, 95% says mostly true.
So this is a dilemma.
If they don’t see the soft skills, is that important?
Why would they want to keep investing?
Well, they do, they talk about this. Executives, which always talk about soft skills.
See what makes the best organizations in the world the most admired, the most innovative, the most sustainable, the best places to work. All of that comes from.
Not hard skills.
I’m an engineer.
I came up through engineering early in my career and I love the hard skills science, technology, engineering, math, I live with that.
But quickly, I could say it’s the soft skills really make a difference in the organization.
Yes, you, you got to have these hard skills, but you get more value out of those good soft skills.
But if executives don’t see that, we gotta show them that.
So we’ve got our work cut out, we go through that, just to show you that we’ve got some challenges we’ve got to overcome to make this work.
So let me ask you, as you think about leadership development from it, what levels do we evaluation now?
Just check all of these that apply, leadership evolve, check those who are you evaluating now.
Let’s see what we have.
We’ll give you a few lines here to submit your answer before we share those results.
And just a few seconds here, have results streaming and OK, perfect. I’ll share those results now. We have 42% saying reaction, 66% say learning, 53% say application, 37% saying impact, and 16% saying ROI.
As they say, bless you, that’s amazing. That’s a great profile for level 3, 4, and five.
I’m, I’m among some of the best here. This is great news.
And so we’ve got to measure at those levels.
So that’s what we want to talk about here, pushing up the level 3, 4, and five in our work.
So, let’s, let’s think about what keeps us from measuring more at the higher levels.
Well, one is the resources that regard tags, but sometimes there’s a fear of the adverse results, negative results.
We’ve had people tell us that AFI program is not working, why would I want to go measure that in Publisher, Report and give it to my Executives?
What does that do for me?
But made me look bad?
Well, we say, if it’s negative, two things here.
It’s probably not something you did, something missing. They’re not supported very well and then burn.
It is rarely a problem with the content that causes that it’s the support of that content, the use of the content.
So that’s one thing you’ll know whether something’s getting in a way or not.
Another thing is, don’t wait for someone to ask you that question. Do it a proactive basis.
You’re in much better grounds to do that, but even better.
Think about designing with that in mind.
Start with the end in mind, if in practice, where you want to be.
Well, let’s start with that.
And that’s what we’ve done in our process.
We’ve morphed into a proactive evaluation system here that not only uh, evaluate your program all the way to ROI, but he designs for the impact and Roi and helps you deliver it and that’s what we want to show you here 12 Steps you’ve got it printed out hopefully.
Are you got it on another screen possibly, or you see it here in the back?
This is the process, but I’m going to just take you through it quickly, you can see how it works.
We start with the end it, but I’d say we’re thinking about evaluation at the beginning of the process.
At the conception of the program, not at the end. I hope that’s done.
That’s the worst thing you can do, is to say, Alright, we train all of our first level managers.
on the Leadership Challenge Program, that costs a lot of money, and that, tell me, what’s the ROI.
Hey, we’ve had that before, we’ve had people come to us, just had this request, what do we do, and then prepared for it.
So that’s not the good time. In fact, that’s the worst-case scenario.
You want to start with it. Start with why align your program to the business.
In the beginning, we’ll talk about that more in just a few minutes.
That’s not hard to do, but not done so much.
We want to start with behavior.
Hey, we want these behaviors in place.
And the executives would say, yes, behavior is good, we know we need leadership behavior, but it’s the results of the leadership, behavior that we really want to see. It’s like, so, what?
They don’t want great leader behavior; they won’t break results from the leader behavior.
The leader behavior is a means to the end, not the end.
Let’s keep that in mind. But we also got to me we’ve got to make sure this in the second step is just the right way to get to that business measure.
Can we actually change that business measure that you’ve just identified there with this particular program?
Check a deceit.
There may be some discussion. It may be some analysis share.
When we satisfied those, says, we got a business connection, we got the right solution, and then we expect success.
We define the success of this program doesn’t occur until we have impact, not behavior impact.
And we set objectives for reaction learning application, and then back.
And we give those to the entire team, and we now design for that.
Everyone is putting tools, or templates and processes in place to help them get to where we want to be.
Even the managers of the learners are tuned in to support it and encourage it to make sure we have Level three.
So, that’s, these three are so important here.
So, important, if we set it up with those three, you probably have success, for the rest of it.
But, we will collect data here to see if we’ve made it, level one, level two.
Level three, and level four, wanted to make it matters or concern.
Level three is making it stick and American sir.
But you get to level four.
That is make it credible, which we sought out the effects of our program on the data, making sure that we sort out, the effect of leadership development for something else.
We make it credible by converting the impact data that we’ve already identified the money. It’s probably already done there.
That’s the good news.
Uh, then, we bring in the cost of the program. All the cost, we make it credible by including, both direct and indirect costs.
And then, we’re going to calculate this using the two most common ROI measures on the planet.
one comes from governments at … and the other one.
Roy comes in businesses.
Then we make it credible with the intangibles. Those are the ones that we couldn’t.
Could work to money, they’re still important. We make it credible by making sure that this program is connected to those intangibles.
Now, we’ve got a story to tell.
Storytelling is a big part of design thinking.
But, we got numbers: a narrative.
We got reaction and learning application impact, and Roy and intangible six types of data got sterling’s along the way how they reacted to it. What did they learn? What did they do?
Oh, and the … stories that bring it to life. And we collect survey by focus groups, observation, by interviews.
We get those stories, and this is great because it makes our presentation more interesting and engaging and now, we can use that to work on our funding. We’ll just showed our exact gage that.
Leadership development is an investment using the Roi return on investment, which comes right out of the finance, the accounting field. We showed you the return on this investment.
It takes all of the mystery out of this issue, of, is it an investment or cost?
Nope, It’s not a cost. It’s an investment. We just showed you the return on investment. Now, we are going to make it better if it’s not working so well done to improve it.
So we optimizing our results, that means we’re optimizing, They are why we use that to make a great case now to fund our program.
So we don’t have dips in funding.
As the trending magazine data shows tips and funding for leadership development.
Now, let’s think about.
I think this is going to be easy for this group, because of where you are.
So here’s a percent of programs that we suggest you evaluate.
At each level.
You can see pretty high for the lower levels. That’s easy.
And notice 30 2. Hey, you’re already there as a group.
I think, Elise, what you didn’t tell us is what many programs it could be here, just evaluating one program.
And that gets a check mark, of course.
You’re looking at the percent of programs. But we show you some benchmarking of those who are certified ROI professionals implementing this in their system.
And they’re telling us what percent of the programs are the evaluating each year, at each level.
And you can see high numbers there.
So let’s talk about which programs go to those higher levels.
Which programs do you evaluate, and level 4, 4, and five?
I’d like to ask you, what do you think?
Let’s get your input on poling.
When should you measure the impact and ROI level?
Some may put never are very expensive program, very important, and strategic programs, when the executives requested routinely for process improvement just to make it better.
So, check all that apply here.
Check all that apply.
We’ll take a few minutes to collect your answers before we share those responses.
And just a few more seconds here. We have lots of answers streaming in.
OK, great, so we have 0% that said, never, 23% say, for very expensive programs, 34% for very, very important and strategic programs, 11% when executives request it and 80% routinely for process improvement.
Hey, I love this group.
Yep, that last one.
I’d say we think the number one reason to do this is for process improvement to make it better.
You see that last step on our model if you remember that. I was going to go back there.
If you notice, notice we say, use black box thinking to increase funding.
So we have a relentless focus on process and make it better. Even if it’s successful, make it better. So, yes, that’s the number one reason we want to be doing this.
However, you raise a good point, number two, or number three, or two of the criteria that’s often used to select those programs.
See, if you’re trying to decide which one to evaluate, oh, good.
A good reference point is to go to the most expensive one. That’s one that’s more likely to have the question for the executive.
Or if it’s very important to us or it’s very strategic, it, maybe it’s connected directly to strategy, That’s important.
That gets us a nod sometimes.
But you know, it’s only 11% says when executives requested a levy for that.
Because so many people, we’ve had people put 90% there.
You wait for them to, to, to request it, you’re in trouble, because they request it often after the program has been implemented.
You’ve got three things that happen to you. You’ve got a short timeline to deliver results, you probably can get there.
Second, you you’re now defense if you’re defending your program.
The assumption that they’re making is this probably doesn’t deliver a value, why don’t you show me?
You’re defending it.
You’ll want to be on the offense.
you’ve gotten our Roi on someone else’s agenda, you want to keep it on your agenda, so great! This is perfect responses.
So now, let’s think about this.
The model of, I’m gonna go through some key steps here is, behind me, you got it printed out, perhaps on another screen.
But, here’s the first two steps, Why, and is it the right solution. So, if this is a great way to frame your project, we get more comments about this.
This tool, we call it the V model, because of the appearance of this.
You’ve got your evaluation here brown, from reaction up to ROI.
And the data we collect for this evaluation is driven by the objectives. Do you know that? Because, if when you measure learning, for example, you always go look at the learning objectives. So you do the same for application and impact, even ROI.
But those objectives are come from the needs. So we’ve got needs assessment, blown up here, It’s not just learning need.
So as we need to learn these new competencies, but it starts with why, that’s a very precise business needs. So there’s two things there, Pay off need in.
And a business need payoff name means, is it, do you need this to deliver more value than it costs you.
And you can imagine, most of the executives would say, that’s a pretty stupid question. Maybe, yes. But we have some say, you know, that’s not a key thing for me, That’s OK. Most of their code and say, well, yes, OK.
Now, what it means the business major has to improve enough. So that when I convert that, the money is going to overcome the cost of this program.
So these two kinda come together here, that you’re getting points. So, I want to just, there’s a lot of ways you could tackle this in the beginning. The easy way for leadership development is do something so simple.
If you’ve got a great set of competencies that you want a teacher executive, that some of your analysis says, they need it, hey, why not do this?
Have them come to your program and say, would you mind identify two KPIs that you’d like to improve?
two key performance indicators.
Remember, they all have them, two KPIs that you’d like to improve, but only if you can improve those KPIs with your team using these competencies here.
So the good thing about leadership development is so powerful and so flexible, so adaptable that, you know, good motivation, communication, problem solving, listening, all of those things are going to help almost any situation.
So, I want that person to say yes, yes, OK.
What they’ve done just now, as it says, this is the right solution, all you need to say.
I think so. Let’s give it a try.
That’s all you need.
Basically, say there’s some behaviors that we don’t have in place.
I’m going to put them in place more pronounce stronger than ever.
And I’m, I’ll be learning some things here to do that in a much better way, and I see this as something that’s valuable to me, Something I will implement. See, that’s the reaction.
So those three things come together to make your solution.
So that’s the first two steps on them up. Start with, why will the business alignment make it feasible by making sure you have the right solution. Then we set those objectives. Objectives are so powerful.
We, we understate that we under use that we want to objectives at all five levels here.
Remember, we define the success of the program at impact. That’s where we want to pay.
But to get to that, you’ve got to have this reaction, this much learning, this much application, this much impact. The impact comes from those objectives that they had to beginning. They set the objectives around those two KPIs that they want to improve.
Objectives are so powerful. Let’s have a little quick quiz here.
Let’s think about the levels of objectives.
So, I want you, on a piece of paper, we still have paper, don’t wait.
Make a note, a 1 to 5 here.
What number goes in here?
Put the complaint objective in here.
After completing this program or project, participants should decrease employee transfers by 20% in one year, if we collect data to see if it met that objective at what level are we evaluating?
Is it reaction? Is that learning? Is that application is an impact?
Is it put that number in place?
And the next one we should listen intently to team members routinely.
We should be able to demonstrate first steps, too: Diffuse conflict given to individuals in a heated argument.
Write the content 4 or five on relevance.
Decrease the amount of time required to complete budget.
Those are all objectives.
What levels do we have?
You’ve got those numbers.
How does it stack up?
I’m doing something, that’s two and learning something, and one is, oh, my reaction?
So we say, well, is this really that important?
I’m gonna give you a quick summary of the research around objectives.
Since, for 60 years, objectives are so powerful.
Let’s put some different criteria around objectives.
Let’s talk about Performance Share.
Not having objectives, the objectives. Remember, if we just measure level three.
That’s our objective at level three, that’s all of our discussions at Level three behavior, that we have no objectives at Level four.
We’ll still get some performance there because they go do it.
But if they had a measure themselves that they identified upfront, and they said objectives, like, I want to reduce employee turnover, nothing specific, but just, they. OK, because I know that’s my objective, I’m gonna focus on it more, I’m gonna get more performance.
But you said, look, I’m gonna reduce turnover from, it’s, it’s an annual amount.
At 31%. Now, I’m going to bring that down to 22%.
I’m going to do that in six months, OK, now we got very specific, some people call that Smart objective.
Some people call it, OK, our objectives were key results, See, now you get more, hey, that’s, that’s amazing.
But do you even get more if you put a stretch objective?
Stretch objective says.
Oh, look the objective that you set.
Is your minimum acceptable performance?
If we are developing it, we might have a learning test. We put it there to measure learning. We might say, score seventy out of one hundred.
That means that’s a minimum objective, but we’re OK if they score 90 or 98, We want more, so tell them to set at a stretch objective.
Beat them up if they don’t get there, but you’re gonna get some amazing results.
So use objectives. They’re so powerful. We gotta do that. We don’t do enough of that.
We’ve got a book on that.
It’s called Beyond Learning Objectives.
This is published by ATD.
The other folks, notice I put the title here beyond learning object, because this is where their words, but the subtitle develop measurable objectives, that link to the bottom line.
The point they’re making is that in the learning and development space were quite good at developing learning objectives, but we’re not good at level three, application, and level four, impact, and, of course, level five, ROI.
Push them up a little higher. There’s another book, here I would recommend you is called Measure: What Matters.
And it’s from a person named John Doar.
And it’s got a powerful book about O.k.r.s.
The subtitle, that book is How the Gates Foundation, motto, and Bono rocked the world with o.k.r.s objectives from key results. So, amazing story of how object is making a big difference, but you see it right here on this chart.
Let’s do more of that. That’s our level three.
It’s good match, step three here. Step three, expect success. Then we start collecting data. Hey, this is not new to you.
This is from our benchmark, and just show you what’s being used. The good news we see is that We’re getting a lot of data right out of the records. This is mostly level for data.
And we’re putting in things like action plans. There’s so powerful, a contract. Performance. Contract, I should say.
Is up It’s an action plan, uh, with your boss involved in it with a pre-programmed commitment to achieve a certain amount for level four, impact.
Hey, that’s growing and choose.
So, lots of ways to get there with data collection.
After data collection, that’s step a five. Step six is that we’ve got impact here with data collection. Now, we’re going to sort out the effects of our program.
A lot of different ways we can do that. We can look at experimental versus control group, and 34% of the time, we can do that. We can do a similar trend line analysis. I won’t get into that now.
So, there’ll be on what we want to do here today, 40% of the time, or just saying, hey, the other influences that are out there driving the same measures are exit, predicted, and a Mathematical model.
And I’m going to use that to show the value of what I’m doing, because it’s all predicted that month, that’s a rare occasion, but it works sometimes, or it may be, we have to have an estimate. The point is, we want always to use this.
As you look at this process, you’ll be looking at more detail with that 32-page document. You can see, we have standards in there.
one of the standards is this, is we always separate, oh, our impact Separate what we’ve done.
For other influencers, if you don’t do that, you have no credibility in front of an executive because they know almost always something else is out there.
Now converting data to money, good news here, OK, we’ve got standard values that are out there Already in front of us, they can calculate because if it’s a problem, we have the way we have money collected, that is some executive says what is it costing us when they have days? So you have money on things that you didn’t think you had? If you just go, ask the people who know?
And the people who you know issues it, those internal experts, internal standard value means it’s already communicated to us.
So sometimes we say, stop, these lost time accidents. It’s costing us this much money, every time we have one of those.
Now, what that’s doing is telling us how important it is.
The money got our attention.
So the good news is, the money is already there. If not, we get, we can go to the next term, their database, and find so much of it done there. And we can find credible value for most of our measures.
And when you get money compared to the cost to get to ROI, and the kids are two calculations, PCR is Benefit Cost Analysis.
And its benefits to die because that’s a monetary benefit, all ended up for the measures we have for the people in our program that we’re evaluating.
And then the cost is all of the costs in there.
Benefits minus the cost, divide by cost times 100. Let’s just put a percent out it.
And I’ll show you what that looks like with an actual example here.
So let’s take an example. This is one of the case studies. This is a program called the Leadership Challenge off the Shelf Program.
This is a coffee and mixed precision parts primarily for the automobile industry.
And so, they bring this out to all of the first level managers, and they decided, Let’s connect it to the business by asking each one of these first level managers.
two measures that you would, you can improve if you’re using these competencies, and that becomes a solution that they could make, that confirmed, that will say, I think I can improve that using these competencies and they are the results here.
It gives him a lot of insight into, oh, how this kind of soft skills can work?
And Roi helps executives to see if this is worth the investment that that’s what Roy does for them.
Is it worth it?
Curiously, what’s measured here in this case study again, we’re going to send it to you, relevance importance and tend to use three very powerful ones because these competencies they’re using.
And here’s the, they actually use, that the views frequently see views and the success that are having with it, because of the kinds of measures they select, and each 1 has 2 measures. When they get it to money, you get the money. And you’ve got a calculation that looks like this. All the money comes together here.
This is the total cost of the program. We’re evaluating only a sample of these.
And you get the benefits minus the cost divided by cost times 100.
And that’s 47%.
This is forever, $1 we invest.
We get our dollar back plus 47% unless you’ve got intangibles.
Jaap engagement, job satisfaction. Stress.
Now, when you’ve done that, you communicate your results, see group, here’s for roof, we want to make sure they get this executives, the participants, the participants managers, and our own team, and we’re going to make changes, make improvements for the next group.
And we’ve learned how important this connect, that business measure upfront, and now we’ve got very good reaction to this, keeping this budget, and helping this budget won’t be a problem for us going forward.
And there you have it, 12 steps.
Just that easy.
I know you’re thinking, maybe not.
Let’s do one more poll quickly.
What do you think?
What are your concerns about ROI?
Select all that apply.
There are concerns about ROI.
And just take a few moments here, too, Submit your answer.
Just 10 more seconds here, off, and we’ll share those results.
OK, great, so we have 38% saying, It’s too difficult. 50% saying it takes too much time. 75% saying no one is asking for this. 4% saying it will kill my programs and 8% said, I wish it would go away, way we can relate to the last one. We understand that. So, difficulty.
So, let’s try to address it this way, it’s a logic model, you gotta have drop-down menu for each part of the model, to make it easier to use pay. And we keep the mathematics at fourth grade level.
Does that help?
You can have a credible study, proved by his CFO. I never go beyond fourth grade math.
It’s user friendly from that perspective.
It takes too much time, there’s some software that can help, but do you get some tools and templates in place? It won’t take as long as you think, but that’s just want one reason. You want to limit the use of this. You’ll want to do this very, very, small number. Be selective.
No one’s asking for that. Hey, this is the time to do it.
If you’ve got people asking for it.
You’re in trouble, because you’ve gotta go do it, and you didn’t have time to get it going in your organization.
So, don’t wait for that asking, will it kill the program? I’d say Roy is the best way to save your program.
It kills it only if it’s the wrong solution.
And if it’s a wrong solution, it needs to adapt because you got the wrong solution. It will tell you what went wrong until you can make it better, so keep that in mind.
And yes, we understand, we wish would go away, we do want to give you these resources, so you can go with this, work with it more that we described, and they are beginning.
There’s a QR code here you can use to get those.
We’re going to put in another case study, Not published yet, but will be published a new book that we’re just going to talk to you about next, but it’ll be in there as well, so you get all of that process there with this QR code.
And now here’s a project with, uh, HRDQ. We want to publish a book of case studies. Here’s the title of the book, Proving the value of leadership development.
Real studies from top leadership development provider.
So think about it, do you have a study already done?
I think some of you do, based on what you’re telling us here.
If not, maybe you want to learn now, we can get you into a program to teach you how to do this quickly, and we’ll give you a discount to get into that program is really to learn this, but let us know if you do you’ll, you’ll get more information from us about this particular project, so you can opt in to be considered for this.
Or you can let us know you can come directly to me, our Director of Publications, Hope at ROI institute dot net, and building that serious capability.
We’ve had about 16,000 people.
Get involved in this certification here, Roy, a certified … professional.
Hey, would you believe the next one coming up is in person? Salt Lake City next week?
Yes, and we’ve got some in person, but also live virtual, going on, and we have the synchronous as well.
It’s there. Let us know if you have questions.
If you have questions here, we’re going to leave a little time.
But I think Sarah wants to close out here first, and then we’ll stay around for any Q&A that you may have.
Yes. So, if you have any questions, you can type them into the questions box. We are at the top of the hour here, so we’ll be able to fit in a question or two that you may have.
As Jack mentioned, be on the lookout for a special offer for our attendees today and your e-mail, and we will be sharing something great with you there. And, today’s webinar is sponsored by HRDQ-U, Virtual Seminars. Be sure to check out our curriculum, more than 80 virtual instructor led online seminars. You can go to ww.hrdqu.com/virtualseminars for more information, and make sure to join HRDQ-U on your favorite social media site for quick access to all of our latest webinar events and blog posts.
You can find us at HRDQ-U, and we will conclude today’s session at jack with one question that we received from the audience today, and that is: isn’t ROI too complicated for most managers and non-financial professional?
That’s a good point.
I’d say, no, based on art hour experience.
So you just think about, When we make a purchase, we buy anything.
We think it is this worth it.
Is it worth the money? I’m paying for this; I’m going to getting enough out of it.
Well, the finance accounting field has helped us by putting some structure around that with a, with an equation.
So that gives you a term that managers learn. If you’ve got a manager with an MBA, they’ve got ROI all in the MBA.
And so, they, that you’re spending this much money, it’s showing you how much you’re getting out of it.
So it’s a simple concept, money out, money.
This money out, cost avoided.
That’s a comparing those two. And again, the calculation shows our rate of return.
The Roi 10% ROI means I got my money back plus 10%, for every dollar invested.
So it is a concept.
Most managers can appreciate they because they deal with it so much, they see it so much.
So we try to keep it simple.
So if you are in a position where you never ever wanted to see need to see ROI, then you may not want to go down this path. But it is a great way for you to see that value is working.
Just want to mention this, if you’re in government, there’s an issue of cost benefit analysis. Has been there for centuries.
And even a best-selling book recently called The Cost Benefit Revolution, say this has got to be in budgets for governments, and non-profits and NGOs.
That is, you’re spending money, you’re using money.
What are we getting back for the donors, or the taxpayers? So it’s a concept that’s not that difficult for most to understand.
So if you have other questions, please reach out to us with this e-mail, and we’d be happy to set up a call and talk through some of those with you.
Thank you for coming.
Thank you for being involved.
Good luck with your programs.
If you’ve got one that you want to work with us in this book, we’d love to have it in there.
Yes. That is all the time that we have for today. Thank you so much for joining us today, Jack.
And thank you all for participating in today’s webinar.
© 2021 HRDQ-U. All rights reserved.
HRDQ-U is a free learning community for trainers and facilitators, coaches and consultants, organization development professionals, managers, supervisors and leaders; really anyone who shares a passion for soft-skills training and performance improvement. We bring exciting content to you through webinars from subject matter experts and thought leaders to help you explore new ideas, gain industry insight, and improve people skills in your workplace.