5 Critical Success Factors for Evaluating Operational Performance

“Ready, fire, aim.”
“If you don’t know where you’re going, any road will get you there.”
“Shoot from the hip.”

These are not sayings you want to hear when you’re trying to set direction and plan for operational performance improvements. Yet, this is often how situations are described in hindsight when the vital first steps are done poorly. Not to mention those situations where months and months (and MONTHS) of deep dive analysis result in a pretty presentation but isn’t rooted in any kind of operational reality and shortly thereafter, falls flat on its face. (Sound familiar anyone? Bueller? Bueller?)

Bueller, Bueller, Bueller

Baseline First

To develop an improvement plan and portfolio you must first get an accurate baseline of the current situation and describe the gaps. Doing this part well ensures better results because you’ll have practical, and more importantly, credible, information starting out.

In this post, we outline 5 critical factors that we use to ensure a high quality, yet efficient performance evaluation process. One that leverages internal experts and external viewpoints, is focused on both the operational realities of the front line and leadership priorities / objectives, and that gives everyone the same baseline measurement system from which to make an objective assessment of “what good looks like.” This process can take as little as a couple of days. Check out Kaufman Global’s Rapid Performance Evaluation.

Operational Analysis Basics

But, before we dive into the 5 factors, a short history of business consulting analysis…

For many years, the traditional way of getting started with clients involved a request for lots of data, a long list of questions and a team of four or five consultants onsite for 4-6 weeks to conduct a formal analysis. This would result in an implementation road map and ROI calculation that validated the path forward project. There are certainly situations today where the business case needs to be defined and this type of analysis is the right approach – especially when trying to evaluate extended, cross-functional value streams. We have found that many clients no longer have the appetite or time for this and want to move almost directly to implementation.

On the complete other end of the spectrum was the famous “kick the tires” visit of the 90’s. This was basically a site walk-through to look around and get a feel for the location and how good or bad it seemed, but very little data was used to make the assessment and the structure of the visit varied greatly depending on the consultant and their personal preferences.

These approaches both have their pros and cons. As the consulting model continued to evolve, we observed that getting a client a great answer quickly required a fundamental rethink of the approach with emphasis on the following issues:

  • Speed – busy operations can give you a day or two to get some good information, but more than that feels burdensome
  • Standardized rating systems that can be used for comparison were largely missing from systems we’ve seen. Or if there is a rating system, its relevance was questioned. We’ve heard, “Those ratings aren’t completely applicable to our situation.”
  • Many organizations skip the people side of the equation during the analysis altogether – leaving it to be dealt with during the project. This is a serious flaw that will impede progress when you try to get the organization on board AFTER the fact.
  • Most traditional attempts to evaluate the operation have not adequately incorporate an assessment of the organizations ability to change and engage themselves. Evaluations are typically all soft stuff (people skills, organizational dynamics, engagement, etc.) or all technical stuff (productivity, quality, cost, supply chain, etc.). These two dynamics are so connected that both must be considered in any meaningful evaluation.

Rapid Performance Evaluation

To address these issues and meet the demand of clients who really needed a solid baseline assessment but didn’t want a full-blown analysis, we developed the Rapid Performance Evaluation (RPE) process. A holistic, effective and efficient approach to the initial gap analysis that could be done in a day and provides team-validated, benchmark scoring with an identified improvement portfolio of ranked opportunities for both the people and technical sides of the equation.

How did we do it? We looked at the gaps in traditional approaches and developed a set of 5 counter measures aimed at making the evaluation fast, relevant, user friendly and credible. Design considerations include:

  1. Use a good template
  2. Limit the evaluation of operational metrics on the first pass
  3. Apply change management techniques from the start
  4. Use a credible rating system
  5. Prioritize opportunities and describe an implementation approach

Use a good template – To achieve speed, consistency and quality, use a great template. People are busy and have not planned for the interruption. Our templates are based on the idea, “This is what good looks like, let’s see how this operation compares.”

The template must be standard, yet flexible. Too rigid and the response will be “That doesn’t apply to us.” So, be prepared to modify to make it more usable for the situation at hand. We’ve found that the people and organizational parts change little between locations, industries, sectors, etc., but the technical pieces like flow, supply chain, quality, etc. usually benefit from some adjustment.

Limit the evaluation of operational metrics on the first pass – For any given operation there are about 10 metrics that tell the story. Things like inventory, quality, productivity, turnover (people), etc. We like to ask how difficult is it to get this information too. This tells us a lot about the operation and location’s leadership style. Vast amount of data does not always (read: rarely) translate into usable information. By usable, we mean available in a standardized format and used for running the business.

Identify and ask about the best few metrics for the operation and then use them as a guide for where to look more closely during the evaluation. Later on, more information and more analysis may be required, but for your initial look, keep it simple.

Apply change management from the start – When you ask people what they are doing and why they are doing it, its personal. But this engagement is essential because they know more about what they are doing than any outside observer. A non-threatening environment, supported by a skilled facilitator, will yield the best information. Don’t wait for “the real project” to start respectfully engaging the knowledge of the organization.

To make this work, the key is to quickly develop internal evaluation experts. For us, that means having a joint KG + Client team conduct the RPE. This is where standardized templates and processes prove their value as it enables us to easily on-board and train the client participants in how to do the evaluation. With a 60-minute orientation, the joint team is aligned and up to speed on what to look for during the tour and how we will do the individual and team scoring later. This facilitated, open and honest team discussion helps ease the dialog that occurs later when the team must come to consensus on certain performance measures.

Use a credible rating system – A rating system gives the organization a way to measure their improvement. The score is the baseline performance of the operation and clarifies the real gaps. As improvements proceed, better ratings show tangible evidence of the gaps closing. We use a series of three assessment tools that provide different ratings:

  • Opinion – What do people think and how do they feel about what’s going on
  • Systems Factors – The big picture. It looks at things like quality, governance structures and how the organization deals with complexity
  • 20 Keys® – A drill-down into the details. They provide guidance on specific improvement activities and exactly what must be done to get to the next level of performance.

Prioritize opportunities and describe an implementation approach – Once the evaluation is complete and the gaps described, the question becomes, “What next?” This question must be answered at least at a high-level. A one-day RPE won’t yield a detailed implementation plan and business case. But, it will answer things like: top operational priorities, engagement problems, leadership mis-alignment and recommended next steps.

Internal and External View

When evaluating performance, it’s difficult to achieve adequate results without incorporating an external objective view. That’s why we’ve found the RPE process to be so powerful. We have found a formula to bridge the gap between providing an unfiltered outside view, while incorporating the expertise and knowledge of insiders to accurately assess the current situation. It’s natural for those who are directly involved as part of the team to defend the work they do, so a balance must be struck. In one memorable situation the supply chain was so bad that simple commodity replacement parts were taking 8 weeks to obtain – crippling the operation. Yet in talks, the manager being hurt by the problem rated the supply chain a 5 out of 5: Perfect. These difficult discussions are almost impossible to have without a baseline measurement of “what good looks like” to test against and some objective external facilitation.

When done well, an evaluation of operational performance is a fantastic starting point for real change and improvement. When done poorly, it becomes simply another exercise in futility and another data point for “Why this stuff never works.” If our RPE process seems like it’s the right fit for you, give us a call or drop us a line.


Learn more about how we think about the topic of Analysis as it relates to our service offerings, click here.

Or, see our post on analyzing extended value streams: Making the Case for Analysis.

Rapid Performance Evaluation – Speed Matters

Rapid Performance Evaluation: Standard Work for Identifying Operational Performance Gaps

Kaufman Global helps clients solve complex problems and drive fundamental improvement. We engage when people and process collide – places where expertise and leverage can speed results. Even when an organization knows there is a problem, understanding operational performance, getting to solutions and knowing which levers to adjust can often benefit from outside perspective.

Rapid Performance Evaluation (RPE)Over the past few years, we’ve observed that clients want answers faster than ever before. And while it could be that “time is money,” it seems to us that it’s more related to the frenetic pace of, well, everything these days. Headlines and “apps” often don’t dig deep enough and the “Ready, Fire, Aim” approach has great potential for missteps.

To meet the demand for fast but thorough answers, we devised an innovative method for quickly getting to the heart of the matter – operationally and organizationally. Our Rapid Performance Evaluation (RPE) uses a standard work approach to cut the time required for credible solutions to about a day. How, you ask?

Instead of only identifying and prioritizing process problems, the RPE delivers tangible feedback and scores that can be used to immediately take action to improve. The RPE:

  • Provides a comparison against well-defined standards and benchmarks
  • Ensures the leadership team is aligned on the issues
  • Establishes specific and prioritized things to work on now
  • Engages the organization out of the gate, reducing rework and improving data fidelity 10x
  • Is fast and agile – minimizing disruptions. Getting accurate info doesn’t have to take weeks

The process begins by on-boarding the team and communicating with site leadership. It ends with a report to same. Core attributes of the Rapid Performance Evaluation are noted below.

Visit the Gemba

The gemba is the place work is done. It’s the shop floor, the office, the warehouse, the lab or the medical unit. It embodies the concept of “Go look, go see” and is a vital step in collecting information for analysis. We’ve found it helpful to review immediately before the visit what we’re trying to “see”, so we use a standard set of definitions to focus our attention. For example, in an office environment one thing we look for is communication between functions. In a factory we want to understand how material is moved (pushed or pulled) and stored (inventory) from one location to the next. Lots of paperwork is not needed for reference. Our optics have been adjusted ahead of time so the visit to the gemba can be for observation and understanding.  We need to keep our eyes and ears open.

Template Driven for Simplicity

Templates are used to compare existing practices to best known practices. With the RPE, simple but proven definitions and an intuitive measurement system make it easy to get everyone on the same page when it comes to scores and ratings. We look at factors that correlate to overall performance, such as quality systems, teamwork, continuous improvement capability and material and information flow. The correlation factors provide a big-picture view and point to overarching or systemic causes affecting performance.

For more discrete aspects of the operation, we use the Kaufman Global 20 Keys ® to evaluate 20 critical elements that affect efficiency and effectiveness. For each key, the tool ranks the current level of performance using a 5-point scale where 1 is “Traditional” and 5 is “Currently Invincible”.  Levels are described simply so the requirements for achieving the next level of performance are easily understood. The 20 Keys dig a little deeper than the correlation factors by identifying and prioritizing specific things to work on.

Alignment  Speeds Change

No matter how good the templates and rating systems are, they don’t account for the human factor. Opinions matter. During the course of the visit we interview key leaders and stakeholders. This usually means functional heads who have valuable insights and who will play a critical role in any changes moving forward. We start to see how much (or little) agreement there is about the underlying issues. This is an area where being external to the organization is a key advantage. Functions are typically protective of their turf. Outsiders can ask more probing questions. If there is a significant difference between what we hear from the leaders and what we see on the ground, we sometimes opt to survey the organization. This can help identify broader organizational issues.

Balance Speed and Accuracy

The RPE is done with a small joint team comprised of Kaufman Global and the client. Since the method is standard, well defined, intuitive, and template driven, training for the client participants can be completed at the start of the day. The real benefit of this simplicity becomes clear at the end of the day when scoring begins. After we’ve completed the tour and interviews, we individually rate and rank based on our personal observations. Then, we come together to discuss and negotiate consensus results. The evaluation is better because it consolidates multiple views, experiences and vantage points and compares actual performance against intuitive and easy to understand benchmarks.


Any operation can be assessed for performance quickly if the method considers all of the output requirements and integrates change management approaches. The RPE gives leaders rational ratings of performance, a clear understanding of organizational challenges and confidence that they’re spending energy in the right places.

Find an example of the approach and results here: Rapid Performance Evaluation (RPE) Case Study: Automotive Electronics.