Navigation
Home
News
Newsletter
Knowledge Bases
Web Workshops
Tutorials
Directory
Books
Maintenance-Tips
Conferences
Forums
Photo Albums
Reliability Radio
Benchmark
Calendar
Gadgets
Jobs
Twitter
Network Links
Archives
en Español
XML/RSS Feeds Advertise

Reliability Program Scorecard
— Description and Use
by John S. Mitchell

Click here to download the Reliability Scorecard (.xls)

Click here to download the Reliability Scorecard (.zip)

Introduction:

Scorecards are becoming an increasingly popular method for identifying the elements necessary for success in a specific category of Maintenance and Reliability, weighting their importance and providing an objective score to measure performance. Scorecards are available for evaluating the effectiveness of work and stores management processes (Reliability Magazine Volume 10, Issue 3). During 2005, Jack Nicholas introduced a comprehensive RCM scorecard to measure the extent and compliance of a RCM program. The Reliability Program Scorecard adds to this body of knowledge with a list of the principal elements of a comprehensive reliability program and proposed weights. The Reliability Scorecard is presented in a spreadsheet format that is easily optimized for specific applications.

A draft version of the Reliability Scorecard was thoroughly discussed during four 90-minute workshops conducted at IMC 2005, sponsored by reliabilityweb.com. Additions and modifications suggested by workshop participants were incorporated into a second draft circulated for comment. This first version incorporates all comments and suggestions received to date.  

Scorecard Objective:

The Reliability Scorecard is the result of a several year effort to describe the total scope, content and effectiveness of a comprehensive reliability program. The scorecard is intended for use to define elements that should be considered for inclusion in a new reliability program as well as provide a basis for auditing the content and effectiveness of an existing reliability program.

Scorecard Overall Description and Content:

The scorecard, in spreadsheet format, divides a comprehensive reliability program into twelve program categories. The scorecard consists of two worksheets as follows:

1.      Category Summary

2. Detailed Scorecard

The first worksheet of the Scorecard, Category Summary, lists the twelve program categories and the highest weighted elements within each category. The second column on the Category Summary, Element Weight - %, calculates automatically through the Detailed Scorecard (second worksheet) from Category Weight - % entered manually in the third column. This will be explained in detail later.

Note: Elements in each of the twelve categories listed on the Summary worksheet have been sorted by weight in descending order. This is intended to emphasize relative priorities of the elements listed. It results in a different sequence than will be found on the Detailed Scorecard. If there is any question regarding origin, the element weight cell will identify the row in the Detailed Scorecard.

It also should be noted that the wording of elements on the Category Summary may differ slightly from wording on the Detailed Scorecard. The description and quartile ranking on the Detailed Scorecard provides an assessment of effectiveness. The Element Weight listed on the Category Summary is the maximum. If a program has high plant and system reliability, optimum maintenance costs, minimum emergency work, to name three elements, the Element Weight listed on the Category Summary is the contribution to total program effectiveness. Taking the logic further, the Category Summary implies that a reliability program with the elements listed, performing at best practice levels, constitutes almost 80% of an optimum program.

The second worksheet is the Detailed Scorecard. The Detailed Scorecard lists elements of each of the twelve program categories in ten columns. The leftmost column lists individual elements that are considered necessary for each program category. The next three columns, from left to right, define necessity, comparative metrics and any additional requirements such as a charter or procedure that must be published to meet requirements.

The fifth and sixth columns assign percentage weights for each element within the category. Individual weights for each element within the category are manually entered as a percentage of 100% in the fifth column.  Element weights within the reliability program as a whole are automatically calculated based on the relative weights entered in the third, Category Weight, column on the Category Summary (first worksheet) and repeated in the top row of each category.

Note: The spreadsheet is constructed such that changing a Category Weight (third column) on the first worksheet, Category Summary, will apply the change to the corresponding category in the Detailed Scorecard. All program element weights within the scorecard category automatically recalculate using the relative weights in the fifth column of the Detailed Scorecard. The only requirement when adjusting either category weights in the Summary sheet or element weights in the Detailed Scorecard is to manually maintain the total at 100% shown in the last line of the Category Weight column in the Summary, top line of the Element Weight column in each category on the Detailed Scorecard. 

To provide a check on the math and formulas, the Program Weight listed in the top row of each category in the Detailed Scorecard, shown as a percentage, is the value transmitted from the Category Summary. The last row in the same column, expressed as a decimal, sums the individual program weights for each element within the category. If the formulas and entries are correct the two numbers (absolute) will be identical.

The four rightmost columns on the Detailed Scorecard denote quartile spread between the best performers and the rest. As an example, first quartile performers should have an overall plant, system and equipment availability greater than 95%. An overall availability less than 70% indicates fourth quartile performance. It is recognized that there may be adjustments required for specific processes and industry groups.   

Several essential elements of a total maintenance program such as Work and Stores Management processes are not listed beyond metrics that are key to the effectiveness of a reliability program. For other essential elements such as RCM and RCFA, the scorecard only asks whether the program is in place and effective. In both cases, details necessary to evaluate the completeness of these processes / programs and assess their effectiveness are available in literature on the subject. The details will be necessary to assign partial scores indicating the completeness and effectiveness of the program.

An abbreviated glossary from the draft fourth edition of the Asset Optimization Handbook is located directly below this description to answer any questions regarding terminology or how a specific term is used within the scorecard.

Use of the Scorecard

The scorecard is yours and can be used in any way you wish. As one way to get started you might begin as follows:

1.      Review the Category Summary worksheet to determine if all the elements you consider important to your site are listed.

During the preparation of the scorecard, and especially during the workshops conducted at IMC 2005, it has been very apparent that a wide diversity of opinions exist on the composition of a reliability program and weight of elements depending on the specific facility and type of process. Hopefully this scorecard will provide you with a good starting point.

2.      Review the Detailed Scorecard, second worksheet. Are there any elements listed on the Detailed Scorecard that are important to your site but don’t appear on the Detailed Scorecard or Category Summary? You can easily add to the Scorecard or Summary

It is assumed that that you are sufficiently competent with Excel to be able to add rows on both the Summary and Scorecard, insert the correct formulas in the weight cells and the necessary formulas on the Summary and Detailed Scorecard to link the two. If there is any doubt, you can view current cell formulas and simply copy an existing cell formula into an added cell and revise the references as necessary.

3.      With all the important elements listed on the Category Summary, adjust the weights to obtain more or less the importance you consider applicable for your facility. You may have to adjust weights on the Detailed Scorecard as well to obtain the results you are seeking.

Remember, to make the weights work you must manually adjust the weights of each category on the Category Summary and individual elements on the Detailed Scorecard to total 100%. 

Acknowledgements

The contribution of all participants in the IMC 2005 workshops is greatly appreciated. Your comments and suggestions contribute significantly to the range and depth of the Scorecard and the value that will be gained by all users. Special thanks to Terrence O’Hanlon for encouraging the effort and sponsoring the four workshops at IMC 2005. Very special thanks to Jack Nicholas for continuing words of encouragement, thoroughly reviewing multiple drafts and numerous highly constructive comments.

I hope the Reliability Scorecard is helpful for you. Please feel free to contact me if you have any questions or ideas for improvement.

Sincerely,

John S. Mitchell
Post a message for John at MaintenanceForums.com

Glossary of Terms

Availability:

The time a system or asset is in an operable state capable of meeting all mission requirements

Asset, Physical:

Fixed and rotating equipment, fired and unfired heat exchangers, electrical distribution components and cabling, control instrumentation, piping, valving and structures

Asset Utilization:

Percentage of time a plant is operating at Maximum Demonstrated Production Rate, with perfect quality and defined yield

Benchmarking:

A process for measuring “best practice” performance and comparing the results to current performance in order to establish performance objectives and identify opportunities for improvement. The comparison to “best practice,” often called a GAP analysis, leads to a prioritized array of optimizing changes directed to gaining “best practice” levels of effectiveness.

CBM:

See Condition-Based Maintenance.

Computerized Maintenance Management System (CMMS):

A computer system for measuring, managing, and analyzing the maintenance process. Includes Maintenance Repair Overhaul task planning and scheduling, inventory control and management, labor and material cost accounting.

Condition Monitoring (CM):

The process of recording measurements that define condition without disrupting operation (e.g., vibration, fluid and electrical characteristics, and thermal gradients) and comparing each to limits.

Condition Based (or Predictive) Maintenance (CBM / PdM):

Maintenance based on actual condition (objective evidence of need) obtained from in-place, non-invasive tests and operating and condition measurements

Effectiveness:

Performing the correct task efficiently – results oriented

Efficiency:

Performing a given task (not necessarily the correct task) well – task oriented

Emergency, break-in Work Orders:

Work Orders for reactive work that break / interrupt an existing work (typically weekly) schedule. Some organizations may have several levels of break-in work depending on safety or production impact, start requirements (immediately) and automatic overtime authorization. 

ERV:

Estimated Replacement Value: The current cost to reproduce the asset capacity or production output, same as RAV

Failure:

No longer capable of performing the intended function

FA / Failure Analysis:

See RCFA, Root Cause Failure Analysis

FMECA:

Failure Modes, Effects, and Criticality Analyses: An integral part of RCM directed to determining type, probability, cause and consequences of potential failures.

KPI:

Key Performance Indicator – metrics that indicate performance to high priority objectives

Metrics:

Objective, numerical measures of performance and effectiveness

MTBF:

Mean Time Between Failure: Calendar time from the onset of one failure to the next; includes time to repair.

MTBR:

Mean Time Between Repairs: Calendar time from the restoration of one failure to the onset to the next; excludes time to repair.

MTTR:

Mean Time To Repair: Time necessary to complete a repair, restore to service

MRO:

Maintenance, Repair, Overhaul

PdM:

Predictive Maintenance: See Condition-Based Maintenance.

Planned Maintenance:

Maintenance planned and scheduled in advance of commencement by a specified lead-time (planning interval), typically one week.

Predictable Capacity:

Ability to meet future product delivery requirements on time, quality and cost

Preventive Maintenance (PM):

Time based maintenance tasks including inspection, service and / or replacement conducted at regular, scheduled intervals based on average statistical / anticipated lifetime. Task intervals may be based on calendar or operating time.

Proactive Maintenance:

Typically non-repetitive activities and actions applied to equipment prior to and during operation to prevent problems, gain greatest reliability, and minimize failure. Examples include pipe flange and precision shaft alignment, precision balancing, installation of lubrication reservoir filter breathers and moisture traps.

RAV:

Replacement Asset Value: The current cost to reproduce the asset capacity or production output

RCFA:

Root Cause Failure Analysis: A disciplined, formalized process to identify the fundamental defect that caused a failure

Reactive Maintenance:

Corrective action taken upon failure or obvious, unanticipated threat of failure

Reliability:

The probability that a system, device or component will perform its prescribed duty without failure for a given time when operating correctly in a specified environment.

Reliability Centered Maintenance (RCM):

A systematic, disciplined process to ensure safety and mission compliance and system function.

The process defines system boundaries and identifies system functions, functional failures, and likely failure modes for equipment and structures in a specific operating context. It develops a logical identification of the causes and effects (consequences) of system and functional failures to arrive at an efficient and effective asset management strategy to reduce the probability of failure.

Repair:

An invasive process to correct a defect; interrupts availability

Risk:

The probability multiplied by the consequences of an event

ROA:

Return On Assets: Profit divided by asset value

ROE:

Return On Equity: Profit divided by corporate equity

ROI:

Return On Investment: The profit gained from an investment divided by the monetary value of the investment

ROCE:

Return On Capital Employed

RONA:

Return On Net Assets: sales minus expenses divided by total assets

Scheduling:

Combining the resources: procedures, people (skills), parts and operating line-up to perform a given task effectively. Sequencing work to assure most effective use of resources, production availability.

Streamlined RCM (SRCM):

Streamlined Reliability Centered Maintenance: An optimized form of RCM that begins with a risk rank prioritization to assure resources are applied most effectively to equipment and systems with highest potential for value and return and builds from templates and predetermined maintenance lists to assure all potential failures and corrective action are considered.

Total Productive Maintenance (TPM):

A multi-discipline, team-based approach to equipment management that emphasizes cooperative maintenance (autonomous maintenance), team activities, zero defects / zero loss operations, and cleanliness. Pillars of TPM include:

§         A multi-discipline, team-based approach to equipment management that emphasizes cooperative maintenance (autonomous maintenance), team activities, zero defects / zero loss operations, and cleanliness. Pillars of TPM include:

• Improve equipment effectiveness and reliability

o        Target management and quality metrics

o        Failure source location and correction

•  Autonomous maintenance by operators

o        Checklists and accountability

o        Basis for Clean, Lubricate, Adjust, Inspect, Repair (CLAIR)

• Preventive and Condition-Based Maintenance by maintenance department

• Training to improve operation and maintenance skills

• Early equipment management – improve equipment design, manufacture and installation

o        Prevent problems occurring during new plant or equipment startup

Unplanned Maintenance:

Post a message for John at MaintenanceForums.com