| GISdevelopment.net ---> GITA 2000 ---> It's A Brave New World |
Successfully Managing Performance-Based Regulation
Larry Kuhl
Manager – Business Development
Coherent Networks, Inc.
One Adler Drive
East Syracuse, NY 13057
Telephone: (315) 433-1010
Fax: (315) 433-0070
E-mail: lkuhl@coherentnetworks.com
Larry Kuhl
Manager – Business Development
Coherent Networks, Inc.
One Adler Drive
East Syracuse, NY 13057
Telephone: (315) 433-1010
Fax: (315) 433-0070
E-mail: lkuhl@coherentnetworks.com
Background
Utilities have always used “key performance indicators” to measure how successful they have been in managing certain aspects of their business. Some of this information has been gathered to support rate cases or to report performance on a periodic basis as requested by Public Utility Commissions (PUC’s). Over the years, the source of this information has evolved from a paper-driven series of organization hierarchical consolidations, to information system silos, to data warehouses. The methods for developing and analyzing reports have also evolved from preparing manual reports at each level in the organization, to printing off and consolidating reports from each information system, to setting up data repositories for ad-hoc queries. Much, if not all, of the data is now available in electronic form directly from automated systems. However, there are relatively few integrated reporting systems. Even the ones that have been developed don’t provide a mechanism to focus your attention to where it is most needed. The importance of monitoring and managing key business processes is even more important today as utility commissions are beginning to incent utilities to either maintain or improve levels of service quality by linking profitability to service quality performance metrics. This shift in regulation has led to a new term – Performance-Based Regulation (PBR). Not surprisingly, utilities are looking for ways to focus their attention on the key business issues that can improve overall company performance, maximize their rate-of-return, and avoid costly penalties.
The Situation
Executives and managers have more responsibilities, more pressures, and less time to peruse a multitude of reports and make decisions in today’s more competitive and ever-evolving electric utility industry. Silos of information exist that can supply all kinds of data concerning the company’s performance. Work Management Systems report on crew performance. Scheduling systems report on commitment date performance. Outage Management Systems report on service reliability performance. Consequently, management has to access each system separately, analyze the information, and then compare the analyses from the different systems or sources to determine where they need to focus their attention. Even recently deployed ERP systems present difficulties in accessing the data and presenting it in a format that is intuitively useful to a utility manager. There are numerous reports available either directly from the systems where the data is housed or from efforts to compile and consolidate the data from different systems and organizational levels. The big questions now are: “Where should I be focussing my attention?” and “Which aspect of how the company is performing should I be spending time on?”
The whole company knows that with the unprecedented industry changes, it can’t be business as usual. Employees are told to be customer-focused, but how does this translate to the company’s overall performance and to what they can truly impact? What are the important areas that they need to focus on? Performance-based Regulation for some utilities provides the list of key metrics that must be monitored.
PUCs’ require distribution system reporting by geographic area (e.g. Operating District) and even by circuit. Other metrics have been developed and are being tied to the company’s rate-of-return. There are other corporate performance targets that have been established by some utilities that are monitored by system staff to guide strategic direction. However, is all this information immediately available in a format that intuitively focuses the appropriate management personnel on the “key” performance issue that needs the most attention? The answer from most utility executives is no.
The Solution
Provide web access to “key” performance information for everyone in the company. Everyone (not just management) needs to focus on what the company has determined to be “key” to the company’s success. Provide a comprehensive geospatial representation of performance metric information. This allows senior executives, managers, and everyone else in the company to quickly focus in on the “key” areas that need attention.
To support this representation, all of the key metrics need to be available from one application. This information needs to be integrated with geographic boundaries and the organizational hierarchy. Rather than providing a list of the summary metric data in a table, different views at different organizational levels are necessary. Having information displayed at different levels allows everyone to see the impact of their group/organization on subsequently higher levels. Conversely, this capability also allows management to see which metric(s) and organizational group(s) are driving company performance. Different views are necessary to display the same information in different formats as some people relate better to one format versus another. Management needs to be able to quickly “drill down” to both where and what requires his or her attention.
A color-coded thematic map view (Figure 1) provides a clear picture indicating where a particular performance metric is either off-target for the most recent reporting period or if it is projected to be “off-target” by year-end. A progressive color-coding scheme can be used to depict different levels of target goal achievement. This scheme works with a thematic map or an organizational hierarchy rendering depending on the nature of the metric. Typically, operating performance metrics (e.g. service reliability indices, etc.) are based on geographic areas of responsibility. Other metrics, like budget performance, are monitored at the department level as well as to geographic areas.
It is also important to review trends of information. Consequently, a view of time series data for each metric is necessary. Time series projections (Figure 2) can be used to show what level of achievement is necessary to meet target performance level and to indicate where the metric will be if the current year’s trend continues.
Having these different types of views, year-end projections, the ability to quickly “drill down” to lower levels, and company-wide information access allows all employees to intuitively focus on the metrics that require immediate attention. Being able to quickly assess performance information provides the opportunity to ask appropriate questions, develop plans, and deploy resources to address business issues impacting PBR and other key performance metrics.
Here are more detailed descriptions of information viewing options needed to support the performance assessment process:
These following categories and specific metrics have been selected because they represent a cross-section of the data that utilities and PUCs’ have traditionally monitored. Some metrics were chosen to illustrate a more effective way of monitoring performance.
By providing a comprehensive and integrated geospatial view of key performance index information, with a color-coding scheme, time series based projections, and the appropriate navigational mechanisms, management can quickly find which business processes need attention and where in the organization they need to start their efforts. Having this information available to the entire organization in a hierarchical format allows everyone to see the impact of efforts on each of these key business functions and focus on what is important to management. The ability to quickly determine which performance target needs attention so management can focus resources to address critical business issues will be key to successfully managing Performance-based Regulation and company operations in general.
Utilities have always used “key performance indicators” to measure how successful they have been in managing certain aspects of their business. Some of this information has been gathered to support rate cases or to report performance on a periodic basis as requested by Public Utility Commissions (PUC’s). Over the years, the source of this information has evolved from a paper-driven series of organization hierarchical consolidations, to information system silos, to data warehouses. The methods for developing and analyzing reports have also evolved from preparing manual reports at each level in the organization, to printing off and consolidating reports from each information system, to setting up data repositories for ad-hoc queries. Much, if not all, of the data is now available in electronic form directly from automated systems. However, there are relatively few integrated reporting systems. Even the ones that have been developed don’t provide a mechanism to focus your attention to where it is most needed. The importance of monitoring and managing key business processes is even more important today as utility commissions are beginning to incent utilities to either maintain or improve levels of service quality by linking profitability to service quality performance metrics. This shift in regulation has led to a new term – Performance-Based Regulation (PBR). Not surprisingly, utilities are looking for ways to focus their attention on the key business issues that can improve overall company performance, maximize their rate-of-return, and avoid costly penalties.
The Situation
Executives and managers have more responsibilities, more pressures, and less time to peruse a multitude of reports and make decisions in today’s more competitive and ever-evolving electric utility industry. Silos of information exist that can supply all kinds of data concerning the company’s performance. Work Management Systems report on crew performance. Scheduling systems report on commitment date performance. Outage Management Systems report on service reliability performance. Consequently, management has to access each system separately, analyze the information, and then compare the analyses from the different systems or sources to determine where they need to focus their attention. Even recently deployed ERP systems present difficulties in accessing the data and presenting it in a format that is intuitively useful to a utility manager. There are numerous reports available either directly from the systems where the data is housed or from efforts to compile and consolidate the data from different systems and organizational levels. The big questions now are: “Where should I be focussing my attention?” and “Which aspect of how the company is performing should I be spending time on?”
The whole company knows that with the unprecedented industry changes, it can’t be business as usual. Employees are told to be customer-focused, but how does this translate to the company’s overall performance and to what they can truly impact? What are the important areas that they need to focus on? Performance-based Regulation for some utilities provides the list of key metrics that must be monitored.
PUCs’ require distribution system reporting by geographic area (e.g. Operating District) and even by circuit. Other metrics have been developed and are being tied to the company’s rate-of-return. There are other corporate performance targets that have been established by some utilities that are monitored by system staff to guide strategic direction. However, is all this information immediately available in a format that intuitively focuses the appropriate management personnel on the “key” performance issue that needs the most attention? The answer from most utility executives is no.
The Solution
Provide web access to “key” performance information for everyone in the company. Everyone (not just management) needs to focus on what the company has determined to be “key” to the company’s success. Provide a comprehensive geospatial representation of performance metric information. This allows senior executives, managers, and everyone else in the company to quickly focus in on the “key” areas that need attention.
To support this representation, all of the key metrics need to be available from one application. This information needs to be integrated with geographic boundaries and the organizational hierarchy. Rather than providing a list of the summary metric data in a table, different views at different organizational levels are necessary. Having information displayed at different levels allows everyone to see the impact of their group/organization on subsequently higher levels. Conversely, this capability also allows management to see which metric(s) and organizational group(s) are driving company performance. Different views are necessary to display the same information in different formats as some people relate better to one format versus another. Management needs to be able to quickly “drill down” to both where and what requires his or her attention.
A color-coded thematic map view (Figure 1) provides a clear picture indicating where a particular performance metric is either off-target for the most recent reporting period or if it is projected to be “off-target” by year-end. A progressive color-coding scheme can be used to depict different levels of target goal achievement. This scheme works with a thematic map or an organizational hierarchy rendering depending on the nature of the metric. Typically, operating performance metrics (e.g. service reliability indices, etc.) are based on geographic areas of responsibility. Other metrics, like budget performance, are monitored at the department level as well as to geographic areas.
It is also important to review trends of information. Consequently, a view of time series data for each metric is necessary. Time series projections (Figure 2) can be used to show what level of achievement is necessary to meet target performance level and to indicate where the metric will be if the current year’s trend continues.
Having these different types of views, year-end projections, the ability to quickly “drill down” to lower levels, and company-wide information access allows all employees to intuitively focus on the metrics that require immediate attention. Being able to quickly assess performance information provides the opportunity to ask appropriate questions, develop plans, and deploy resources to address business issues impacting PBR and other key performance metrics.
Here are more detailed descriptions of information viewing options needed to support the performance assessment process:
- Thematically – Geographic representations that are color-coded to depict progressively different levels of meeting or missing the desired metric target. One potential scheme would be to use 5 levels (colors):
- a neutral color for being on target (target value to better than target by 1 confidence level that represents a significant deviation)
- orange for values worse than the target value within 1 confidence level
- red for values even worse than the yellow level (greater than 1 confidence level)
- green for values that better than the target value between 1 and 2 confidence levels
- blue for values better than target by more than 2 confidence levels
The thematic map should have a minimum amount of land base features (e.g. state boundaries, county boundaries, major roads, and major rivers) with just enough information to orient the user to his/her geographical region. The primary items on the thematic maps will be the geographic areas. Selecting a geographic area should provide a mechanism to “drill down” into that area and then give a thematic mapping breakdown of that area in the same view format. There should also be a mechanism to view a breakdown of the organizational hierarchy. This view should operate in a manner very similar to the thematic view. - Historical Bar Graphs – The data viewer should provide the user the ability to see historical trends and year-end projections via bar graphs. For each geographical area or organizational level, the historical trend over the last 13 months plus the three previous years (for comparison) should be available. Again, the color-coding scheme should be consistently applied to indicate level of target achievement. Target values and actual monthly values should also be clearly shown.
The Bar Graph View year-end forecast should only use values for the current calendar year. The graph should show the threshold level values with horizontal lines on the bar graph to guide the eye. When a value within the graph breaks the threshold, the bar will be filled with the appropriate color and fill pattern. The absolute numbers should also be indicated at the top of each bar graph projection. - Tabular Data – The data should also be available for viewing in a tabular data format. The tabular format should first start with some header/summary information about the table, and then list out all entries for that particular data view. The initial highest level view of the Tabular Data view should summarize all the metric information for the entire Service Territory. The listing should be prioritized from worst case to best case. A mechanism should be made available to view even more detailed information. Again, the color-coding scheme must be consistent across all views.
The detailed tabular data view should show a table for an individual metric. There should be a table view for each metric at each level (area/sub-area or circuit/sub-circuit). This detailed view should show both current data and year end forecast values.
These following categories and specific metrics have been selected because they represent a cross-section of the data that utilities and PUCs’ have traditionally monitored. Some metrics were chosen to illustrate a more effective way of monitoring performance.
- Service Reliability - The most common reliability performance measures used by utilities are referred to as SAIDI, SAIFI, CAIDI, and ASAI. These measures are typically monitored for the utility’s entire service territory, sub-segments of the service territory (e.g. operating districts), and for each distribution circuit. These indices are also compiled and reported both with and without major storms. Other types of outages that can be considered outside the control of the utility such as disconnecting service to a building on fire or planned outages for making new service connections are usually excluded from these indices. The calculations are made for outages lasting at least a few minutes (typically greater than 5 minutes duration). Those “temporary faults” or “momentary” interruptions that last less then this time which are cleared by automatic reclosers or feeder circuit breakers get compiled with another less common performance measure MAIFI – Momentary Average Interruption Frequency Index.
- System Average Interruption Duration Index (SAIDI) - The average number of minutes within a year that the typical customer is interrupted.
- System Average Interruption Frequency Index (SAIFI) - The average number of times in a year that the typical customer is interrupted.
- Customer Average Interruption Duration Index (CAIDI) - The average duration of a customer interruption measured in minutes.
- Average System Availability Index (ASAI) - ASAI is a positive percentage measure of service availability. It is the number of SAIDI minutes divided by the total number of minutes in a year represented as a percentage.
ASAI = ( 1 – Total Outage Minutes / Total Minutes in a Year ) x 100
- System Average Interruption Duration Index (SAIDI) - The average number of minutes within a year that the typical customer is interrupted.
- Customer Complaints - For customer complaints, many utilities look at just the raw number of complaints by type and by organization/service area. One problem with the raw numbers is that the number of customer complaints within a metropolitan geographic area could be quite different from the number in a rural area. Even when compared to a target value, these numbers can be misleading. For example, the rural area has an annual target number of 25 complaints. They actually had 30, 5 more than the target. The metro area had a target number of complaints set at 100. They actually had 105, also 5 more than the target. By using raw numbers it appears that both regions have similar performance target?
- Customer Average Complaint Frequency Index (CACFI) - The ratio of the total number of customer complaints with a user-defined service area to the total number of customers in that same service area. This index can be broken down into a number of complaint categories (like billing, missed commitments, power quality, etc.) and have an attribute of whether or not the complaint received by the company turned into a PUC complaint.
NOTE: If the time to resolve complaints is available, then the Complaint Resolution Average Duration Index (CRADI) can be monitored by complaint type as well.
- Customer Average Complaint Frequency Index (CACFI) - The ratio of the total number of customer complaints with a user-defined service area to the total number of customers in that same service area. This index can be broken down into a number of complaint categories (like billing, missed commitments, power quality, etc.) and have an attribute of whether or not the complaint received by the company turned into a PUC complaint.
- Customer Commitment Dates - Customer commitment dates can also be reported on a raw number basis. However, a more meaningful number would be one that is normalized by the total number of customers within the service area. These metrics are similar to those used for Service Reliability performance.
- Past Commitment Average Frequency Index (PCAFI) – The average number of times per specified timeframe (month or year) for a particular geographic area that a customer commitment has been missed.
- Customer Past Commitment Average Duration Index (CPCADI) – The average number of days a customer commitment is missed by.
- Past Commitment Average Frequency Index (PCAFI) – The average number of times per specified timeframe (month or year) for a particular geographic area that a customer commitment has been missed.
- Customer Contact Metrics
- Customer Call Answering Index (CCAI ) – Percentage of calls answered within a fixed period of seconds which varies between utilities and PUC’s.
- Customer Average Call Waiting Index (CACWI) – The average time a customer waits on hold before he/she is connected with a customer contact person.
- Billing Accuracy Index (BAI) – Percentage of bills required to be re-sent due to any form of utility error.
- Customer Call Answering Index (CCAI ) – Percentage of calls answered within a fixed period of seconds which varies between utilities and PUC’s.
-
Safety - The Safety Metrics should track the number and duration of lost time accidents by different utility defined geographic areas or departments. Raw numbers can be tracked and reported. However, the following indices can be used to normalize the numbers and make them more meaningful:
- The Lost Time Average Duration Index (LTADI) - The ratio of the total number of person days lost to the total number of workers in the defined geographic area. This is a measure (in days) of the amount of time that has been lost.
- The Lost Time Average Frequency Index (LTAFI) - The ratio of the total number of lost time accidents to the total number of workers in the defined geographic area. This is a measure of how often an accident occurs that results in some lost time.
- The Workers Average Lost Time Duration Index (WALTDI) - The ratio of the total number of person-days lost to the total number of accidents. This is a measure of how severe, on average, the accidents are for each geographic region.
By providing a comprehensive and integrated geospatial view of key performance index information, with a color-coding scheme, time series based projections, and the appropriate navigational mechanisms, management can quickly find which business processes need attention and where in the organization they need to start their efforts. Having this information available to the entire organization in a hierarchical format allows everyone to see the impact of efforts on each of these key business functions and focus on what is important to management. The ability to quickly determine which performance target needs attention so management can focus resources to address critical business issues will be key to successfully managing Performance-based Regulation and company operations in general.