| GISdevelopment.net ---> GITA 1998 ---> Data Evolution |
Migration: The Gateway to Company-Wide GIS for REMU
Wim J.A. Smit
Executive Manager, Utrecht Region
REMU
Croeslaan 28, P.O. Box 831
3503 RH
Utrecht, Netherlands
John Buechler
Director of Client Services
MSE Corporation
941 North Meridian Street
Indianapolis, Indiana 46204
USA
Wim J.A. Smit
Executive Manager, Utrecht Region
REMU
Croeslaan 28, P.O. Box 831
3503 RH
Utrecht, Netherlands
John Buechler
Director of Client Services
MSE Corporation
941 North Meridian Street
Indianapolis, Indiana 46204
USA
Introduction – Why Migration?
System obsolescence can be a powerful motivating factor for utility companies. Engineering departments that have come to rely on the powerful design capabilities of GIS technology may find themselves bogged down in backlogged workorders created by a system than can no longer handle the workload. CIS and other information and data dependant departments are finding their analysis outmoded. Information technology strengthens the utility organization in many ways – efficiency, cost effectiveness, accurate analysis, timely customer service – it can be crippling to discover that the very tool that you rely on so heavily may be letting you down. REMU looked at this situation very closely and determined that system migration was a critical business decision.
ln addition to internal workflow issues, deregulation in the Netherlands is producing very real business pressures onutility companies. REMU, inparticular, recognized that certain strategic advantages could be gained in the new, unregulated marketplace if the utility was able to utilize the power data housed in its GIS. Increasing competition for customers being the overriding factor, REMU knew that enhanced customer service would be a key to conquering some of the uncertainties of the future.
Deregulation Process in the Netherlands
First, it’s important that there is an understanding of the Dutch energy situation. Energy production, main grid, transport/distribution and customer services are traditionally local activities. For more than 100 years, the large energy companies in the Netherlands have been owned, and often operated, by the local governmental authority; municipalities and provinces. There were approximately 180 of these companies; some large and horizontally and vertically integrated, most small. The government’s involvement was restricted to legislative affairs, agreeing on the production planning for the main grid within the local operation, and setting the prices for the customer and the industry. The main grid operator (the local entity) had responsibility for energy exchanges with other European countries.
About 10 years ago, the government introduced the first major changes to the Dutch energy system. From now on, production (and the main grid) would be separated from transportation and distribution. The resulting pressure on the horizontally integrated distribution companies forced an industry fallout that created four (4) production companies where there once were 12, and about 30 distribution companies.
Statistically, the Netherlands as a market looks like this:
Two actions result from this. First, the activities of the distribution company (core business electricity, gas, district heating) and the commercial activities (street lighting, telecommunication, HVAC, etc) must be separated into different companies. Second, the network (WIRECO) and electricity sales (ENCO) must be separated into different companies. In REMU’s case, the company will be split-up into at least three (3) new entities with their own Boards of Directors and, most likely, different shareholders.
The WIRECO has a geographical monopoly, therefore requiring the introduction of a regulating agency that watches the behavior of this company very closely. Performance, price, and responsibility for the development of sustainable energy are the key issues for the future.
One of the most important issues for the WIRECO near-term is asset management. A state-of-the- art geographic information system (GIS) is required to answer future questions. REMU’s GIS will be a significant contribution to the utility’s ability to deal with the specific issues brought by deregulation, such as asset inventories, load profiles, customer demographics, billings, and operating a transmission and distribution system. There is some speculation that one utility’s system information may have to be compatible with other utility systems (a true open architecture issue). This thought process means that old, outdated GIS platforms will not only be inconvenient, but a potential hindrance to the ability of the enterprise to operate effectively.
Applications have driven REMU’S migration decision. Justifying these enhanced applications required some creativity on the part of the GIS project team. The first step was getting buy-in from current and future users. Each was asked to outline what functions from the old system they used and which ones weren’t used. Then they were asked for a “wish list” of applications. Once the applications goals were identified, the project team had the users commit to the plan in writing. This approach allowed the project team to expend valuable resource (both time and money) only on the applications that would be truly useful to the utility organization.
Project Description - Keeping IT Simple
REMU is an investor-owned utility from a legal viewpoint, but the investors/shareholders are the two main municipalities of Utrecht and Amersfoort and the province of Utrecht. However, the company is operated as a normal investor-owned company. REMU serves a territory of approximately 216,000 customers in a 120 square kilometer (30 by 40 km, about 20 by 30 miles) area in and around the city of Utrecht. Services provided include gas, electric, heating, cable and telecom.
REMU is the result of a 1992 merger between five completely different companies. By different, we mean size, product, area served, historical background, etc. One of the results of the merger was a kaleidoscope of information systems. Starting with the review of the design of our desired information system landscape, we discovered that a major change would be required to the approach of the systems. The old systems were functional information systems. For example, financial information, personnel, and an engineering or “drawing room” system for the drawing room. This approach does not adequately support a process-oriented organization.
In addition, the management issues of tomorrow will be different from those of yesterday. No longer will a technical approach suffice for decision-making. Instead, an integrated managerial approach will be necessary. For example, maximum achievable reliability is not the goal, guaranteed reliability is. And of course, the cost of that reliability will need to be considered.
The data migration decision took into consideration network separation as well as rulebase enhancements to identify and report data discrepancies and inconsistencies. Two of the old systems were based on drawing room (engineering) functions. One addressed gas and street lighting facilities (built in the mid 1980s) and the other was for district heating (built beginning in 1990). We faced the following problems:
In the main-design model we have defined three major information areas. They are:
Organizing the System Renewal - Decision-Making 101
Once we figured out what to do, we needed to address the issues of how to do it. We were most concerned with two questions:
A complicating factor of our conversion revolved around the fact that in the old system two firmly interconnected databases operate together, a graphical database and an alpha-numeric database. In our RAISE design concept we define the database as a description of the assets.This philosophy means that the graphical database is NOT a core database but an application database, not very different from network calculation, network loading, historical data, etc.
We checked our new database philosophy against our experience with the old systems. Our conclusion was that interconnected databases give more trouble than benefits, particularly in the operational environment during conversion. Production of applications is also complicated because the world of the Oracle developer and the world of the FRAMME developer are quite different. It is much easier to build two separated applications with a well-defined interface. So we couldn’t come up with a single justification to preserve the interconnected approach. Most AM/FM/GIS systems use the interconnected or integrated design because these systems have their roots in the drawing room (engineering) environment. Regular checking is needed (off line) to compare databases.
Risks – and Rewards
On the REMU project, risks boiled down to two key areas – quality control, and the physical and cultural differences between REMU and its vendor partners. With regard to quality control, REMU took advantage of the AM/FM International organization and its strong network of worldwide utility companies that have already completed conversion and migration projects. In other words, peer to peer networking is extremely important; the REMU case study is a perfect example. As a starting point, this was an important step.
Second, REMU involved its migration vendor in an intensive review of the data model. Data discrepancies HAD to be dealt with because errors in the rulebase were likely to cause problems with the applications. The vendor supplied several pieces of proprietary software designed to ensure quality and enhance the rulebase prior to migration.
The existing data contained cartographic inconsistencies, connectivity problems, and invalid attribute data. REMU and the data migration vendor jointly customized a set of data acceptance tools. The migration process included a set of tools to standardize the cartographic appearance, insure 100% correct connectivity, and validate and update attribute data to a new set of standards. Check connectivity ensured that both the topological and FRAMME connectivity models always matched., In addition, the process generated a “remainders” file with text and features that did not fit the final data set. The DAAT, Data Acceptance Audit Tools included:
Ultimately, the attention to detail undertaken prior to and during migration ensured the integrity of the legacy data. One of the important features of the overall quality assurance process on the project was the on-site assistance. Throughout the entire migration, teams from REMU’s conversion partner lived and worked in Utrecht. The on-site staff were able to resolve problems promptly, saving time and money for both REMU and its partner.
The on-site support was a small example of how the entire project team was able to overcome the risks involved in working with a vendor separated physically and culturally from its client. It was very important from REMU’s perspective that its conversion team was willing to work on-site. In addition, problems and questions that arose as a normal part of the migration process were resolved in a true partnership approach, rather than the more traditional (and sometimes adversarial) vendor/client relationship.
For data migration we looked for a partner who would be able to anticipate the consequences of different rulebase structures on the final, converted system. We felt that if we could find that level of expertise, only then would it be possible to achieve an automated data conversion with a high quality level. It doesn’t need much explanation to understand why it’s important to have a partner and not just a supplier – quality of the solution, time needed for conversion, only minor correction work afterwards, and a reasonable price. From a business perspective, this was really the perfect solution.
During the migration process it proved necessary to have on-site support. A member of our conversion team stayed on-site in our offices approximately four months. He facilitated communication with the home office and was instrumental in communicating with the software development teams. For us, on-site support has been essential to the project’s success. The shortest lines of communication are the ones between professionals who are searching for the best solution, regardless of which side of the ocean they are on.
During the migration process user representatives were involved in the design and building phase. The advantage was that training took place “on the fly”, but this training and knowledge could be filtered back through the organization in a much more broad way. Before we delivered the system to all users there was an introductory training session. We believe that the best final training will be in the operational area. The users will learn the system best just by doing their job using the new tools. The goal becomes delivering exceptional help functions and user descriptions.
REMU’s vendor partners had a very good understanding of the business reasons why the utility needed to migrate its system. Issues that would affect REMU’s bottom line were dealt with using business-case approaches rather than just purely technology-driven solutions.
Lessons Learned
Migrating legacy data requires the same attention to detail and careful specifications process that a full conversion does. Clear specifications, with user buy-in and expectations satisfied, is the first step. Well-defined specifications must include user input and a willingness to engage in a brief negotiation process is critical. Once this foundation is determined, experience becomes the key. Make sure you consider options like multiple databases, Intemet access, and networks, but keep your business goals firmly in mind.
Migration is not the same and conversion, and migration is definitely NOT data translation! Be careful to choose your migration vendor carefully. Ask lots of questions and make sure that the vendor understands how to preserve data integrity through a careful review of rulebases/data models. Make sure your vendor isn’t sacrificing integrity for technology – they aren’t the same thing. Find a business partner that you can work with – if on-site work will make a difference to your company and your project, then ask for it.
Build in quality control to the migration process. Training is critical, as is controlling scope-creep. Migrating old systems to the newer, more robust technology is a powerful and seductive activity to consider. It will be easy to envision all kinds of new applications and enhanced functionality, but make sure that those things are attainable. If the data is not pure at the beginning, even the most wonderful applications won’t run correctly. Take the time to be a careful evaluator – it’s your data and it will be your responsibility (and your migration vendor’s) to ensure that it is of the highest quality.
System obsolescence can be a powerful motivating factor for utility companies. Engineering departments that have come to rely on the powerful design capabilities of GIS technology may find themselves bogged down in backlogged workorders created by a system than can no longer handle the workload. CIS and other information and data dependant departments are finding their analysis outmoded. Information technology strengthens the utility organization in many ways – efficiency, cost effectiveness, accurate analysis, timely customer service – it can be crippling to discover that the very tool that you rely on so heavily may be letting you down. REMU looked at this situation very closely and determined that system migration was a critical business decision.
ln addition to internal workflow issues, deregulation in the Netherlands is producing very real business pressures onutility companies. REMU, inparticular, recognized that certain strategic advantages could be gained in the new, unregulated marketplace if the utility was able to utilize the power data housed in its GIS. Increasing competition for customers being the overriding factor, REMU knew that enhanced customer service would be a key to conquering some of the uncertainties of the future.
Deregulation Process in the Netherlands
First, it’s important that there is an understanding of the Dutch energy situation. Energy production, main grid, transport/distribution and customer services are traditionally local activities. For more than 100 years, the large energy companies in the Netherlands have been owned, and often operated, by the local governmental authority; municipalities and provinces. There were approximately 180 of these companies; some large and horizontally and vertically integrated, most small. The government’s involvement was restricted to legislative affairs, agreeing on the production planning for the main grid within the local operation, and setting the prices for the customer and the industry. The main grid operator (the local entity) had responsibility for energy exchanges with other European countries.
About 10 years ago, the government introduced the first major changes to the Dutch energy system. From now on, production (and the main grid) would be separated from transportation and distribution. The resulting pressure on the horizontally integrated distribution companies forced an industry fallout that created four (4) production companies where there once were 12, and about 30 distribution companies.
Statistically, the Netherlands as a market looks like this:
- 200 by 300 km(130 by 200 miles)
- 15 million inhabitants
- average electric load is 4,000 kWh
- maximum load is 11.500 MW
- delivered 80 billion kWh
- main energy source for residential and commercial uses is natural gas
Two actions result from this. First, the activities of the distribution company (core business electricity, gas, district heating) and the commercial activities (street lighting, telecommunication, HVAC, etc) must be separated into different companies. Second, the network (WIRECO) and electricity sales (ENCO) must be separated into different companies. In REMU’s case, the company will be split-up into at least three (3) new entities with their own Boards of Directors and, most likely, different shareholders.
The WIRECO has a geographical monopoly, therefore requiring the introduction of a regulating agency that watches the behavior of this company very closely. Performance, price, and responsibility for the development of sustainable energy are the key issues for the future.
One of the most important issues for the WIRECO near-term is asset management. A state-of-the- art geographic information system (GIS) is required to answer future questions. REMU’s GIS will be a significant contribution to the utility’s ability to deal with the specific issues brought by deregulation, such as asset inventories, load profiles, customer demographics, billings, and operating a transmission and distribution system. There is some speculation that one utility’s system information may have to be compatible with other utility systems (a true open architecture issue). This thought process means that old, outdated GIS platforms will not only be inconvenient, but a potential hindrance to the ability of the enterprise to operate effectively.
Applications have driven REMU’S migration decision. Justifying these enhanced applications required some creativity on the part of the GIS project team. The first step was getting buy-in from current and future users. Each was asked to outline what functions from the old system they used and which ones weren’t used. Then they were asked for a “wish list” of applications. Once the applications goals were identified, the project team had the users commit to the plan in writing. This approach allowed the project team to expend valuable resource (both time and money) only on the applications that would be truly useful to the utility organization.
Project Description - Keeping IT Simple
REMU is an investor-owned utility from a legal viewpoint, but the investors/shareholders are the two main municipalities of Utrecht and Amersfoort and the province of Utrecht. However, the company is operated as a normal investor-owned company. REMU serves a territory of approximately 216,000 customers in a 120 square kilometer (30 by 40 km, about 20 by 30 miles) area in and around the city of Utrecht. Services provided include gas, electric, heating, cable and telecom.
REMU is the result of a 1992 merger between five completely different companies. By different, we mean size, product, area served, historical background, etc. One of the results of the merger was a kaleidoscope of information systems. Starting with the review of the design of our desired information system landscape, we discovered that a major change would be required to the approach of the systems. The old systems were functional information systems. For example, financial information, personnel, and an engineering or “drawing room” system for the drawing room. This approach does not adequately support a process-oriented organization.
In addition, the management issues of tomorrow will be different from those of yesterday. No longer will a technical approach suffice for decision-making. Instead, an integrated managerial approach will be necessary. For example, maximum achievable reliability is not the goal, guaranteed reliability is. And of course, the cost of that reliability will need to be considered.
The data migration decision took into consideration network separation as well as rulebase enhancements to identify and report data discrepancies and inconsistencies. Two of the old systems were based on drawing room (engineering) functions. One addressed gas and street lighting facilities (built in the mid 1980s) and the other was for district heating (built beginning in 1990). We faced the following problems:
- increased unreliability of the hardware;
- problems maintaining the hardware because of a lack of spare parts and knowledge;
- users with knowledge of the software became fewer;
- exchanging information with other systems was difficult, i.e. digital topo was a process performed by hand and was rather risky; and
- the year 2000 problem.
In the main-design model we have defined three major information areas. They are:
- information related to customers (MVS, marketing, sales and service)
- information related to production processes (IRIS – we use a system of the BAANcie)
- information related to assets (RAISE – REMU Asset Information Systems for Energy networks)
Organizing the System Renewal - Decision-Making 101
Once we figured out what to do, we needed to address the issues of how to do it. We were most concerned with two questions:
- how to rebuild the system in a 6 month timeframe (constructing the old system took about 5 years), and
- converting data from two different platforms to an entirely new one
A complicating factor of our conversion revolved around the fact that in the old system two firmly interconnected databases operate together, a graphical database and an alpha-numeric database. In our RAISE design concept we define the database as a description of the assets.This philosophy means that the graphical database is NOT a core database but an application database, not very different from network calculation, network loading, historical data, etc.
We checked our new database philosophy against our experience with the old systems. Our conclusion was that interconnected databases give more trouble than benefits, particularly in the operational environment during conversion. Production of applications is also complicated because the world of the Oracle developer and the world of the FRAMME developer are quite different. It is much easier to build two separated applications with a well-defined interface. So we couldn’t come up with a single justification to preserve the interconnected approach. Most AM/FM/GIS systems use the interconnected or integrated design because these systems have their roots in the drawing room (engineering) environment. Regular checking is needed (off line) to compare databases.
Risks – and Rewards
On the REMU project, risks boiled down to two key areas – quality control, and the physical and cultural differences between REMU and its vendor partners. With regard to quality control, REMU took advantage of the AM/FM International organization and its strong network of worldwide utility companies that have already completed conversion and migration projects. In other words, peer to peer networking is extremely important; the REMU case study is a perfect example. As a starting point, this was an important step.
Second, REMU involved its migration vendor in an intensive review of the data model. Data discrepancies HAD to be dealt with because errors in the rulebase were likely to cause problems with the applications. The vendor supplied several pieces of proprietary software designed to ensure quality and enhance the rulebase prior to migration.
The existing data contained cartographic inconsistencies, connectivity problems, and invalid attribute data. REMU and the data migration vendor jointly customized a set of data acceptance tools. The migration process included a set of tools to standardize the cartographic appearance, insure 100% correct connectivity, and validate and update attribute data to a new set of standards. Check connectivity ensured that both the topological and FRAMME connectivity models always matched., In addition, the process generated a “remainders” file with text and features that did not fit the final data set. The DAAT, Data Acceptance Audit Tools included:
- feature count’
- feature verify
- graphic verify
- attribute verify
- check connectivity
Ultimately, the attention to detail undertaken prior to and during migration ensured the integrity of the legacy data. One of the important features of the overall quality assurance process on the project was the on-site assistance. Throughout the entire migration, teams from REMU’s conversion partner lived and worked in Utrecht. The on-site staff were able to resolve problems promptly, saving time and money for both REMU and its partner.
The on-site support was a small example of how the entire project team was able to overcome the risks involved in working with a vendor separated physically and culturally from its client. It was very important from REMU’s perspective that its conversion team was willing to work on-site. In addition, problems and questions that arose as a normal part of the migration process were resolved in a true partnership approach, rather than the more traditional (and sometimes adversarial) vendor/client relationship.
For data migration we looked for a partner who would be able to anticipate the consequences of different rulebase structures on the final, converted system. We felt that if we could find that level of expertise, only then would it be possible to achieve an automated data conversion with a high quality level. It doesn’t need much explanation to understand why it’s important to have a partner and not just a supplier – quality of the solution, time needed for conversion, only minor correction work afterwards, and a reasonable price. From a business perspective, this was really the perfect solution.
During the migration process it proved necessary to have on-site support. A member of our conversion team stayed on-site in our offices approximately four months. He facilitated communication with the home office and was instrumental in communicating with the software development teams. For us, on-site support has been essential to the project’s success. The shortest lines of communication are the ones between professionals who are searching for the best solution, regardless of which side of the ocean they are on.
During the migration process user representatives were involved in the design and building phase. The advantage was that training took place “on the fly”, but this training and knowledge could be filtered back through the organization in a much more broad way. Before we delivered the system to all users there was an introductory training session. We believe that the best final training will be in the operational area. The users will learn the system best just by doing their job using the new tools. The goal becomes delivering exceptional help functions and user descriptions.
REMU’s vendor partners had a very good understanding of the business reasons why the utility needed to migrate its system. Issues that would affect REMU’s bottom line were dealt with using business-case approaches rather than just purely technology-driven solutions.
Lessons Learned
Migrating legacy data requires the same attention to detail and careful specifications process that a full conversion does. Clear specifications, with user buy-in and expectations satisfied, is the first step. Well-defined specifications must include user input and a willingness to engage in a brief negotiation process is critical. Once this foundation is determined, experience becomes the key. Make sure you consider options like multiple databases, Intemet access, and networks, but keep your business goals firmly in mind.
Migration is not the same and conversion, and migration is definitely NOT data translation! Be careful to choose your migration vendor carefully. Ask lots of questions and make sure that the vendor understands how to preserve data integrity through a careful review of rulebases/data models. Make sure your vendor isn’t sacrificing integrity for technology – they aren’t the same thing. Find a business partner that you can work with – if on-site work will make a difference to your company and your project, then ask for it.
Build in quality control to the migration process. Training is critical, as is controlling scope-creep. Migrating old systems to the newer, more robust technology is a powerful and seductive activity to consider. It will be easy to envision all kinds of new applications and enhanced functionality, but make sure that those things are attainable. If the data is not pure at the beginning, even the most wonderful applications won’t run correctly. Take the time to be a careful evaluator – it’s your data and it will be your responsibility (and your migration vendor’s) to ensure that it is of the highest quality.