GISdevelopment.net ---> GITA 1998 ---> Data Evolution

Data Conversion - Still The Biggest Challenget

Frank Jarko
North York Hydro
5800 Yonge Street, North York, Ontario M2M 3T3


Background
In 1984, North York Hydro prepared a report that recommended a project be established to develop an information system and database containing the information and records needed for the planning, design, construction, maintenance and operation of the electrical distribution system. As part of the plan, North York Hydro began an investigation of automating the existing, predominately manual, map and record system. The plan identified the following systems:
  1. A SCADA system called EDICS (Electrical Distribution Information and Control System)
  2. An AM/FM/GIS system called ARIES (Automated Records and Information Engineering System).
In 1985, a feasibility study was approved to determine the cost, institutional requirements and technical feasibility of implementing ARIES at North York Hydro. The results of the study found that the ARIES project was feasible and would provide widespread and significant benefit to the customers of North York Hydro and improve the productivity and effectiveness of the organization.

The project was delayed until 1990 due to resource shortages. At that time, an ARIES project scope for the phased implementation was approved and the ARIES Implementation Plan Phase 1 was developed by the ARIES project team. This phase defined as the pilot/prototype phase was used to develop “core” functionality for an automated working environment (ie. engineering design and document management) within a small geographic pilot area.

Aries Vision
ARIES was envisioned to be an integrated system that will provide for the automation of mapping and facility records information to support the Engineering and other related corporate processes. It will capture and record information for all North York Hydro facilities through the planning, design, construction and ongoing maintenance cycles. The ARIES database and software to support this entire facility life cycle requires the following four distinct functional components that must be clearly integrated.
  • AM/FM/GIS - traditional AM/FM database to support the design, mapping, record keeping and maintenance of the electrical distribution system.
  • Applications - functionalities required to create data and analyse the electrical performance of the distribution system and plan the necessary expansion, upgrades and modifications to the system.
  • Technical Document Management (TDM) - A common storage and access medium for the retrieval, display, edit and review of engineering maps, records and other documents.
  • CAD - will support the automation of the numerous drafting, requirements, in addition to mapping, that are necessary to support the entire engineering process.
In addition, ARIES will serve as an integrating environment for other systems which may store data relevant to its central fhnctions or may benefit from data stored primarily in the ARIES database.

Aries Objectives - Benefits to Customers
The primary objectives of ARIES are to increase productivity and contain the growth of administrative costs while providing improved management control and responsiveness to customers. The following direct customer benefits are those we identified to achieve by the successful implementation of ARIES.
  • contain future rate increases by more cost-effective use of capital through better decisions and efficient designs.
  • improve safety to staff and public by responding with more accurate plant information in a timely manner.
  • provide timely and accurate supply of information to developers and customers.
  • improve system reliability by effectively targeting and scheduling maintenance and capital rebuilding programs.
The ARIES system will provide both records information management and engineering decision and design support. The long term project objectives, which are achieved as a result of the development of ARIES, that benefit our operations and ultimately the customer are:
  • improve the productivity of staff in performing current and anticipated mapping, facilities management, and engineering design work, thereby reducing these costs.
  • reduce redundant map and facilities information so that resources are expended on improving records rather than duplicating them.
  • improve inter-departmental communications and co-ordination of work by facilitating the creation of shared, specific and accurate maps and records.
  • improve the integrity of facilities information to reduce the number of revisions and field visits required for the design and operation of the electrical system.
  • improve safety through more accurate and complete records that are in agreement with themselves and the field.
  • provide support to Planning in identifying future supply and capacity needs.
  • support more efficient maintenance planning, through more complete and timely access to historical records and the use of facility inventory reporting.
  • provide a comprehensive distribution facilities database for related work integration.
  • improve North York Hydros’ marketing analysis ability to help identify opportunities for growth, conservation, or load management.
Pilot Phase Results
Prior to proceeding with the Pilot Project development efforts, the Project Team identified the following key requirements that would be used to measure the success of the pilot:
  1. AM/FM/GIS Component
    • establish a full featured prototype database.
    • demonstrate drawing and report generation capabilities.
    • demonstrate linkage between facility representations in various drawing files (eg. in layout and related detail drawing) and associated attribute files.
  2. Technical Document Management Component
    • demonstrate computerized indexing and storage of technical documents.
    • demonstrated accessing, display, and plotting of stored documents.
  3. Computer Aided Drafting Component
    • demonstrate ability to create drawings using CAD tools.
    • demonstrate ability to edit digital files using raster/vector edit tools.
    • demonstrate user-friendly plotting environment.
  4. Engineering Design Process
    • demonstrate document of facilities through design and construction processes.
    • demonstrate capability to apply geometric constructs to facilities design and placement.
    • demonstrate ability to manage facilities through various stages of design and construction.
The project team confirmed that the success criteria were met in many cases exceeded expectations.

A conversion vendor was used for conversion of existing records during the pilot. One of the objectives was to evaluate the conversion process by using a conversion vendor. The data provided was found to be of good quality and the process cost-effective.

Data Conversion
In late 1994a data conversion RFQ was prepared to invite and review potential vendors for the project. In early 1995, a short list of 3 vendors were identified and an RFP was issued. The winning vendor was selected primarily based on price and although they had significant conversion experience, they had little experience with our platform for an electric project.

A “kick-off meeting” with the vendor took place at North York Hydros’ in the fall of ’94 to plan the roll-out of the conversion project. North York Hydro stressed that it was the people and not the technology that had the most significant role and would ultimately determine the success of the project. Therefore it was essential that there was total and continuous communication at all levels and at all times through out the project. Regular reporting routines were planned as well as “Trouble Report Forms” and documenting processes. The meeting provided an opportunity for both parties to become fmiliar with each others roles and exchange information to try and minimize anticipated questions and problems. Planned site visits were also discussed to ensure better understand of each others processes. Scheduling, data delivery format and size were also reviewed.

Source Document Verification/scrub
North York Hydros’ map documents were found to be in relatively good order in terms of completeness, accuracy and consistency. The map series to be converted consisted of 5 classes:
  1. Electrical distribution maps - primary and secondary circuits
  2. Underground structural maps - all non-electrical plant (manholes, duct structures, poles)
  3. Street lighting maps - all overhead, underground, controlled and 24 hrs supply circuits
  4. Water heater maps - all overhead and underground circuits
  5. Joint Use maps - all poles with foreign utility attachments
Before the map document scrub process could begin, the map maintenance process would have to be frozen. This would ensure consistency of chronologic data. The scrub process consisted of the following types of data verification and review:
  • completeness - are the maps complete with current maintenance data.
  • accuracy - are there obvious errors such as duplication of equipment numbers.
  • consistency - does the plant data correspond between all map classes.
These verifications could not have been done without the involvement of field staff that were able to confirm the accuracy of the questionable data on the maps. North York Hydros’ approach to field verification was mostly a reactive one. Only overhead secondary services were verified entirely before conversion, as well as data found suspect during the scrub process. The time and resources were not available to verify all plant data prior to conversion.

A relatively small pilot area was first converted. It contained a high density and wide variety of facilities. This step was found to be very useful in flushing out potential problems for the balance of the areas. As a result, adjustments were made to the conversion software, new features were identified, estimated feature counts adjusted, digital data bases cleaned up and scrubbing processes improved. As the project progressed it became clear that more staff would be required to provide an effective level of scrub. This proved beneficial to both North York Hydro and the conversion vendor in quality of final product and schedule.

Deliveries
Map deliveries were made to the conversion vendor only when all map classes were scrubbed, compared to each other and co-ordinated for consistency of data. The map sets were bundled geographically, based on one of the current map grids. There were 50 grid areas measuring approximately 2.5 km. x 2.5 km, a manageable size for us and the conversion vendor.

QA/QC
With the assistance of a consultant a quality assurance/ quality control process for data conversion was designed. It had three key components:
  1. An automated routine check for alphanumeric attribute data population.
  2. A manual cursory check for graphic data
  3. A manual random check for graphic and attribute data.
It was envisioned that these programs could be modified after conversion for use in day-to-day database maintenance. Once files passed the acceptance criteria, any errors were immediately corrected and data was posted to the master database. At this point, a significant amount of backlog had accumulated and would have to be addressed. Outstanding work orders were entered immediately so that data could be made available to users as soon as possible.

Staffing
As the conversion progressed, our staffing resources grew due to unforeseen activities, especially in the area of scrub. More important than the amount of staff, the quality of staff should be noted. Since the data integrity is greatly reliant on people, an investment in well trained, skilled staff will yield the best possible system and data. In creating a facilities model, complete with connectivity, it was imperative that staff had firm technical understanding of our electrical system. Equally as important was the continuity of staff throughout the project to help maintain the consistency of methods and information flow.

Points to Consider
A data conversion project is an enormous undertaking that is the most demanding, time consuming and costly part of an AM/FM/GIS implementation. Even with well prepared plan and well defined objectives, a project is going to experience unforeseen obstacles. Learning from others’ experience can help minimize those obstacles. Here are some of our suggestions:
  • ensure fill and continued executive sponsorship and support
  • determine an achievable schedule and adhere to it
  • ensure sufficient staff resources for unforeseen efforts
  • where possible, utilize suitable expertise
  • verifi as much data as possible prior to conversion
  • take close prior audit of accuracy and completeness of maps and drawings
  • freeze your specification before or early in the project
  • focus on the people to ensure project success
  • make regular site visits to vendor, and them to you
  • ensure open communication and team work with your conversion vendor
© GISdevelopment.net. All rights reserved.