How the west was mapped - The good, the bad and the ugly
John Milligan Navajo Tribal Utility Authority PO Box 170 Fort Defiance, AZ 86504 Jim Nelson Global Mapping Solutions Cheyenne, WY Tom Taber Miner @ Miner Consulting Engineers, Inc. 4701 Royal Vista Circle Fort Collins, CO 80528 Abstract About three years ago the Navajo Tribal Utility Authority began modernization of its operations and financial management systems. As part of this effort, we have completed perhaps the most ambitious, innovative, and geographically extensive GPS/GIS project to date mapping some 800,000 utility components with GPS based observations in a 26,000 square mile service area. Due to the limitations of existing GPS data collection systems a GPS and pen based computer field data collection and software specific to our information needs was developed for the program. Integration of the results with a new core business system and an existing energy management system is underway. This GIS data is maintained centrally and made available to our 8 local and remote offices at modest cost using our microwave radio based LAN and terminal services software. Existing raster and vector maps have been supplemented with inexpensive 1 meter and 1 foot resolution aerial photographs to provide and accurate and up to date land base. Our experiences, the good, the bad and the ugly, are described. Requirements and History of the Navajo Tribal Utility Authority ( NTUA) The Navajo Tribal Utility Authority (NTUA) is the largest Native American owned and operated utility company in North America. It was established by the Navajo Nation in 1959 to provide electric, gas, water and wastewater service to the Navajo people. The NTUA serves an area of about 26,000 square miles – more than many States - in the “Four Corners” portions of Arizona, New Mexico and Utah. The NTUA employs about 500 people at its central Headquarters and seven widely dispersed regional offices. The utility is unique in serving all utilities in an area with the near lowest population density in the U.S. Some 33,000 electric (including 219 photovoltaic), 29,000 water and and 12,000 natural gas customers are served by about 7000 miles of electrical distribution lines, 85 individual Public Water Supplies, 70 wastewater collection and treatment systems, and 13 natural gas distribution systems. Like many utilities, NTUA prepared and maintained drawings of its facilities that soon became outdated, inaccurate, less than useful and potentially dangerous. Problems with maintaining records were compounded by the size of the service area, the number of remote offices and robust rate of growth. In 1990, a joint project was undertaken with three other REA borrower utilities to develop geographic information system intended to simplify map maintenance and dissemination, and to interface with an electrical distribution analysis software package. The project used the ESRI software suite, which NTUA has used since. In a few years the software development project failed. A few staff continued in-house development of the NTUA GIS by automating as-built drawings and geographically registering these to an in-house developed landbase with the assistance of GPS equipment. GPS mapping of proposed facilities was done with great benefit as well. It eventually became obvious that more focused effort would be required to provide a system in which accurate maps and supporting records of the utilities could be created and maintained. It would also be necessary to create and maintain accurate maps and records. In addition, utility modeling software packages – Stoner Synergee™ for electric systems, GasWorks™ for natural gas systems and Cybernet™ for water and wastewater systems had been previously acquired. These were not being used effectively because accurate spatial data about utility system components needed to model systems correctly simply did not exist. The potential for using the product of a GPS field data collection effort to construct these models was recognized. Money was budgeted and a Request for Proposal to provide what NTUA believed to be such a system was issued in late 1999. The basic requirements were a GPS inventory of all utility system components and supporting hardware and software. Some 185 separate Electric circuits, Water and wastewater, and Gas systems – Poles, transformers, meters, valves, etc., etc. were to be mapped in the field. A software system to store and maintain the data, and permit its use for mapping, maintenance and system modeling purposes was included as part of the RFP. As it has turned out, about 800,000 point and line features were mapped during this project. This required collection of about 6,000,000 items of information in the field. The project represents the geographically most extensive inventory of which we are aware. It is the only utility GPS project that we are aware of that set out to design the utility system model around the possibilities and limitations of GPS data collection rather than around conversion of paper or digital records. More than 20 proposals with widely varying approaches to mapping and data display management needs were received. Proposed data capture techniques ranged from some sort of aerial image capture to survey-grade GPS mapping of individual utility system components. Software proposals varied from out of the box use of available utility system packages through customization of existing CAD and GIS software to development of new, custom software. A committee comprised of engineers from each of the utilities and a GIS/GPS specialist considered these. In April of 2000 Global Mapping Solutions (GMS) of Cheyenne, Wyoming was awarded the contract. GMS was to provide the GPS data collection services. GMS partnered with Miner & Miner (M&M) of Fort Collins, Colorado to provide GPS data conversion, Quality Analysis/Quality Control (QA/QC) and ESRI software based ArcGIS™/ArcFM™ customization services for the product of the fieldwork. A hardware and software system consisting of 10 ArcGIS/ArcFM™ and 15 ArcView8™ licenses, three Windows based Compaq™ servers supporting the GIS software and an Oracle™ based geodatabase was acquired to serve as the medium for the GIS. Citrix Metaframe™ software is used to permit access to the data over the corporate WAN, which includes a 900 MHz microwave component for the 7 remote offices. GMS, in conjunction with Spatial Data Resources of Topeka, Kansas developed a GPS field data collection application, FMGPS™, for the NTUA project. FMGPS was developed using ESRI’s MapObjects™ and Trimble Toolkit™. The software allowed us to build a GPS data collection system around the framework of the existing ESRI and Arc-FM data models, permitting smooth conversion of field data into ArcFM. Simply put, this software allows the operator to map and describe utility systems directly in the field rather than by collecting points (like poles or valves) in the field, then having someone else connect the dots (generally incorrectly in NTUAs experience) in the office. Past experience lead to our placing a great deal of value on being able to construct geometric networks such as pipes correctly in the field rather than in the office. It proved possible, with proper preparation, to collect more information about utility systems on the ground than it would have been to reconstruct them from records, even if accurate records had existed. Database Design and Implementation The GIS database design effort started with the ESRI Electric, Gas and Water/Wastewater Distribution Data Models. Comprehensive data models based on these generic models were developed for each of the utilities. NTUA like other utilities has specific requirements and needs beyond what existed in the prototype data models so significant modifications were required for each utility. For example, new fields were added to the electric model to accommodate RUS/REA assemblies. A great deal of time and effort was spent fitting the GPS field data requirements specified by NTUA into the ESRI Electric Distribution Data Model standard model. A number of meetings between NTUA staff, M&M representatives and GMS personnel were held over a period of months before the model was declared final. A second database design effort was required. FMGPS uses a MS Access database to define categories and permissible attribute values to be collected in the field. The database controls what information could be collected in the field. Customization and precise coordination of the FMGPS model with the ArcFM data models was essential so data in the ESRI shapefiles produced by the field data collection effort could be converted in to the format required by the ArcFM geodatabase. The FMGPS data model consisted of 79 feature classes, accommodating 1435 data fields, and 125 Valid Value tables. In hindsight it may have been technically much easier to use the field data model as the starting point for modeling the system. Refinement of the models continued for more than a year and one half after initiation of the GPS field work – sometimes because of situations discovered in the field and sometimes because NTUA decided that a valid business need existed for additional or different information from the field. These changes presented a moving target to data collectors, database and conversion developers and GIS implementers and significantly complicated the project. A Pilot field data collection effort was made for each utility in August 2000 – prior to finalization of the data models. M&M and GMS used the results to finalize the QA/QC software and conversion routines and the field data collection project was in progress. Field Work Field data collection was performed using Trimble ProXR GPS mapping grade data collection equipment connected with Fujitsu pen-based computers with the, and FMGPS™ software. All data was corrected in real time using Omni-Star ™ GPS corrections. The equipment was attached to a custom fabricated operator harness that permits hands-free mobility between data collection points. GMS deployed four two-man crews to perform the fieldwork for the project. Each crew was extensively trained on the technical aspects of the NTUA utility system. Each crew was equipped with a Fujitsu Pen Computer, a Trimble Pro-XRS GPS receiver, and FMGPS ™ field data collection software. In addition, each GMS vehicle had NTUA VHF radios installed to facilitate communication. Mobility consisted of a four-wheel drive pick-up truck and an All Terrain Vehicle for each crew. The major challenges in the conducting the field inventory included communication, logistics, and remoteness of portions of the NTUA service area. Often times the field crews operated in areas of the Reservation where communication was very difficult. It was fortunate that each crew was equipped with VHF radios programmed to operate on all 13 of the NTUA radio frequencies. This allowed the crews to communicate with the GMS Project Manager and with the NTUA. In addition, each crew had a Cellular phone to aid in effective communication. Still, because of the remoteness of the area, there were many ‘dead spots’, from which crews could not communicate. Logistical issues in conducting the field inventory were also of great concern. Each crew was equipped with first aid kits, inverters to charge batteries, extra batteries and cables, emergency compressors to pump up flat tires, and other items to facilitate data collection efficiency. It was common for a crew to be 3 hours away from any support by GMS management, which required each crew to be self-sufficient. The remoteness of the area may have been the greatest challenge of overcome. The Navajo Reservation is some of the more remote area of the United States. Many areas where the NTUA has facilities are essentially un-populated. Elevations on the reservation generally range from 5000 to 9000 above MSL. Temperatures range from over 105 F in the summer to –30 F in the winter. In addition, even though there are a number of maintained primary and secondary roads on the Reservations, many of the roads are un-maintained, and become impassable in rainy or snowy weather. Also, most of the NTUA facilities are not located near any road, making access to facilities to conduct the field inventory even more difficult. Some of the measures GMS employed to cope with the remoteness of the area were the use of All Terrain Vehicles to conduct the inventory. In addition, each crew possessed a Laser Range Finder to allow for facility inventory when the feature was impossible to access. Asset Recovery Since most of the Water, Sanitary Sewer, and Gas systems lay underneath the ground, unique methodologies were employed to recover and record features related to these systems. NTUA field personnel from each of the District Offices were responsible for locating these underground features. Some extraordinary measures were employed to assure this. As an example, Gas locate crews many times had to ‘dig up’ lines due to the fact that some older gas lines constructed of polyethylene did not have tracing wire. Many times locating water lines required the excavation. While the pipelines and other features were located by the NTUA, marker post and ‘whiskers’ were placed on the ground in the proximity of the features to assist the crews in conducting GPS mapping. In addition, while mapping water and sanitary sewer systems field crews utilized NTUA As-Built maps. There were no As-Built maps and very few other reference maps for gas to assist the crews in their field inventory mapping. NTUA personnel constructed reference maps in the field utilizing 1 meter resolution aerial photos in conjunction with locates to produce very useful maps for the crews. Quality Assurance and Quality Control Another unique feature of this project was the high level of Quality Control and Quality Assurance. It is commonplace in the utility mapping industry to accept a field data collection accuracy rate of 98%. At this rate, knowing that there were some 6,000,000 fields of data to be collected in this project, there would have been approximately 120,000 errors and omissions. The goal of the Quality Assurance/Quality Control program in this project was to facilitate an accuracy rate of 99.5%. The field level Quality Control/Quality Assurance measures in this project consisted of:
A second level of QA/QC was provided by Dog Creek™ Software. This was used to ensure that field-collected data matched the attribute domains in the data model. Therefore if a field contained a value outside of the list of acceptable list of valid values it was reported in an error log. This information and reports from the M&M tests was provided to the GPS contractor who reviewed and corrected the source GPS shapefiles as needed. In many cases no real problems existed. Questionable values were listed in logs to be scrutinized by knowledgeable NTUA staff. In some cases – especially with electric system data- NTUA engineers were not available to evaluate features shown in error logs, so an item was added to the geodatabase with a code indicating which QA/QC test a feature did not pass. About 70 types of errors were flagged. Over 1,000 features remain to be inspected, error codes checked problems resolved. Once the data was deemed satisfactory or when it was flagged, a second conversion routine converted intermediate products into the next intermediate feature. These are ArcInfo coverages for features like primary overhead electric lines or water system valves. These features were then imported into the Oracle geodatabase with standard ArcGIS™ tools. Data Maintenance Challenges Many GIS systems fail because the data is not maintained up-to-date. The NTUA utility system grows at a rate of about 3% every year. This means that about 25,000 new facilities, including services, poles and other components are added every year to the utility systems. The difficulties in maintaining the data are compounded by the vast space in the NTUA service area. A data maintenance plan, which has not been fully implemented as this is written, involves deploying FMGPS field data collection systems and software identical to those used in the inventory to each of the seven field offices. Staff from the GIS department at Headquarters will support District personnel in systematically collecting GPS data for new and changed facilities in a timely manner This data will be analyzed and converted using the QA/QC and data conversion techniques developed for the original inventory. FMGPS Designer and Estimator In the summer of 2001, NTUA employed Global Mapping Solutions and Spatial Data Resource to create a Field Designer™ Software product to assist engineers and technicians with planning and designing system additions in the field. Additionally, Field Designer has the capability for preparing cost estimates. Field Designer™ was developed as a series of tools within FMGPS™ software. Through the Designer, engineers will have the luxury of viewing current landbase sources to assist with the preliminary design work in the field. With the integration of GPS, engineers will be able to produce spatially accurate plans and as-built drawings. As design drawings are made the Designer works in concert with the Estimator to produce estimates and materials lists for the warehouse and accounting departments. NTUA GIS Landbase An accurate, comprehensive landbase is the foundation of a utility GIS system. The NTUA decided before the project began to obtain the most complete and detailed landbase available. The products that existed prior to the project consisted of 24k, 100k and 250k Digital Raster Graphs (DRGs), and a quantity of vector data from a variety of sources. Some of this data was over 15 years old. With the assistance of Landbase Systems, Inc., it was determined that 1 meter resolution Digital Ortho Quarter Quads DOQQs, having an average age of 1997, were available for the NTUA service area. The NTUA purchased these images in the winter of 2001, and with the assistance of Landbase Systems, tiled and mosaiced these images. Utilizing Mr. Sid ™ compression software, the NTUA reduced the size of the images from approximately 100 GB to 10 GB. The cost to obtain these images was about 75 cents per square mile. The DOQQ landbase was further enhanced by contracting an area photogrammetry provider to obtain one-foot resolution aerial photographs from the air. 2.7 x 2.7-mile photos were obtained for 114 of the more densely populated areas of the Navajo Nation. Images have been registered to the DOQQs producing a current and complete landbase at a very modest cost. There is ongoing discussion about improving the existing GPS road centerline data. Throughout the reservation there is a unique and extensive, and often times unmaintained network of un- or marginally- improved roads. Being able to navigate these roads efficiently is a very important factor in providing customer service and emergency response, as they are the final transportation link to many NTUA customers. Locating customers is a time consuming activity and we are strongly considering deploying computerized truck maps to assist field crews in maintenance, customer service and emergency response. It is vital to have accurate and complete road maps to facilitate this operation. We are experimenting with placing GPS units in meter reader’s vehicles to map the rural roads of most interest to us semi automatically. Lessons Learned – The Bad There were some things we did wrong including: Timely Data Model Design and Implementation. Continuing modifications to the data model for more than a year after inventory commencement required modifications to the complex QA/QC and data conversion processes and a great deal of tinkering to convert and load all of the field data. The model should have been locked down before data collection began in earnest. Company modeling requirements will change over time, but changes would have been made better later. Manpower and Skills Resources This was a very innovative and ambitious project. It was essential to involve NTUA field staff deeply involved in the process to correctly locate and identify utility features and characteristics such as underground valves and electrical wire sizes. Company personnel were sometimes taken away from the project because of outages and other pressing problems. A great deal of flexibility on the Contractors part was required to keep the inventory moving at acceptable speed. There is probably no solution. Others planning a project like this should be aware that timely company participation is vital to keeping an inventory moving. The computer-related portion of the project required a great deal of adaptation and effort on the part of NTUA staff. The original plan was for M&M to process and load all of the data. Early on, it became obvious that this was neither efficient nor practical. NTUA staff began performing QA/QC testing and data loading early on. ArcFM8 is new technology and the learning curve was steep. Training courses were not available at the beginning of the project, so ArcGIS/ArcFM was implemented more slowly and painfully than we anticipated. Others planning a project like this should recognize that they are not purchasing a turnkey system. QA/QC Process The QA/QC tests were developed and finalized very early in the project. Many of them flagged potential rather than real data deficiencies. Also, some of the tests got out of synch with the data model changes, rendering the tests invalid. No entirely satisfactory process for systematically reviewing and correcting data was developed until near the end of the project. Locking the data model early and more paying great attention to using QA/QC results would have improved the effectiveness and efficiency of the process. Data Maintenance Too little attention was paid to keeping the data up to date. Much of what was collected is more than two years old. Identifying line extensions and system modifications and incorporating them into the geodatabase is just being addressed as this is written. Preliminary review of work order data for the period indicates that about 25,000 capital additions have been made to the utility systems. Bringing the GIS up to date will be demanding. Data maintenance procedures should be in place soon after an inventory like this is begun. Integration of the GIS with Business systems An implementation of the SAP Utilities™ business software was started shortly after the GIS inventory began. Efforts to coordinate data in SAP with the GIS were overshadowed by the importance of getting the business software functional. Information regarding customers, device settings and much more could not be obtained in the field. Preliminary efforts to convert data from the SAP system to the GIS by way of text transfers have just begun. We hope to achieve more complete integration through Oracle table sharing at some point. Focused efforts to integrate the systems early on would have saved time and money in the long range. Such efforts were simply not practical at the time. Lessons Learned – The GOOD Completion and Quality of Field Work As we write, the GPS field inventory is nearly complete, basically on time and within budget. Thanks to the nature of the data collection software and process, extensive QA/QC process and quality of work on the part of field staff, we are confident that the data is excellent. Add on Products The Field Designer™ and Estimator™ modules were developed at NTUA’s request to work on top of the basic FMGPS™ data collection software that is being used for data maintenance. These are extremely useful tools for speeding development of plans for making additions to our far-flung utility systems. A single interface and hardware setup for field data collection simplifies planning for NTUA field staff and saves the company time and money. The digital photo based landbase provides a current backdrop for fieldwork and overall use of the GIS. The availability of low cost aerial photography combined with NTUA more familiar raster and vector landbase significantly increased the popularity and usability of the utility system data collected during this epic project. Similar imagery is economically available for most areas. An interface was developed by Miner & Miner to export electric features by feeder section to the Synergee ™ engineering analysis software package. This interface allows users to select a feeder based on its circuit ID and it will trace the feeder, collect the relevant information, transform some of the data attributes and then populate a series of ASCII transfer files. These text files can then be imported to a Stoner MiddleLink product. These transfer files need to follow a very specific data format that varies significantly from the way that data is stored in the GeoDatabase. The MiddleLink software reads the transfer files and then matches them to devices and conductors in an equipment database. A Synergee model is built from the resulting information that can be used to analyze the distribution system such as load flow analysis and short circuit. NTUA Field Staff Participation The intangible but perhaps most important product of the saga is the involvement of NTUAs field staff in performing the inventory. Many people – those who should benefit most from the digital maps – helped to prepare it, and know what it is. All of us involved in creating the GIS have genuine pride and a personal sense of ownership. | ||
|
|