Data Sharing:-The National Pipeline Mapping System

Data Sharing:-The National Pipeline Mapping System

Kandice A. O'Malley
Area Manager, AM/FM/GIS, Coler & Colantonio, Inc.
16360 Park Ten Place, Suite 215
Houston, Texas 77084

Historical Perspective
With the continuing advancement of technology evolution, geospatial information, its benefits and power, are becoming a critical part of mainstream business strategy in multiple sectors. In 1995, the United States Department of Transportation Office of Pipeline Safety initiated a program to map all major pipelines in the continental United States to assist their efforts in regulatory oversight of these facilities. A team was assembled consisting of representatives from both industry and various state and federal regulatory agencies to negotiate a fair and equitable program to meet the needs of the regulatory agencies. The result of this ground-breaking cooperative effort between government and industry was the design and implementation of the National Pipeline Mapping System.

This program was initiated and tied into to several state run programs (e.g. Texas, Louisiana, Oklahoma, Minnesota, California), already in process. As more and more governmental agencies begin to accumulate significant data, the desire and feasibility of sharing this data becomes a reality.

Once the information is provided to an authorized government agency, it most often becomes what is known as "public information" and available to interested parties under the Freedom of Information Act.

Pipeline companies may want to acquire much of this data as part of their business strategy in the de-regulated environment. Other state and local governments may want to obtain and offer different data layers to assist in infrastructure planning, public safety monitoring, as well as a variety of other planning and analysis functions. The data can be most helpful with a myriad of analysis and planning activities as long as the quality and content of the data is taken into account.

'Accuracy' is a relative term
Since the inception of geospatial technologies the accuracy of the positional information contained in them has been extremely subjective. Each system, each organization mapped data to their individual requirements. In order to effectively utilize any data set that is acquired, meaning it did not originate within your organization, it is imperative to discover as much as possible about the origins and maintenance history of that information.

There may be some feeling that these concepts are plebeian and obvious. There is significant anecdotal evidence that in the realm of geospatial data management of corridor data elements these are not so obvious and widely understood. For this reason metadata is a requirement for submissions to the National Pipeline Mapping System. A simplified version of the FGDC metadata standards were used. Many expressed great difficulty with the concept of metadata. If one ever was inflicted with a really poor data conversion from an external source, it would create a zealous metadata convert. Metadata simply provides the "content label" for the data set - what is in it, when it was created, how was it created, etc.

When dealing with a data set that was created and maintained by another organization or group, it is critical to learn as much as possible about the history of that data. It also provides a modicum of protection for the provider of the data set should the data become corrupted during the conversion and/or import process. If the appropriate details are documented in the metadata, it limits the responsibility of the source. There are some considerations that may or may not be obvious from metadata content, such as some of the early software platforms did not facilitate floating point double precision mathematics in their software. This means that you could potentially inherit a data set with rounding errors in the coordinate data. These rounding errors can compound, dependent upon the number of times the positional data has been moved, recalculated or re-projected.

If you receive a data set that was created and maintained in this system, you would need to be aware of the creation process and maintenance history with regard to this data, if you can possibly obtain this type of detailed information regarding your source files.

Project and Re-Project
A surprising number of people in the business of managing geospatial data sets have come to it via very indirect paths. This means that they may not have been afforded the benefit of training and exposure to the concepts of cartography and map projections. There are at least 60 different commonly used map projections for North America, each with its own idiosyncrasies and drawbacks. Some organizations do research to determine the best map projection for their data, which can result in a projection that may or may not be in common use.

With the volatile business environment, businesses merging, buying, divesting; this necessitates the import and recalculation of geospatial data sets from various business units into a cohesive, consistent database for the enterprise.

Certain popular calculators provided to translate coordinate systems and provide new values in a different map projections, have been found to have intrinsic errors with respect to certain map projections. In other cases, not all of the critical information was provided to the algorithm to complete the translation correctly such as a custom min/max value and design plane centroid value. This may not be apparent or even known about the data, but it is wise to keep in mind if the resulting locations of your import process are suspicios.

Errors may also be promulgated through the process of unloading data from one platform and importing into another. The mathematics may not be consistent throughout the process.

Map Bases
In some cases the originating survey data was extremely accurate, but the data was 'adjusted' to a map base with variable accuracy. Some of the systems do not retain the original coordinates, but recalculate the values based on the location relative to the map base. Thus, this same data imported and referenced to a fully rectified orthophoto base or different resolutions of quad bases, the relative positional accuracy of the information is significantly degraded.

A consideration with the NPMS data is also that a significant representation of the pipeline operators did not have GIT databases at their disposal to assist with submission of the data. Therefore, it may have been hand drawn on quad maps and digitized in one of the state or national repositories from those records, or digitized from alignment drawings and warped or rubber sheeted to fit a quad map base. As for other public data sets, many government entities use the TIGER maps as their base map, which is appropriate for their needs. However, a very accurate pipeline data set, imported and overlaid onto a TIGER base may create some significant errors in the relative positional display of the linear features.

Conversion from Alignment Drawings
A CAD drawing is a two dimensional plane. Some companies have been known to 'stretch out' the slope of a pipeline so that engineers would have the capability to scale actual distances along the pipe on the drawings. This is most problematic for a GIS technician. The length calculation algorithm in most GIS software returns a value for the horizontal map distance, therefore the slope chain measurement along the top of pipe must be accounted for in the database somehow. Reconciling the drawings with the actual pipe lengths can be a challenge to say the least during a conversion effort. These CAD alignment drawings were the primary source of engineering data as well as the positional information on the pipeline for the past 50 years or more. The scale and accuracy varies greatly depending on the organization and the industry. Standards and specifications evolve and change through the years. This method has served the industry well. However, the conversion process to an enterprise GIS will result in the scrutinization of all documents and reconciling to a consistent environment.

GPS - 'X' does not mark the spot
If one is fortunate enough to receive data from an organization that was an early adopter of high quality GPS survey methods, the survey standards and specifications must be considered. When performing due diligence investigation on a spatial data set, do your best to also get a copy of the engineering standards and specifications for the period that the particular data was assembled as companies will change and evolve their standards over time.

It should be noted however, that some organizations did allow the gathering of coordinate information from their field units using the handheld GPS units while selective availability was still in effect. When selective availability was removed in May, 2000 this became much less of an issue - as long as you are aware that most handheld units provide an accuracy level of approximately 15 meters. This is certainly acceptable for certain high level analysis of the data. Data gathered via handheld units prior to May, 2000 may only provide a proximal location within 100 meters or more. One organization may have defined accuracy tolerances for their survey data that is more or less stringent than another. This information is not always obvious from the metadata alone.

There is also a danger that an organization may have defined very rigorous accuracy standards for the incoming survey data, but placed the data into a mapping system with a base map that is inappropriate for and not capable of supporting the more accurate coordinate data. Hence, a very accurate coordinate position is severely degraded in order to show the correct relative position to ground features on the base map.

Summary
Please don't misunderstand that this is meant as a judgment of such data as bad or inferior. This is not the case at all. In many cases it is the best available at this time. Most pipeline companies are making Herculean efforts to convert this data to an enterprise tool that will facilitate more functions and greater access to this critical data. However, due to those challenges discussed here, it is an ongoing process. The data that is available is the best that can be presented at this time with the resources available. Nearly every organization has recognized the issues and has implemented a process of continuous improvement.

This information is offered with the utmost love and respect for all the organizations making the effort to evolve to the world of spatial data. These companies have a vested interest in the accuracy and completeness of this information and its appropriate use. There are real issues with the misuse of this data, accidental or otherwise. The considerations put forth here are offered in the sincere hope that the wealth of information such as that contained in the National Pipeline Mapping System can be fully realized. There is genuine substantial value as long as the data mines and the tools are utilized properly with full awareness of the advantages as well as the limitations of both.

For additional information:
United States Department of Transportation, Office of Pipeline Safety
Research and Special Projects Administration
National Pipeline Mapping System
www.npms.rspa.dot.gov

Federal Government Data Committee Metada Standards
www.fgdc.gov