Strategies for Interoperation between a Network Modeling System and a GIS

Strategies for Interoperation between a Network Modeling System and a GIS

Gary R. Graybill
Stoner Associates, Inc., P.O.Box 86 Carlisle,PA 17013

Abstract
This paper will explore various techniques and difficulties which must be overcome to share data between a GIS and a pipeline distribution network modeling system (NMS) . Hydraulic analysis, one of the most demanding applications for pipeline facility data, is possible only if strict topology rules are enforced and the data has nearly perfect accuracy.

After the structural and accuracy requirements have been addressed, there are numerous software development tools and strategies to consider. The resulting software can be used to facilitate bi-directional transfer of data between the GIS and the NMS. Four different interfacing approaches will be explored, along with a discussion of tools, methodologies, and other technical issues.

Introduction

Basic Hydraulic Analysis Model Requirements
Geographic Information Systems that are designed to store and manage pipeline facility data will typically contain the following types of attributes:

Typical Piveline Facility Attributes Used For Hydraulic Analysis

Facility Identifier
Pipe Diameter (Nominal or Actual)
Pipe Material
Pipe Length
Year of Installation
Year of Rehabilitation
Type of Rehabilitation
Measured roughness/friction
Pressure level/zone

Missing values for any of the above attributes may degrade, or even prevent data from being used by the Network Modeling System (NMS) . The transfer process should have a provision to provide replacement values for attributes that are mandatory, or required to be non-zero in the NMs. Values that are selected as replacements should be close to the acceptable range, but be uniquely identifiable as having been substituted. Using this strategy will assist in the detection and manual replacement of these values within the NMS.

Spatial Issues of Networks
Connectivity is one of the most critical requirements necessary for the use of pipeline facility data in a NMS. If the model is to generate the correct results, then it must correctly represent the facilities that are found in the field. In pressure/flow modeling, any molecule of gas or water must be able to travel the same path within the model, that it could take within the real-life pipeline system. If enough mis-representations of pipeline connectivity occur within the model, the results generated by the analysis would be to inaccurate for use.

Accessing the GIS data

Native GIS Customization Languages
Many GIS systems provide very effective capabilities to access the underlying data in its native structure. Examples include ESRI’S Arc Macro Language (AML), Smallworld’s Magic, Autocad’s Autolisp and Bentley Systems’ MicroStation MDL.

Database Extractions
This approach to transferring data to the NMS uses an export, or “dump” file, as the transfer mechanism. Once the export file structure is understood, all the information that is needed must be parsed and separated from that which is not needed. Examples of this type of format include ESRI’S shape files, SHL *VISION systems GINA, and Intergraph’s Standard Interchange File (SIF) .

In an effort to move toward open systems, many GIS vendors have provided support to read and write new file formats. Some of these formats, such as the Federal Government’s Spatial Data Transfer Standard (SDTS) , have been well defined and extensively documented. Software can be written to directly access the export file’s facility data, if it is sufficient to meet the needs of the NMS.

Hybrid Systems Data Access
Some GIS systems integrate existing commercial software into their product functionality. Example configurations of these integrated systems include CAD systems as a “front end” to the GIS, or a commercial relational database(RDBMS) as a “back end” to the GIS. Accessing data in this type of environment may ultimately provide advantages if specialized multiple data gateways are used.

Dealing with multiple access data gateways, or moving data from one component to the other, are also some of the disadvantages of hybrid systems. As an example, if facility data is accessed through the RDBMS, then spatial data must be moved from the spatial repository to temporary tables in the RDBMS before it can be transferred to the NMS.

Open DataBase Connectivity (ODBC)
The ODBC standard was started in the late 1980s by the SQL Access Group and X/Open to provide a standard for database interoperability. Prior to ODBC, applications had to be re-linked, and possibly even re-written, to work with each relational database vendor’s system. Microsoft decided to implement the call level interface of the ODBC standard, and has ensured its success with the participation of all the major database vendors.

ODBC offers the capability for a NMS to directly access data stored in the GIS relational database. As discussed in the Hybrid Systems section above, the spatial component of the GIS database may not be managed by the RDBMS. In these cases, the spatial data must first be moved into temporary tables in the relational database.

Object Linking And Embedding (OLE)
OLE started out as a specification to support compound documents in the windows environment. Version 2 of the OLE standard refined some of the aspects of the first version, but also added significant new features to control and transfer data to other OLE compliant applications. There are tremendous advantages in using 0LE2 to access GIS data, if the applications and tools for its use are in place.

Microsoft has realized some of the deficiencies of ODBC, and has urged software developers to use 0LE2 as a replacement for ODBC. The tremendous popularity of ODBC with database vendors and developers, along with performance improvements by 3rd party developers, has slowed ODBC’S replacement by 0LE2.

Manipulating the Data

Model Selection
Prior to the transfer of data to the NMS, it is sometimes useful to be able to specify that only a subset of the data is to be transferred. This selection process can become quite complex when subsets of the database must be considered.

Data Selection Methods:

Entire Database Dump
Area Subset Defined by Attribute
Area Subset Defined by Enclosing Polygon
Connectivity Trace

To illustrate subset selection issues, a natural gas distribution system will be used in the following example. Suppose that our desire is to transfer all connected pipes of an identical pressure level that are within a particular geographic region. To perform such a selection, both the pressure level and network system identifiers would have to be stored and maintained within the GIS. Most GIS systems are not well suited to the problems of managing gas network definitions.

Another functional technique is to have the user select one pipe, and then trace all connected pipes at the same pressure level. If the trace cannot cross either closed valves or regulators to a different pressure level, the resulting set of defined facilities will represent one hydraulically independent network. Although this method makes sense from a modeling perspective, the performance overhead of requiring a network trace to define a subset may make it unrealistic.

Object Transformations
Most GIS systems that have been designed for facilities management utilize line objects to store pipe data, and point objects to store fittings and valves. This relationship works well for a high percentage of the data which must be transferred. Unfortunately, there are two types of objects which must be transformed when they are moved to the NMs. Most network modeling systems represent regulators and valves as elements, rather than points. When using this type of representation for a network, nodes are the only locations where water or gas can be introduced and removed from the network. The node connecting elements (NCES) are then responsible for performing regulation and isolation.

Point to Element Entitv Transformation Methods:

Simple Insertion
Claim an Existing Element
Element Fragmentation

To set up a regulating element in the NMS, it’s point entity representation in the GIS must have a knowledge of the network to which it is connected. In our natural gas system for example, the regulator point in the GIS must know that the upstream connected pipe represents a higher pressure than the downstream connected pipe. Additional information from engineering support databases may be required to setup the regulator or valve element if all the necessary components are not stored within the GIS.

Once the logical components of the point-to-element transformation requirements are known, strategies can be developed to handle the spatial components. One basic approach to the spatial problem is to simply insert them into the model as new elements. This approach has several limitations. The new elements must be created so that they not only preserve the existing pipe connectivity, but must also be aligned correctly within the model if they function as a regulator.

The second transformation technique makes use of a pipe connected to the point entity for it’s representation within the NMS. The advantage in this situation is a minimal amount of spatial processing must be done. A limitation which must be acknowledged is the loss of the actual pipe length when it is claimed for use as a zero logical length regulator or valve within the model. This second transformation method works well for facility data that are very granular in their segmentation. The loss of segments and pipe length is, therefore, smaller when compared to systems that are represented by a smaller number of longer segments.

The last transformation technique addresses situations where long pipe segments exist. Claiming an entire pipe to create the needed valve or regulator would cause a significant loss of length. As an alternative, one of the adjacent comecting pipes can be fragmented into two sections to represent both the original pipe, and the new valve, or regulator element. This process works well if a new fragmenting node is inserted into the model at a standard offset from the original GIS point entity.

Model Size Reduction
Network Modeling Systems have matured in recent years, and expanded their capabilities to deal with much larger models. Model problem sizes thought to be too large just a decade ago, are now routinely modeled. In large part, this is due to advances in processor speed, larger amounts of memory, and easy to use 32 bit operating systems. Even with these advances, concern must still be expressed about the amount of detail in some GIS databases. The network analyst wants to find a way to represent the model using the least number of pipes and nodes, but still have acceptable accuracy.

Two strategies can be used to reduce models to acceptable sizes. The first is to use the tool set found within the GIS. While processing the data for export, if two pipes are found with very similar attributes, they could be combined to form one pipe in the NMS. The second strategy relies on NMS tools to perform model problem size reduction. The model is first created on a “one-to-one” basis, where every pipe within the GIS has a corresponding pipe in the NMS. After import into the NMS software, the attributes and hydraulic characteristics of the model are evaluated to decide which pipes may be combined. The advantage of the second approach is that hydraulic characteristics may now be considered as part of the reduction criteria.

Transferrring Data from a NMS to a GIS
Transferring hydraulic analysis results from a NMS to a GIS is significantly easier than transferring data in the other direction, but not without its own unique considerations. Most modern NMS software has the capability to create special delimited files which store results from the analysis. This formatted results file can be loaded into relational database tables without significant preparation.

As an example, the Oracle SQLload utility provides a language to specify which table receives the data, the file containing the load data, and instructions on how to parse and select specified columns for the table. ~ alternate technique directly accesses analysis results that have been placed into the model file when it is saved. This approach requires much more knowledge of the NFIS file format, and also takes significantly more work to access the data of interest.

Once the analysis results have been loaded into the specified tables, applications can be constructed within the GIS to provide thematic mapping displays of the data.

It is important to note that when results of an analysis have been loaded into a GIS environment, many persons with no knowledge of the analysis may have access to the data. Additional tables which contain descriptions of the analysis should be added to the GIS to enhance interpretation of the results data.

Software Development Tool Options

Native GIS Languages
As mentioned earlier, many GIS systems provide excellent languages and environments for the development of customized applications. Using an environment that comes bundled with the GIS product may be the most compelling reason for its use. The investment that many utilities have made in the total understanding of the GIS system can be more than enough justification not to look elsewhere.

Third Generation Languages (3GLs)
Third generation languages have been the mainstay of software development since their introduction in the 1960s. Engineering analysis software has traditionally been well suited to the use of the FORTRAN programming language. In the early 1980s, a new language called ‘C’ started to gain momentum in the software development community, thanks to its association with Unix workstations, and its speed and efficiency. These 3GL tools are adequate for use in integration software development, but many other alternatives also deserve consideration.

Fourth Generation Languages (4GLs) and Object Oriented Programming (OOP)
The current 4GL marketplace makes many promises to its users, such as shorter software development cycles, linkages to automated design tools, and less coding through the use of business rules. One of the issues associated with many 4GL products is their lack of official standards. Some systems interact with GIS data through ANSI standard Structured Query Language (SQL), but most other aspects of their use are completely proprietary. Powersoft’s PowerBuilder is an example of a 4GL, 37.client/server development product that has become very popular in many large Information Systems shops.

Object oriented languages such as Visual Basic and Visual C++ have also become very popular with software developers. C++ provides an exceptionally powerful 00P environment, but the effort to use it effectively may be too much for those who are less than fully committed to these products. The Microsoft Foundation Class (MFC) allows very detailed access to the Windows Application Programming Interface (API), but such intricate manipulations are usually not warranted in customized analysis interface software.

Microsoft’s Visual Basic and Borland’s Delphi products have provided easier access to the Windows API while giving Rapid Application Development (RAD) capabilities. One of Visual Basic’s biggest criticism is that it suffers from poor performance because it is an interpreted language. Delphi’s compiled executable address many of the performance issues of Visual Basic, but rely on Borland’s Object Pascal, which is a proprietary, object oriented, extension of conventional Pascal. It is worth noting that the use of Dynamically Linked Library (DLL) components with Visual Basic can relieve some of its poor performance. The DLL is a compiled binary file, which is incorporated into the application at run time.

Componentware
The component market has evolved significantly since its introduction as simple VBX additions to Microsoft’s Visual Basic. With the release of Visual Basic4, new 32 bit components called OCXS now provide object linking and embedding capabilities. Many GIS vendors also supply components that can provide embedded GIS functionality to customized applications. ESRI’S MapObjects is an excellent example of a component library that provides data manipulation and visualization for ArcInfo and ArcView databases.

Conclusion
After reviewing the data requirements necessary for interoperation between a GIS and a NMS, hopefully some of the strategies are now clearer.

The selection of a software development environment should be based on the requirements of the interface, and the experience of the interface developers. Adequate planning and investigation into available supporting tools and libraries should precede any significant software development project.