Building a Modular Interface Between a GM and three (3) Engineering Analysis Packages

Building a Modular Interface Between a GM and three (3) Engineering Analysis Packages

Bruce R. MacAlister
Principal Consultant + President, MacA1ister & MacAlister, Ltd.,1805 Grove Avenue Richmond, VA 23220-4505

Abstract
This paper describes a rare opportunity to design an intetiace between a single AM/FM/GIS software package and three different external engineering analysis packages. These were for potable and waste water, natural gas, and electricity. The paper describes what was in common and what was not among these packages. It describes the modularity of the design in order to share the common elements.

Learning Objectives

Separate common versus unique requirements of the target analysis systems.
Select source AM/FM system functions
Verify user selection through traces

Requirements
To meet the tasks, the design and the program “code” must meet these requirements:

Produce a valid network; there should be no requirement for human intervention to make sure that each facility is validly connected. Export directly from the AM/FM/GIS; intermediate software is undesirable owing to the additional interfaces and program maintenance.
Export to more than one external system; over time, many utilities will have more than one analysis package they wish to export to. KYPIPE 2, for example, models both potable water and sewer, so it is a good base for a combined utility.
Packages like EPANet add water quality, but they are designed only for potable water.
Produce a format acceptable to each analysis system; the assumption is that the AM/FM/GIS must conform to the input requirements of each analysis package.

User Interface
The standard functions of the AMiFM/GIS are used for the process of selection. We are using Laser-Scan¹ Gothic + Williams Associates² Utility Libraries. Standard systems fimctions are:

¹Laser-Scan, Ltd.,in the U.S., Willow Pond Plaza Sterling, VA 20164.
²WilliamsAssociates, 93HiddenHollowLane,Millwood, NY10546.

Query by location, class, attributes or a mixture of these.
Stored query by class and attribute; the current stored query does not support stored location, e.g., spatial.
Trace, to include all of the supported traces, not just the traditional trace-to-source. This is especially important for piped systems where trace to tanks, pumps and regulators is the best way to pick a sub-network.
Mouse selection, obvious perhaps, but the most useful in selecting a very small sub-network.

Functions were added specifically to support selection and export. They are:

Integrity trace; to state the issue bluntly, there are too many examples where the AM/FM/GIS produces a network that is “almost correct”. The work required to fix the network seems to this author more work than that required to build it by reading the paper map and typing the table. The integrity trace is designed to produce a topologically correct network if the AM/FMIGIS has connection integrity within it.
Export; this is the process of modi~ing the list of facilities produced by the export process into the format and field contents required by each package.

Typical user process
Mouse in hand, the user sits before the map display to select the facilities to be exported as a network. Clearly the user has to be an engineer or technician who knows how the analytic package works and is clear on the issue she or he is trying to resolve. Put your imagination in gear; here are some examples.

Add sewage collection for a new development

Pan and zoom, use a query, pick from the key-map or use the gazetteer to get to the area on the map where the new subdivision is to be built.
With the mouse, select the nearest node to that location.
Do a trace from that point. A trace-to-lift-pump will get a network that can show the effect of the new load on the nearest lift pump. A trace-to-plant will provide the effect on the both the pump and the treatment plant. Unless the treatment plant has only marginal capacity, including it only complicates the study.
Examine the network highlighted by the trace to be certain that you have the network path you want. Most likely you want to see all of the load on the pump, not just the new load represented by the trace. The simple way to do that is to select a node on another path near the pump. When you get to the KYPIPE 2 editor, you will add the estimated load of the new development to the first node selected and the current load to the second path you selected. You can “deselect” facilities with the mouse or select additions.
Select the export icon. This runs the integrity trace and highlights all of the facilities to be exported. Surprise! The trace included some facilities you did not select. The purpose of the integrity trace is to be certain that the network you export has topological integrity based on the facility connection validation within the AM/FM/GIS. You can use the mouse to select and “deselect”, but you will not get a file exported until the integrity trace specs are met.
You take the file – via a sneaker-net diskette or a shared network file – to the analysis computer and start up KYPIPE 2. You enter loads, capacities, flow rates and the “what if’ scenarios using the KYPIPE 2 editor. At this point, the AM7FIWGIS is no longer referenced. This is now a classic hydraulic modeling task.

Determining the capacity of a gas pips loop

By query, pan and zoom, from the key-map or the gazetteer center the pipe loop on the map display.
With the mouse, pick a node – a meter, a junction – at an end point of the inner-most loop of interest.
Pick the trace icon for trace-to-regulator-station.
Examine the network highlighted by the trace to be certain that you have the network path you want You will see all of the paths that feed that end point node. You will see the extent of loop nesting. You may see other unexpected paths for the gas that your eyes never saw looking at paper maps.
Select the export icon. R runs the integrity trace which uses identical methods to the to the trace-to-regulator-station, so there should be no surprises. Go to the GASWorkS computer, import the file and use its editor to add load to the end point node(s) until you see major pressure drops. You can then look at the intermediate nodes to see what is constraining capacity.

Packages
Interfaces have been – or soon will be – written for three specific packages to suit the requirements of Greenwood (SC) Commissioners of Public Works. The packages are:

GAS WorkS³, natural gas distribution
KYPIPE 2⁴, potable water and sewer
Scott& Scott⁵, electric distribution

Since we know that this system will need to support several packages, we built it to be a base for at least these:

EPANet⁶, potable water with water quality
WaterCAD⁷, potable water with water quality

³Asoftwareproduet andatrademark ofBradley B. Bean,PE, 1517NorthWeberStreet,ColoradoSprings,CO 80907.
⁴Developed by the Civil Engineering SoftwareCenter,University ofKentucky, CollegeofEngineering, Lexington, KY40506-0046.
⁵A product Of Scottk Scott Systems, Inc., 500 Union Street, Suite 745, seattle, WA 981012332.
⁶Developed by the Risk Reduction Engineering Laboratory, Office of Research and Development, U.S. Environmental Protection Ageney,Cincinnati, OH 42268
⁷A product and trademark of Haestad Methods,37 Brookside Road, Waterbury, CT 06708.

Stoner SWS-W⁸, water

Stoner SWS-G⁹, gas distribution and transmission

Requirements of the AM/FM/GIS System
For export to an external analysis package to be worth the trouble, the network within the AM/FM/GIS must be correct. Facilities have to be checked for valid network connection as they are added or changed. Not many AM/FM/GIS systems can do is.

Entity relationships and cardinality must be strictly enforced
The user’s tools should allow a subset of the entire network to be extracted. Although the entire network may be needed every year or two for a complete end to end study, most of the time the user wants to make estimates on a limited sub-network.

Finding the “ends” of the sub-network is the key logic required
The sub-network must have valid connections between facilities. If the entity relationships and cardinality are inherently correct within the AM/FM/GIS, the next requirement is to make sure that they do not get inadvertently corrupted during export. Here we have the advantage of a true object-oriented AM/FM/GIS with a true object-oriented database. Since the facilities are objects with behaviors, we simply have them “report” their connectivity, so there is little additional code to write or maintain.

Either there must be absolute connection between facilities, or export must have the ability to re-validate connections during export
Looped flow must be accommodated for piped networks. In the past we could ignore looping for electrical networks, but that is changing. In an effort to improve power reliability, some utilities are practicing limited looping. Looping is a challenge to the export process since its detection is not always easy. If it is not detected, the AMFM7GIS can export duplicates of nodes and segments. Often the analysis package will detect this, so all you have wasted is time. Worse is when the package does not detect it and the resulting calculations are wrong and misleading.

No duplication of a node or segment is permitted in the exported file
The normal output to an analysis package is a node-segment table constructed by the AM/FM/GIS. The core is a table that maps the facilities in the AIWFIWGIS to their equivalent in the analysis package. An example of the differences we see are:

Some AM/FM/GIS facilities can be either a node or a segment in the analysis package. The assumption in most of these packages is that a human looks at the paper map sheets, sees a junction and codes it as a pipe fitting because it has no other significance to the study. But, if you need to know the pressures at that point or it is an end point of the sub-network being exported, it has to be exported as a node and not a pipe fitting.

⁸An abbreviation for Stoner Workstation Serviee- Water by Stoner Associates,Inc.,P O Box 86, Carlisle,PA 17013-0086.
⁹An abbreviation for Stoner Workstation Serviee- Gas by Stoner Associates,Inc.
KYPIPE 2 has separate sets of “cards” for each facility catego~. set must be in precise sequence.
GAS WorkS has just two “files”, node and pipe
Some AM/FM/GIS facilities can be either a node or a segment In addition each

Matching AM/FM/GIS facility types to the categories in the analysis package differs greatly with each package
We have found the most challenging data format limitations of the analysis systems are in field size and required data that are not naturally in the AM/FM/GIS. For example, GASWorkS limits the node name to 10 characters. As noted below, getting a totally unique, system-wide facility ID within 10 characters is nearly impossible. KYPIPE 2 requires header data that are meaningless to an AM/FM/GIS.

Finally, the AM/FM/GIS export process must be able to export in DOS, Windows and UNIX file formats, That is not usually a problem for UNIX-based AM/FM/GIS systems, It maybe more difficult with DOS or Windows systems exporting to UNIX.

Limits placed on the external analysis system
Most AM/FM/GIS systems keep the network as many short pipe and conductor segments. It is exported as a very large set of nodes and segments. This can be a problem for some packages. Some are priced based upon the number of nodes and/or segments they process. Number of pipes or number of feeders are common terms. Some packages run very slowly with large numbers of nodes and segments. “Intelligence” within the AM/FM/GIS to join segments is not dependable. Since network validity has the highest priority, our export design will not risk loss of topology to reduce the output size.

It must accommodate a large number of segments and nodes I An AM/FM/GIS with many classes or layers tends to have duplicate IDs or name when merged into a single extract. So, with thousands of facilities, AMiFM/GIS facility IDs need to be very long if they are to be unique. We have partly solved this by using “names” that are really unreadable hexadecimal hash in order to keep the size small. The major disadvantage is that the IDs are meaningless to the user and they cannot be imported back into the AM/FM/GIS, as discussed below.

It must handle computer generated node and segment IDs or names 1 Some data needed for analysis are not stored in the AM/FM/GIS. An example is the roughness coefficient of a section of pipe. Ideally the package will accept whatever the AM/FM/GIS can deliver. The user will spend much more time with the analysis package than with the AM/FM/GIS. The user will change data exported from the AM/FM/GIS, insert data not available in the AM/FM/GIS, add new parameters and loads, “change” pipe or conductor sizes and lots of similar work to ask the “what if’ questions of the package. So, the analysis package needs a good interface for user changes to the imported network.. In most cases, a “mass change” fmction is important to produce a more sparse network than can be provided by the AM/FM/GIS.

It must be able to import without all parameters complete I

Design issues and choices The issues we found are described below.

How much should the AM/FM/GIS system “know” about the external package? The minimum acceptable to the analysis package is one determinant, The more the AM/FM/GIS has to “know”, the more changes in the analysis package will cause costly maintenance to the AM/FM/GIS. Some are unavoidable. For example, KYPLPE 2 requires fidl set of valid header “cards” before it will accept the file, so we had to add user prompting in the AM/FM/GIS to gather these data.
How much choice should the user have in excluding facilities from a network? In case you did not get the point earlier, we do not allow the user to exclude anything that will corrupt the integrity of the network topology. So, the user can “deselect” facilities to his or her heart’s content, but the AM/FM/GIS will not export until the integrity trace has run successfully.
Should the AM/FM/GIS attempt to produce a sparse network? If so, how sparse?

The objective is to duplicate what a skilled technician or engineer would do in choosing only pipes of certain sizes so that smaller sizes are not included in the export. This works well in our AM/FM/GIS using queries by class and attributes. It gets tricky if there just happens to be some smaller pipe within the flow, since the integrity trace will pick it up and export it in order to ensure network integrity. Ignoring nodes and fittings by merging pipe or conductor segments of the same size is another common technique to produce a sparse matrix for the analysis package. We do not support this completely. If a facility is a node to the AM/FM/GIS, we export it as a node to the analysis package. We are working to develop a Spockian mind meldl” that figures out what the engineer is going to do with the data.

Common features of analysis packages
In the process of developing this interface, we categorized these common features of the analysis packages we examined:

They are based on nodes connected by segments.
Computations – pressure, flow rate, voltage – are focused on nodes.
The route is supply to sink, high pressure to low, source to demand.
Nodes must be uniquely named and/or numbered.
Node to segment connection must be identified either by the proximity of the records – difficult to program – or by ID or name.
Duplication of a node or segment causes invalid analysis results as described above.

¹⁰Thiswillbe in the Star Trekversion.

Typical diffemcea among anatyaii packages
Storage modeling may or may not be permitted:

GAS WorkS permits no storage
KYPIPE 2 provides storage via tanks
EPANet computes chemical change overtime for water quality, a kind of in-situ storage.

Minimum data and the nature of that data varies

KYPIPE 2 requires full headers
GASWorkS limits node “names” to 10 characters
Certain fields must not be missing
Certain required fields can be null

The table – the exported file – has varying delimiter requirements:

Some permit only commas with no spaces
Some let the user choose the delimiter
Some require delimiters in place of missing fields
Some permit columnar fields of fixed length
Some will import from a database; for example GASWorkS will read dBasel 1

Issues in importing results
Importing the analysis results into the AM/FM/GIS is not of great importance. Most modern analysis packages produce very readable graphic displays of results for engineers. Most package displays provide a graphical schematic, much more compact than a spatial or map display. Besides, time-based results, e.g. chemical or level changes, cannot be practically displayed on a map.

Limits to importing
Then there is the practicality of it all. Few packages are designed to export the results in a machine readable form for importing to a GIS. For example, KYPIPE 2 provides a printer-type report so variable in content that it cannot be reliably parsed. On the other hand, GASWorkS “fills in” the results fields in the original file so it can be imported to a GIS. Establishing readable, unique, unchanging facility “names” shared by the AM/FM/GIS and all analysis packages is currently not possible because the thousands of facilities in the AM/FM/GIS require longer names than the analysis packages can accommodate.

Benefits to importing
The primary benefit to importing the data into the AM/FM/GIS is that a much broader user community can have access to the information. Displaying selected results on an AM/FM/GIS map is more readable for most users.

¹¹dBase is a trademark of whatever company now owns it.

There is a trap that must be considered, though. You need policies to prevent misinterpretation of capacity and other results. In the hands of staff who like to give “no problem” answers to capacity questions, this information could get you in serious trouble. We have worked out some of the technical requirements. A series of additional display methods, behaviors or procedures need to be written for the GIS to display the results. A mechanism is also needed to manage multiple analysis results within the AM/FM/GIS.

Conclusion
Exporting network data from an AM/FM/GIS for use by an external analysis or modeling program has two major benefits. First, ~the AM/FM/GIS keeps accurate, validated network topology, the exported file will be more accurate than having an engineer read the paper maps and type the network into tables. Second, you save some time for your most costly professional staff.

The ability of an AM/FMIGIS to export to an analysis package can be exaggerated. It cannot produce as sparse a network as a skilled technician or engineer. That can mean higher software fees, slower performance and more voluminous output. Like most technology, you have to judge when to use it and when not.

Exporting from an AM/FM/GIS fits best for frequent, simple analysis of sub-networks. It is the sort of thing we avoided in the past because getting the data in was too costly and time consuming.

Acknowledgments
This work started out as a prototype for electric. David Turvey of Laser-Scan did the work to prove the design concepts. Mark Kennion of Laser-Scan then extended this for piped networks and took on the daunting problems of looping. Finally, Ramesh Kumar of Williams Associates had the pleasure of putting it all together into a practical system. They have my thanks for adding their great technical insight and corrections to what I envisioned.