Moving Geospatial Applications Towards a Mission Critical Scenario

Moving Geospatial Applications Towards a Mission Critical Scenario

Rodovia Campinas Mogi-Mirim,
Km 118,
5 Campinas,
SP – Brazil 13088-902
Email: delli@cpqd.com.br | geovane@cpqd.com.br

Abstract

Geospatial applications have become a fundamental part of OSS (Operation Support Systems) solutions. Their integration with other corporate systems, which are components of the whole enterprise architecture, is crucial. These applications have proven to be very effective for back office activities that are already solved with some automation. Now they are pushed to a mission critical, 24 x 7, scenario. With their powerful databases systems, quality and large amount of data and precise location information, they are moving towards the front office activities. This presentation will show the way to this new scenario and two successful integration cases of SAGRE, a Geospatial Outside Plant Management System, with CRM (Customer Relationship Management) and Workflow systems.

Introductoin

The deregulation of the telecommunication market, the constant technical evolution and the customer oriented business pushed all companies to review their IT architecture in order to be competitive. To be able to compete with the incumbents, new companies (CLECs) have to offer new services, focus their market and speed up their operations. On the other hand, incumbents have also to offer new services and to reduce operational costs.

Enterprise Application Integration (EAI) provides the enabling technology for incumbents and CLECs to integrate their bussiness processes through an integration bus that will link the variety of systems, in-house or COTS (Commercial Of The Shelf), via connectors and adapters.

Geospatial applications have proven to be very effective for mapping, engineering and market analysis but it is not enough to fulfill the requirements of a competitive market. Usually these activities are carried on by the company’s engineering areas and are not integrated with the customer relationship activities, the operational area. Now Geospatial applications are pushed to be integrated with other corporate systems. Not all geospatial systems have the requirements to fit the new integration requirements. The solutions adopted in the past have to be reviewed within this new environment.

Fit for purpose

To do a correct choice of the components and approaches to data conversion and data definition that will endup in a correct IT architecture definition and in an expected return of investiment (ROI), some key aspects have to be analized.

Process Mapping

The primary need of the enterprise is to establish business processes of each operational area. Usually telecommunication companies have in mind these operational areas:

Marketing and Planing
Sales
Engineering
Construction
Operations

These areas have their own processes with specific needs and flows for data gathering and data analyses. On the other hand, some processes, such as sales, have communication with all areas of the company. After the processes and procedures are mapped it’s necessary to identify which are already automated by which system and the ones that are not and those which have to be. With the processes and systems mapped, the needs for system integration becomes clear. This task will end up in a big picture of the IT architecture and will drive all the efforts of the integration activities.

With so many systems of different vendors or in-house solutions, the semantic data problem is one of the biggest problems to have a real integrated enterprise-wide solution. Usually systems that were built for different purposes have different views of the data. This semantic problem has no immediate solution. A global data model that embraces all components of the whole IT architecture solution will be necessary.

Another problem related to the semantic of the data for a given area is the definition of the master database for each area. This decision is hard to be take because it involves technical and non-technical issues.

Data Conversion The data gathering and conversion process is well known to be the most time and cost consuming in a GIS startup projects. This is the case mainly in developing and continental countries, such as Brazil. Good digital maps are very rare, they are usually very expensive and in most cases have to be done from the beginning because of the poor quality of the sources. In order to break this barrier it is necessary to identify the need of data based on process mapping, i.e., to start building the maps with only the necessary data and enhance the maps along the usage of the system with the field crews, for example. This approach is very useful for CLEC startup process. It reduces the time and costs to start the operations.

The SAGRE solution for this problem is to classify the land base types by the accuracy and amount of data for each type. Each land base type has its purpose and is appropriate for some module of SAGRE. The description of each land base type, on a incremental degree, is shown as follow:

Land Base 1 – City limits and Urban area limits
Land Base 2 – Center lines, Districts, Street Names
Land Base 3 – Blocks, Address ranges and Zip codes
Land Base 4 – Lot boundaries and Addresses
Land Base 5 –Address profiles (consumer class, etc)

There are also a number of elements that are optional such as limits of administrative areas, counties, state and country, and other features such as curb, hydrography, roads, railways, etc. These elements are not always necessary for a Network Management applications but can be useful.

Scalability

In order to fit the requirements of a competitive market, systems must be capable of running on different hardware architectures of different vendors (mainly for Unix environment).

The scalability is not a matter of appropriate sizing of server machines. Modern applications need to have the ability to either do a load balancing by using powerful clients or use a web based approach to be able to spread the process throughout the application servers. This is called a three or N-tier architecture.

Geospatial applications have other problems mainly because of the large amount of data and also the complexity of the spatial data structures. The best approach to deal with this problem is to split the maps, i.e., databases, into operational areas such as: central office area, cities, states, operational divisions. This approach will solve performance issues. However, in an enterprise solution several other issues may arise that should be solved at application level. Examples of these problems are objects that pass through two or more areas like trunk cables or microwave links or some business process that involve objects of two or more different areas like customer address change from one city to another.

Early GIS were based on CAD tools that were simple mapping tool. The graphical data were stored in files direct into the file system. In the 90s the Geo-relational model dominated the GIS. This model is based on attribute data stored in database systems and the graphical data stored in files. Until now only a few vendors have specified a GIS with all data inside a RDBMS and provided libraries for 3rd generation languages to make spatial queries and/or have provided a 4th generation language for the same purpose.

A couple of years ago the leaders of RDBMS started to support spatial data types and spatial indexes in a object-relational model approach. They also supported the SQL3 standard that allows more complex queries to the database, including spatial predicates.

Projects based on GIS that use proprietary databases to store graphical information will have many problems trying to integrate the geospatial information to the enterprise architecture. They will not benefit from the standards, gateways, reliability and security that are given by a commercial RDBMS.

Another important issue provided by GIS based on RDBMS is the capability to handle short-transactions and long-transactions in the same environment. Short duration transactions are the most popular model that all non-spatial applications are based on. For engineering environment, this approach is not enough as a work on the field can usually take a few hours and sometimes even weeks. The concurrency control of short-transactions is well known and is implemented by all RDBMS. The locks policies are very robust to support several transactions at a time because they usually take milliseconds. This concurrency control is not appropriate for long-lived transactions. (Dias 95)

This capability is necessary because of the nature of engineering and operational environment. While the engineering area may be designing and constructing the network, the operational area may be doing assignments, changing customer addresses and selling new services. All these tasks must be done on the existing network, i.e., existing data on the database. (Magalhães 97)

The capability of constructing very complex data models is another important issue that database oriented GIS possesses. Data model is the logical and physical representation of the real world objects. These objects can be more or less complex depending on what their representation and the granularity necessary to the application are. Usually telecommunication network is the most difficult network to be modeled because of the enormous quantity of equipment, information necessary about them, the amount of vendors for each and the lack of standards. Abstraction is highly recommended in this environment with such a diversity.

The object-oriented data model helps construct complex objects because of its characteristics like inheritance, encapsulation and polymorphism. Geographic relationship between objects would also be mapped by an object-oriented data model. The geographic operators (inside, pass-through, overlapping, adjacency) should define the relationship of the objects. For example, aerial cables must be laid through the poles, i.e., aerial cables must be laid adjacent to the poles.

A key factor for all telecommunications companies in a competitive market is the ability to offer new services and new technology for their customers and change rapidly the business processes of the companies. To be able to deploy and also manage new services and changes on the network, the OSS have to be reconfigured, if it is possible, also new systems have to be added into the IT architecture because of the lack of functionality on the actual system to manages this new equipment and the services.

In this scenario legacy systems1 are the ones that have the highest probability to be replaced. However it is a great challenge to make changes on in-house and big systems. The easiest way of doing this replacement is to do it by steps. (Brodie 95) On the other hand the competitors, the need to offer new services and to insert new systems into the company will not wait until the complete replacement of the legacy systems. The solution for this problem is to integrate the systems, even if they are new, old, rigid or flexible.

Solution Decision In order to integrate all systems in an enterprise-wide solution, some key factors have to be analyzed to help decide which kind of middleware should be used.

Transport and Messaging – the ability of abstracting the communication protocol and physical network level.
Performance – the requirement to run on different platforms usually lead middleware applications to be interpreted instead of being compiled on every hardware architecture. As well the need to run on a enterprise-wide distributed environment the network traffic and latency is another point to be verified.
Platform – usually the systems that compound the IT architecture run on different hardware architectures. The most popular ones are Windows, Unix and IBM mainframes. The middleware has to have the ability to run at least in two of these hardware architectures.
Programming language interface – the integration of standard applications to the services provided by the middleware is probably the most important aspect that will determine the adoption of the solution due to the diversity of programming languages the systems were made. Abstractions like API (Application Program Interface) or IDL (Interface Definition Languages) are essential.
Facilities to develop – the quantity and quality of services provided by the middleware solution, such as directory, security, persistence, life cycle and data conversion. The facilities provided by an integration programming environment are also important.
Reliability – this is the ability to write codes or use predefined services or APIs that are trustable, i.e., it will do exactly what it is expected to do. It also refers to the stability of some services such as directory, throughout the days
Scalability – the ability to scale to new applications that come up to the IT architecture. Also the capability of the middleware services to scale through an increasing and seasonally number of transactions.
Reusability – the standardization of APIs and components provided by the middleware that can be used by any system of the company.
Synchronous and Asynchronous – some integration process need a fast and immediate response, these are synchronous type interfaces. On the other hand in other interfaces, the answer and the response do not need to be immediate, this are the asynchronous ones. Synchronous interfaces are usually related to customer relationship and are mission-critical interfaces.
Security – the quantity and quality of services like authentication, encryption, verification, etc.
Fault tolerance – the procedures and facilities that handle abnormal situations and disasters.
Routing and Workflow – the ability to automate the flow of business processes through out the systems.

There are many commercial solutions for middleware. The most popular ones are Java (RMI, Enterprise Java Beans and Enterprise Edition components) (Java 2001), DCE, ORB (Object Request Broker) based on CORBA (OMG Common Object Request Broker Architecture) specifications (OMG 2001), IBM-MQ Series, COM/DCOM. Sockets and RPC are basic tools to integrate applications but they are very low-level approach. They only provide standard network messaging facilities. XML (eXtensible Markup Language) is emerging as a data format exchange and data semantic independence and is usually associated to the middleware solutions. GML (Geographic Markup Language) is another standard that is emerging for Spatial Data interchange. (OGC 2001)

Other approaches are database tables integration and database or file based queue integration. These are specific and in-house alternatives for some specific situations.

Architecture Definition

The relevant aspects that must be considered at the EAI definition are:

Number of systems to be integrated
Diversity of programming languages
Diversity of hardware architectures
Diversity of DBMS
Network/distribution requirements
Existence of legacy systems
Existence of isolated (point-to-point) interfaces between systems
Requirement for synchronous and asynchronous integration
Response time requirement
Data conversion needs
Web-enable requirements

The more complex requirements and diversity of systems and solutions the middleware layer of EAI solution have to provide more facilities. For example, integration between three systems that are Windows based could be solved using COM/DCOM solution. Integration between systems at a Unix and Mainframe environment that have standards APIs available and don’t have asynchronous requirement, could be easily integrated through a Message Oriented Middleware. A big integration with a high number of process integration, with many and diverse systems are a typical scenario for a CORBA based solution.

Geospatial integration has its particular problems:

Complex or non-usual spatial data structures
Short and long transaction environment
Visual or graphic information content of maps and schematics
Proprietary 4th Generation Languages

Intergration Cases

Because of the initial requirements to satisfy almost 28 different telecommunication companies around the Brazilian states, SAGRE is a very flexible, powerful and configurable system. It’s based on a database based GIS and it was designed on a three-tiers architecture (data, application and presentation layers). Some of its modules are already Web-enabled and new components are compliant to the distributed object technology.

SAGRE is running at all brazilian local and long-distance carriers (the incumbents) and was also chosen in a formal bid by three out of four CLECs. The integration cases of SAGRE with a CRM and Workflow Management system that will be shown are from one of the incumbents and one of the CLEC. All integrations were with the Operation module of SAGRE that manages the copper and optical cable and WLL facilities of the telecom network.

The incumbent case is the most critical one because of the solutions adopted and the legacy system involved at the solution. Figure 1 shows the incumbent case. The requirements of this integration were:

Mainframe integration
Performance – very large amount of transactions per day
Land base type 4
Wire line facilities assigned by geography
Short and long-transaction at the same environment
High availability

This company have been using SAGRE for a long time. The geospatial database is been prepared and now at about 7 millions of terminals are converted on top of a land base type 4. The operation module had finished its pilot phase and will be operating about 300 thousand terminals at the end of this year. This module is replacing part of the legacy CRM system: the facility assignment functionality. The integration solutions adopted on this company was restrictive mainly because of the structure already installed, specifically the Workflow system. SAGRE had no choice to offer a better integration solution. On the other hand the flow between SAGRE and CRM were specified by CPqD and this was the better approach between cost and benefit. The requirement was a reliable, fast and asynchronous interface. This pilot project, using a Message Oriented Middleware was very well succeeded and is becoming the enterprise standard.

Figure 1

The second case is of a new company that started its operations with a full integrated IT architecture. As a new startup company, it was easier to start with this kind of architecture because there were no legacy system to be integrated and all COTS had standards APIs to the EAI environment. The short time to start was the challenge of this case. In just six months all the data and IT infrastructure, including the integration bus were ready. Figure 2 shows the CLEC case. The requirements of this solution were:

Fully integrate architecture
Land base type 3
Wireline and Wireless facilities assigned by geography
Short and long-transaction at the same environment
High availability
Wireline and Wireless facilities management

Figure 2

The integration on this company was a great challenge for SAGRE because a real EAI platform was used and we had to adapt our standard APIs to fit the integration requirements of the company.

Also the Land base type 3 and WLL facility assignment was a very strategic change because the Operation module was designed for a land base type 4 and Wireline facility assignment paradigm. The changes were possible because of the object-oriented methodology design adopted in SAGRE. Conclusion

Competition in the telecommunication market is always increasing. Technology innovations and business drivers are in constant shifting. In this scenario, geospatial solutions are a differential that companies are discovering and integrating into the front office activities in order to be able to increase their market share.

To establish a correct paradigm to integrate geospatial applications in an enterprise-wide solution is essential. To choose the right geospatial solution and EAI technology is also the most important decision that must be made for old and new companies. These decisions have to be made not only considering costs but by also taking into consideration the technical and strategic factors.

References

Brodie, M., Stonebraker, M., 1995, Migrating Legacy Systems: gateways, interfaces & the incremental approach, Morgan Kaufmann, San Francisco, CA.
Dias, E., Granado, S., and Magalhães, G., 1995 The use of versions to enforce consistency in hybrid operations and design environments. Proceedings of the IX Brazilian Symposium on Database, SBC, Recife/PE, October 2-4. (in Portuguese language)
Java 2001, java.sun.com
Magalhães, G., 1997 Telecommunications Outside Plant Management Throughout Brazil. Proceedings of XX AM/FM Conference, Nashville, March 23-26.
OGC 2001, Open GIS Consortium Geography Markup Language (GML) Implementation Specification, www.opengis.org/techno/specs.htm, version 2.0
OMG 2001, Object Management Group, www.omg.org