| GISdevelopment.net ---> GITA 2003 ---> System Architecture |
Customization with Standardization: The Architectural Challenge for Corporate GIS applications
Carlos Alberto Previdelli
CPqD - Fundação Centro de Pesquisas e Desenvolvimento em Telecomunicações
Rodovia Campinas Mogi-Mirim, Km 118,5
Campinas, SP – Brazil 13088-902
Email: delli@cpqd.com.br
Carlos Alberto Previdelli
CPqD - Fundação Centro de Pesquisas e Desenvolvimento em Telecomunicações
Rodovia Campinas Mogi-Mirim, Km 118,5
Campinas, SP – Brazil 13088-902
Email: delli@cpqd.com.br
Abstract
Until now, corporate GIS applications have, as one of their main objectives, the standardization of maps, symbology, data model and data exchange formats. These applications usually sprout from engineering areas and, as the geographic data sources increase in size and quality, they eventually begin to be used by other areas of the companies. The technology available and architectural decisions made in the past attempted to leverage a rigid application structure yet fall short of meeting new users requirements. These barriers do not allow the same GIS application to be used enterprise-wide. New technologies, methodologies and emerging standards can now be combined to design a corporate GIS application.
Introduction
Over the years, corporate applications were struggling against process mapping, data standardization, interface definition and many other standards. New ideas were right at that time because the lack of standards had led corporations into chaos in terms of information.
The problems for GIS applications were even worse. Mapping issues such as the use of different scales, datum and coordinate system issues, were problems that traditional applications never needed to deal with. Additionally, the symbology representation throughout different areas of the organization was a difficult problem to solve.
To solve all these organizational problems, utilities and telcos teamed with GIS vendors to develop a corporate GIS application. These solutions started from scratch or with a vertical solution offered by some vendors as a starting point. It was then necessary to customize these basic applications to fit the particular and unique needs of both the users and their organizations.
Those that have already traveled the GIS path are all too familiar with the results. As the need for reports and queries increased and as applications grew more complex the size of the GIS databases quickly became overwhelming. This occurrence catapulted the telco and utility industries to begin to look at a broader solution designed to meet a more complete solution.
Unfortunately, the foundation of many GIS solutions and their associated applications could not support additions or enhancements to existing company requirements. Data was not available to all employees as it was not stored in a real DBMS (Data Base Management System)> Each department operated in data silos resulting in duplicate data throughout the organization, maintained independently of one another under various data access and permission rules and drawing symbology. Additionally, the IT infrastructure was not prepared for any increase in system load.
The data silo problems existed not only between departments within an organization but also between organizations interested in sharing geospatial data. Each GIS solution had its own data format and each organization had its own data model. Exchanging information among different companies such as utilities and government agencies was a difficult job, if not impossible.
All these points raised during these years were very important to help the industry come up with real corporate-wide solution. To this end, the Open GIS Consortium (OGC) [1] , W3 Consortium[7], OMG [6] and OAGI [8] are playing a key role in developing standards for corporate-wide geospatial initiatives.
Emerging Technologies
Developing and following industry standards is the key to enable new applications and COTS solutions to interact. A few of these standards are described below.
XML
XML, eXtensible Markup Language, is an enabling standard for current and evolving EAI (Enterprise Application Integration) platforms. XML is well known as a strategic approach for exchanging self-describing information between applications and businesses. What XML does well is describe data content and allows us to define processes, which makes the process of integration easier and more flexible. Some advantages of using XML are:
- Common data exchange protocol allowing reuse in subsequent projects
- As its effective use increases, standard tools such as parsers and transformation engines, and editors start to grow (www.xmlsoftware.com )
- Independence of hardware platform and database format
- Can be used as a data storage (XML-databases) or can be manipulated through major database vendors
- Minimizes the costs in the areas of expandability, maintainability, scalability and reliability [9]
- Ability to easily apply data types to elements and attributes using XMLSchema
Some of the spatial industry’s leading vendors are implementing EAI interfaces to their spatial platforms that leverage the GML standard. The importance is that this evolving standard will enable the exchange and integration of spatial data across multiple and different platforms that support unique spatial data models. Using XML and standard vocabularies result in a productive environment, maintenance and enhancement tasks are shortened and reliability is increased. These features will allow rapid software development or customization: one of the most important factors in today’s business environments.
Web Services
Software on a common platform is being supplanted with remote invocation using well-defined interfaces for interoperability. Web Service is a programmable application logic accessible using standard Internet protocols. Web Services offer capabilities such as translation, projection, generalization, analysis, and other component services. Emerging standards define how these capabilities will become ubiquitous within industry.
Like components, Web Services represent black-box functionality that can be reused without worrying about how the service is implemented. Unlike current component technologies, Web Services are not accessed via object-model specific protocols, such as the Distributed Component Object Model (DCOM), Remote Method Invocation (RMI), or Internet Inter-ORB Protocol (IIOP). Instead, Web Services are accessed via ubiquitous Web protocols and formats, such as Hypertext Transfer Protocol (HTTP) and Extensible Markup Language (XML).
Service-oriented connectivity is more complex. Instead of dealing with extracting and publishing simple information to source or target systems, service-oriented adapters have to abstract services or application behavior, too. They need to expose application functions for abstraction into a composite application as a local function that actually exists on a remote system. Even though the function appears local to the composite application, the application processing occurs in the remote system connected through a service-oriented connectivity subsystem (adapter). There are some key specifications and technologies being defined or available today that address the basic requirements for service-based development [10]:
- A standard way to represent data - XML is the obvious choice for standardization. Most Web Service related specifications use XML for data representation, as well as XML schemas to describe data types.
- A common, extensible, message format - The Simple Object Access Protocol (SOAP) defines a lightweight protocol for information exchange. Part of the SOAP specification defines a set of rules for how to use XML to represent data. Other parts of the SOAP specification define an extensible message format, conventions for representing remote procedure calls (RPCs) using the SOAP message format, and bindings to the HTTP protocol.
- A common, extensible, service description language - Given a Web Service, it would be nice to have a standard way to document what messages the Web Services accepts and generates. The Web Services Description Language (WSDL) is an XML-based contract language.
- A way to discover services located on a particular website – Developers will also need some way to discover Web Services.
- A way to discover service providers – In many cases the developer will not know the URLs where services can be found. Universal Description, Discovery, and Integration (UDDI) specifies a mechanism for Web Service providers to advertise the existence of their Web Services and for Web Service consumers to locate Web Services of interest, using a publish, find, and bind mechanism.
Application servers distributed architecture ensure delivery of information from one application to the next and also provides at the server side the ability to process information from many different resources, such as databases and applications.
Application server is a consistent and reliable programming model mainly because they require that complex applications be divided into transactions. Application servers control transactions from their beginning to their end, from the client to the resource server, and then back again. They ensure ACID (Atomic, Consistency, Isolation and Durability) properties of transactions. It makes application servers a natural choice for distributed applications, which must deal with different type of interfaces and heterogeneous platforms.
Application servers provide a location for the application code to run and, as a result, a location for business processing and application objects to run, as well as a location where methods can be shared among applications. They can be used to enforce business rules and maintain data integrity or to create entire applications by building many transaction services and invoking them from the client. They also provide load balancing, thread pooling, object recycling, and the ability to automatically recover from typical system problems.
Spatial DBMS
In order to have an enterprise GIS solution it is required to have an integrated data management solution. In the mid-1990s spatial data was tightly integrated with associated tabular data. This type of solution was not enough to support the operation requirements of the enterprises. The object-based and object-relational architecture of the late 1990s provides an open and robust solution to support these requirements.
The real need was to apply the benefits of mature database management technology to geographic data. As the evolving growth of users and consequently transactions, the security and continuous availability requirements and scalability needs [4] pushed the Database and GIS industry to construct an integrated solution that is capable of supporting the new business environment.
The way the database industry solved the problem about spatial data was using the software component technology by introducing components that are stored inside the database. This feature adds the required types, functions, and other pieces, making them available at the SQL level. Different applications can take advantage of different components and as requirements and standards evolve, a component can generally evolve along with them more quickly than the database software itself. This solution was possible because of the extensibility of new object-relational concept of database servers
Some of the benefits that database technology introduced into GIS solutions are:
- Flexibility - SQL provides the flexibility to queries.
- Data integrity – tabular and spatial data stored at the same administrative policies.
- Scalability – applications are able to support a few to millions of transactions because of the proven capability of commercial database systems.
- Performance – all available features of commercial database systems like buffering, B-tree indexes, query execution plan plus the spatial access methods implementations: Quad-tree, R-tree and Grid.
- Object-relational features – support for creating new and abstract data types, capability to create any kind of integrity using functions.
- Open standards – SQL and GML access to all functions and operations.
- Support for enterprise applications – the client/server architecture of database systems is already proved to be the most reliable component of any EAI solution.
- Heterogeneous data – tools to support heterogeneous geographic data including projection management and coordinate transformation.
Reference architecture
Figure 1: An Enterprise-based Architecture
If companies want to take advantage of the geospatial information available at all levels of the company, an enterprise-based architecture like the one in Figure 1, is a starting point to make it available to all users and to fulfill the different needs of each department.
The proposed n-tier architecture is the key point. It is becoming a standard for enterprise applications and EAI integration. Industry leaders support this architecture with emerging standards like EJB, J2EE and .NET.
Moving spatial data management into an enterprise database allows applications and users to access enterprise-wide data. The spatial data can be validated at the data layer and spatial oriented business rules can be performed like a traditional tabular data. This capability will decrease the need of programming business rules on the client-side by using the GIS proprietary language or third-generation language supported.
Spatial data integrity, consistency and availability are guaranteed by DBMS. In practice this makes development rapid and reliable, and it makes the business logic available to all applications that access the data. This assures consistency across the enterprise with minimal maintenance.
Some DBMS have (or will have) capability to answer queries in GML format. If this capability is not available on the DBMS layer, GML can be generated at the application layer by applying the necessary business constraints. Another way of having data in a GML format available is to construct a GML data format cache.
This type of solution is particularly useful for applications that do not need to have the most up-to-date data available or environments that have performance and network response time restrictions. Regarding the latter case, a proxy policy can guarantee the data availability and reduce the traffic within the network. GML cache can also be used to leverage spatial data with a low cost, using freeware Internet tools to display information across the enterprise.
Application Servers and Web Servers play another key role in the new enterprise architecture. As robust and reliable solutions, they are essential components of new Web-enabled solutions. They support the division of application on abstract remote services as a local function that actually exists on a remote system. Even though the function appears local to the composite application, the application processing occurs in the remote system connected through a service-oriented connectivity subsystem.
We know that process integration requires application integration and Application Servers are the key technology to link front-end to back-end processes. The glue that links the n-tier architecture is the XML data format. It provides semantic data abstraction on the integration of different layers of the architecture. No data conversion is necessary because it provides a common vocabulary (if defined) for all application layers or within different applications.
Standard vocabularies should be built to an industry standard model to ensure they are open and not dependent on a proprietary solution or a particular vendor. Standard vocabularies provide an interface that remains stable even though the various applications change.
XML is a key technology for data integration and a powerful mechanism to construct a standard vocabulary within application domains. However, it is not alone in playing the role of providing integration and interoperability. As an example, XML does not handle end-to-end transaction, neither does it guarantee delivery nor does it handle audits.
The Object Adapter layer in the Enterprise-based Architecture diagram is necessary to decouple the application from the GIS. The Object Adapters will wrap GIS from the business logic bringing GIS independency. This layer is fundamental because it enables the integration of the spatial data, business logic, and GIS . Providing integration is required mainly for the following reasons:
- To provide a reusable set of services (adapters) saving time on building new applications.
- To establish a common interface between application and GIS tools.
- To provide reliability to applications, as adapters are trustable components.
Using this type of architecture will force the GIS industry to be in line with the open standards. GIS will be a tool for query, display, analysis, and manipulation of geospatial data from corporate databases in conjunction with Internet public sources of data. The graphics display (symbology) will need to be constructed for each GIS because it is technology dependent.
This architecture will allow flexibility to replace or update a GIS if it does not meet the user’s need and it will enable multiple GIS’s to work together seamlessly. Another advantage is that geographic data will be more available due to its ease of access. Adopting a component-based architecture will be an easier way to deploy applications.
Summary
We have been in an era where “What is your graphics platform?” was the right question to ask but now we are in an era where the right questions to ask are “Can I exchange information with your company?” or “Can we share the survey costs?”. The correct answer should be “Yes, because I’m using standards”. More and more GIS will not be the unique solution to display and analyze geographic data. For different needs of different users, companies will have different GIS solutions but the data, that is the most valuable asset of the company, will be seamless and available enterprise-wide.
References
- OGC - Open GIS Consortium, http://www.opengis.org
- Strickland, Tom M., Web Based Services (WBS), http://eaijournal.com
- Weiss, Gerson M., Dias, E., 2003 (to appear) Definition of a XML format for Telecommunications Outside Plant Network. In: CONFERENCIA INTERNACIONAL DE LA SOCIEDAD PERUANA DE LA COMPUTACION / SPC ( http://www.spc.org.pe) y RMCP ( http://www.rmcp.org) (in Portuguese language)
- PREVIDELLI, C. A. and MAGALHÃES, G. C. Moving Geospatial Applications Towards a Mission Critical Scenario. In: 25th Annual Congress of the GeoSpatial Information Technology Association, Tampa/USA, March 2002.
- Java, http://java.sun.com
- OMG, Object Management Group, http://www.omg.org
- World Wide Web Consortium, http://www.w3.org
- OAGI, Open Application Group, Inc., http://www.openapplications.org
- Hing, Mark Reducing system integration costs with XML. In: 25th Annual Congress of the GeoSpatial Information Technology Association, Tampa/USA, March 2002.
- Waren, David , A Virtual GIS , In: 25th Annual Congress of the GeoSpatial Information Technology Association, Tampa/USA, March 2002.