| GISdevelopment.net ---> GITA 2000 ---> It's A Brave New World |
Building an Enterprise-Wide Spatial Information Intergration Platform
David Barron
O.L.S.
Manager Technical Data Centre
Facilities Engineering
Greater Toronto Airports Authority
Lester B. Pearson International Airport
3111 Convair Drive
Toronto, Ontario CANADA L5P 1B2
Telephone: (905) 676-7747
Fax: (905) 676-4640
Email: d_barron@gtaa.com
George Irwin
P.Eng, MCP
President
Irwin Consulting Services Inc.
30 Ridley Crescent
Markham, Ontario CANADA L3S 3P5
Telephone: (905) 471-9294
Fax: (905) 472-4003
Email: girwin@home.com
David Barron
O.L.S.
Manager Technical Data Centre
Facilities Engineering
Greater Toronto Airports Authority
Lester B. Pearson International Airport
3111 Convair Drive
Toronto, Ontario CANADA L5P 1B2
Telephone: (905) 676-7747
Fax: (905) 676-4640
Email: d_barron@gtaa.com
George Irwin
P.Eng, MCP
President
Irwin Consulting Services Inc.
30 Ridley Crescent
Markham, Ontario CANADA L3S 3P5
Telephone: (905) 471-9294
Fax: (905) 472-4003
Email: girwin@home.com
The Holy Grail
The cup that Christ drank from at the last supper is a find that has long eluded archaeologists and fortune seekers alike. The search for the Holy Grail was arguably best dramatized in the Steven Spielberg/George Lucas epic, “Indiana Jones and the Last Crusade”. In the movie, both good and evil forces battled for possession of this cup, which, in the end eluded all and found its resting place forever out of their collective reach.
Visionaries in the AM/FM/GIS (henceforward referred to as GIS) world frequently found themselves led in a similarly difficult search for the Holy Grail of spatial information management, a world in which all information – spatial and attribute alike – resides in a ubiquitous, universally accessible and eminently usable transaction-based data warehouse. Just as the goal seems about to come into our reach, a new and better GIS technology emerges that, instead of bringing us closer to our goal, introduces a new chasm that must be crossed by the technologically elite of the spatial information management (GIS) community.
The Reality
In “The Last Crusade”, just as our hero’s hand encircles the long-awaited prize, forces beyond his control wrest the cup from him and leave him not only empty-handed, but also with the immediate problem of how to escape from imminent disaster.
Similarly, we in the GIS profession find our sweet daydreams of universal, transaction-based spatial and attribute data management interrupted with the cold, hard reality of spatial and attribute data that spans the spectrum of quality, accessibility and data formats. Most spatial data today is still file-based, is not cross-referenced (tagged) to attribute data and is not object-oriented, to say nothing of it being topologically “correct” (whatever that means to you). Moreover spatial data resides in a variety of formats, almost all of which are proprietary. Worse yet, there is no standard mechanism that exists for querying spatial data from these spatial data sources. There are no SQLs, or ODBCs in our world. Ours is a world where chaos reigns.
The People
In Indiana Jones’ world, there were basically two kinds of people: good and evil. In our world, there are likewise two kinds of people: GIS people (enlightened) and everyone else (unlearned). GIS people know about things like coordinate systems, projections, topology and georeferencing (and lots of other big words). The unlearned concern themselves with documents, spreadsheets, databases, reports and the like.
GIS people perform spatial information analysis, be it digital terrain modeling, network analysis, polygon overlay processing, raster image analysis or merely a minor geodetic coordinate system transformation. The unlearned content themselves primarily with information navigation and retrieval, for example, knowing where a transformer is, how it is mounted and what the maintenance crew will need to repair it.
The only problem with the GIS professional’s view of the world is that we are clearly in the minority. If a company does have a GIS department, they are, at most, 20% of the user community, and more likely 5-10% of the user community. Clearly, if this technology is ever to achieve the impact that it ought to, GIS users are going to have to be found in every walk of life, not just in the backrooms buried in electronic maps. Spatial information management technology should be as familiar to people as their desktop office suite application.
GIS vs. MIS
To the MIS (IT) department there are also two kinds of people: normal users (good) and GIS users (bad). The MIS professionals that GIS professionals so mock for their lack of vision generally know the technologically elite spatial information management professionals that we so admire as “propeller-heads”. As a rule of thumb we GIS users (optimistically 20% of the user community) probably absorb 80% of the MIS group’s time, if they will supportus at all. We soak up network capacity with reams and reams of spatial data, much of it redundant, and bog down their networks with spatial queries that constrict network bandwidth from a superhighway down to a cart path.
Our solution to this problem, of course, is to hypercharge our workstations, and develop our own fibre optic networks to beam information back and forth to one another at the speed of light. We cut ourselves off from the inferior MIS networks that simply cannot keep up with us. Is you’re your spatial query taking too long? Just double up your processors. Of course, the end result of this is that the GIS community ends up more and more isolated from the rest of the IT world, which is precisely the opposite direction that we ought to be going. GIS technology needs to be brought into the mainstream, not cut off from it.
Information vs. Applications: The 80/20 Rule
We’ve all seen them and, to a large extent we covet them. Every year at GITA, we see some of them. Most of us also personally have had a modest role in authoring them: the “killer apps”. Killer apps are the things that make all the wading through endless reams of sporadic software vendor documentation, all the undocumented bugs, all the late nights drinking Jolt Cola, all the hair pulling, nail biting and stomach ulcers worthwhile.
Applications, however, do not exist in isolation. Information is what makes it possible to build neat applications. Just as the verb in a sentence needs a noun, similarly applications need a critical mass of information.
Broadly speaking, most of what we do with computers can be classified in one of two ways: basic information manipulation and advanced information analysis (Information vs. Applications). In the GIS field, perhaps more than any other, applications rule!
Of course, there is a great need for advanced spatial information analysis applications in the GIS world, and to a large extent this has been what has set the industry apart. But the reality of the situation is that most users don’t really need this level of functionality. What most users really require is easy access to pertinent information that will help them accomplish the immediate task at hand.
The problem is that most GIS systems today tend to cater to the 20% (or less) of the user community made up by users who require advanced information analysis instead of the 80% (or more) of the user community that require basic information manipulation.
Building an Information Abstraction Layer
Because, all information, especially spatial information, does not presently fit neatly into rows and columns in a some kind of a “georelational” database, there is a need for some mechanism that will allow information, whether it is textual or graphic, file-based or transaction-based to be easily managed by anyone who is going to (or would like to) encounter that information. This mechanism would handle all of the user interface, business logic and data manipulation issues associated with managing this information in an object-oriented fashion 1 .
We call this mechanism an information abstraction layer (IAL), which is analogous to a hardware abstraction layer. A hardware abstraction layer isolates the operating system from the underlying hardware platform on which the O/S will be running. Similarly, an information abstraction layer isolates the user from the underlying details of how the particular information associated with his objects is managed. The IAL will encompass all significant data manipulation operations required to manage both spatial and attribute data including: information navigation, information retrieval and information input. In addition, the IAL will deal with both file-based and transaction-based data.
The notion of an IAL has been alluded to in the SQL-2 specification, which introduces the notion of making a database self-describing through the use of relational database tables (or views). However, this specification only deals with relational database information, which is generally the least of the GIS user’s problems. The IAL provides a way to combine various scattered data elements into the logical objects that a “normal” user can relate to, and, moreover, the objects that comprise the physical assets that businesses and other organizations must manage well to be successful.
The Role of Standards
How many Microsoft programmers does it take to change a light bulb? None, they just change the standard to darkness. Where would Microsoft be today without standards? It’s not a stretch to say that standards have gone a long way to ensure that their unyielding establishment of and adherence to standards such as ODBC, COM and even Visual Basic have strengthened their leadership position in the industry. These are all fundamental elements of today’s computing landscape. How about Oracle corporation? Their faithful support of the SQL standard has made them the leading database vendor. Even something as simple as the electrical industry’s standardizing on 120VAC and 60Hz has made life so much simpler for appliance manufacturers. What can the GIS industry learn from these examples?
Standards are a vital part of implementing any successful IT project, and are perhaps even more important in implementing a successful GIS project (which, if successful, has probably been implemented as an IT project). Although the GIS industry is woefully lagging in its pursuit of its own set of standards, it is absolutely vital that standards are given due consideration when implementing a GIS project. Many times in the GIS industry, we see a fascinating demonstration in which two formerly incompatible systems are “transparently” and “seamlessly” interconnected into a unified whole (i.e. bridge building). However, when asked how the integration was accomplished, we frequently meet with a response that is tantamount to: “Such knowledge is reserved for those who know the ways of the Force.” or “I could tell you, but I’d have to kill you afterwards.”
What was demonstrated was impressive, but how it was accomplished is equally, if not more, important. Standards are the “how” part of the equation, particularly as it concerns information abstraction. Building a successful system that embraces spatial information technology requires judicious adherence to standards as much, if not more, than any other project. Adherence to standards builds resiliency into our systems.
Building an Application Abstraction Layer
As discussed earlier, both information and applications have their respective roles in any successful information systems implementation. Just as building an information abstraction layer can greatly assist in managing basic information navigation, retrieval and input, building an application abstraction layer (AAL) can assist in abstracting the information analysis capabilities of different applications.
The AAL does for applications what the IAL does for information. It provides a generic way to work with the functionality that different classes of applications provide, but in a vendor-neutral fashion. That is to say, you can choose the individual applications you need to fulfill a particular set of functionality without having to standardize on a single vendor to provide all of the functionality required.
The Role of Software Components
Software components are to applications what standards are to information. Traditionally, applications have only been accessible as monolithic, impenetrable giants. More and more, applications are being broken down into small, accessible and reusable components. Software components encapsulate the functionality of applications into reusable “chunks” that can be assembled into a fully functional solution. It has been said that software components will do as much to further application development as hardware components have done to advance the state-of-the-art in computer manufacturing.
As applications become less monolithic and more “componentized”, there will be more opportunity to choose best of breed software components from a variety of vendors, instead of having to standardize on a single software vendor, who, despite all his claims to the contrary, cannot possibly be good at everything.
Leveraging Internet/Intranet Technology
It’s all well and good to have universal standards for storing and accessing information and an information abstraction layer (IAL) to connect people with facts. Likewise, it’s fantastic, if perhaps a bit optimistic to have applications serving up their functionality in nicely architected software components, all of them accessible through our ubiquitous application abstraction layer (AAL). However, these elements alone cannot achieve the result that we desire. We need connectivity!
We live in a world where information and applications are distributed across an extraordinarily wide area. Fortunately, we also live in an age where the ultimate (so far) wide area network is already in place and is, coincidentally, revolutionizing the way we live and work. Internet (aka intranet/extranet) technology is the plumbing that connects the appliances (applications) with the water (information) that is flowing freely around the world today.
Bringing IT all Together: The Plateform
How do all of these pieces of the puzzle fit together? Well, it works something like this: the IAL is connected to the information, the AAL is connected to the applications, and the wiring is TCP/IP. When it’s all put together, the result is a platform that is both scalable and flexible and empowers the organizations to choose both information technologies and applications based on their own merits. Moreover, it doesn’t force organizations to choose a homogeneous solution and try to make everyone fit their square pegs into the round holes that we’ve carved out for them in our “solution”. The platform is built around what already exists, not around a vision that a particular GIS vendor has of the world. Users who only want information management functionality get just that and advance analytical users are empowered to accomplish more and increase the overall level of information understanding in an organization.
Call to Action
Every year, GIS companies come out with new, exciting analytical capabilities that are supposed to bring us closer to the Holy Grail of GIS. Instead these innovations often only serve to separate the GIS industry further from other mainstream information technologies, and, of course, make the particular GIS vendor who came up with the idea better than every other GIS vendor. This is not to say that such technological innovation is not commendable, but rather that the effort expended on this innovation is disproportionate to the effort that ought to be expended in bringing GIS into the mainstream, in effect bridging the gap between GIS and other enterprise management information systems.
To put it a bit more bluntly, if GIS vendors spend half the time collaborating to make GIS technology fit better into the mainstream IT world that they currently spend beating each other up for a bigger slice of the existing pie, they would have a much larger pie to carve up. Is that to say that GIS vendors should stop doing the things that make their product suites better than the other guy’s? Of course not; healthy competition is the key to any successful industry being – well – healthy. But the need for the GIS industry to bridge the gap between spatial information management technology and other mainstream information technology has never been greater. Did the size of the database market go down because of standards like SQL and ODBC? No, on the contrary, it grew to where databases are fundamental pillars of the IT operations of virtually every business today. It would be in the best interests of the GIS community to do precisely the same thing.
The cup that Christ drank from at the last supper is a find that has long eluded archaeologists and fortune seekers alike. The search for the Holy Grail was arguably best dramatized in the Steven Spielberg/George Lucas epic, “Indiana Jones and the Last Crusade”. In the movie, both good and evil forces battled for possession of this cup, which, in the end eluded all and found its resting place forever out of their collective reach.
Visionaries in the AM/FM/GIS (henceforward referred to as GIS) world frequently found themselves led in a similarly difficult search for the Holy Grail of spatial information management, a world in which all information – spatial and attribute alike – resides in a ubiquitous, universally accessible and eminently usable transaction-based data warehouse. Just as the goal seems about to come into our reach, a new and better GIS technology emerges that, instead of bringing us closer to our goal, introduces a new chasm that must be crossed by the technologically elite of the spatial information management (GIS) community.
The Reality
In “The Last Crusade”, just as our hero’s hand encircles the long-awaited prize, forces beyond his control wrest the cup from him and leave him not only empty-handed, but also with the immediate problem of how to escape from imminent disaster.
Similarly, we in the GIS profession find our sweet daydreams of universal, transaction-based spatial and attribute data management interrupted with the cold, hard reality of spatial and attribute data that spans the spectrum of quality, accessibility and data formats. Most spatial data today is still file-based, is not cross-referenced (tagged) to attribute data and is not object-oriented, to say nothing of it being topologically “correct” (whatever that means to you). Moreover spatial data resides in a variety of formats, almost all of which are proprietary. Worse yet, there is no standard mechanism that exists for querying spatial data from these spatial data sources. There are no SQLs, or ODBCs in our world. Ours is a world where chaos reigns.
The People
In Indiana Jones’ world, there were basically two kinds of people: good and evil. In our world, there are likewise two kinds of people: GIS people (enlightened) and everyone else (unlearned). GIS people know about things like coordinate systems, projections, topology and georeferencing (and lots of other big words). The unlearned concern themselves with documents, spreadsheets, databases, reports and the like.
GIS people perform spatial information analysis, be it digital terrain modeling, network analysis, polygon overlay processing, raster image analysis or merely a minor geodetic coordinate system transformation. The unlearned content themselves primarily with information navigation and retrieval, for example, knowing where a transformer is, how it is mounted and what the maintenance crew will need to repair it.
The only problem with the GIS professional’s view of the world is that we are clearly in the minority. If a company does have a GIS department, they are, at most, 20% of the user community, and more likely 5-10% of the user community. Clearly, if this technology is ever to achieve the impact that it ought to, GIS users are going to have to be found in every walk of life, not just in the backrooms buried in electronic maps. Spatial information management technology should be as familiar to people as their desktop office suite application.
GIS vs. MIS
To the MIS (IT) department there are also two kinds of people: normal users (good) and GIS users (bad). The MIS professionals that GIS professionals so mock for their lack of vision generally know the technologically elite spatial information management professionals that we so admire as “propeller-heads”. As a rule of thumb we GIS users (optimistically 20% of the user community) probably absorb 80% of the MIS group’s time, if they will supportus at all. We soak up network capacity with reams and reams of spatial data, much of it redundant, and bog down their networks with spatial queries that constrict network bandwidth from a superhighway down to a cart path.
Our solution to this problem, of course, is to hypercharge our workstations, and develop our own fibre optic networks to beam information back and forth to one another at the speed of light. We cut ourselves off from the inferior MIS networks that simply cannot keep up with us. Is you’re your spatial query taking too long? Just double up your processors. Of course, the end result of this is that the GIS community ends up more and more isolated from the rest of the IT world, which is precisely the opposite direction that we ought to be going. GIS technology needs to be brought into the mainstream, not cut off from it.
Information vs. Applications: The 80/20 Rule
We’ve all seen them and, to a large extent we covet them. Every year at GITA, we see some of them. Most of us also personally have had a modest role in authoring them: the “killer apps”. Killer apps are the things that make all the wading through endless reams of sporadic software vendor documentation, all the undocumented bugs, all the late nights drinking Jolt Cola, all the hair pulling, nail biting and stomach ulcers worthwhile.
Applications, however, do not exist in isolation. Information is what makes it possible to build neat applications. Just as the verb in a sentence needs a noun, similarly applications need a critical mass of information.
Broadly speaking, most of what we do with computers can be classified in one of two ways: basic information manipulation and advanced information analysis (Information vs. Applications). In the GIS field, perhaps more than any other, applications rule!
Of course, there is a great need for advanced spatial information analysis applications in the GIS world, and to a large extent this has been what has set the industry apart. But the reality of the situation is that most users don’t really need this level of functionality. What most users really require is easy access to pertinent information that will help them accomplish the immediate task at hand.
The problem is that most GIS systems today tend to cater to the 20% (or less) of the user community made up by users who require advanced information analysis instead of the 80% (or more) of the user community that require basic information manipulation.
Building an Information Abstraction Layer
Because, all information, especially spatial information, does not presently fit neatly into rows and columns in a some kind of a “georelational” database, there is a need for some mechanism that will allow information, whether it is textual or graphic, file-based or transaction-based to be easily managed by anyone who is going to (or would like to) encounter that information. This mechanism would handle all of the user interface, business logic and data manipulation issues associated with managing this information in an object-oriented fashion 1 .
We call this mechanism an information abstraction layer (IAL), which is analogous to a hardware abstraction layer. A hardware abstraction layer isolates the operating system from the underlying hardware platform on which the O/S will be running. Similarly, an information abstraction layer isolates the user from the underlying details of how the particular information associated with his objects is managed. The IAL will encompass all significant data manipulation operations required to manage both spatial and attribute data including: information navigation, information retrieval and information input. In addition, the IAL will deal with both file-based and transaction-based data.
The notion of an IAL has been alluded to in the SQL-2 specification, which introduces the notion of making a database self-describing through the use of relational database tables (or views). However, this specification only deals with relational database information, which is generally the least of the GIS user’s problems. The IAL provides a way to combine various scattered data elements into the logical objects that a “normal” user can relate to, and, moreover, the objects that comprise the physical assets that businesses and other organizations must manage well to be successful.
The Role of Standards
How many Microsoft programmers does it take to change a light bulb? None, they just change the standard to darkness. Where would Microsoft be today without standards? It’s not a stretch to say that standards have gone a long way to ensure that their unyielding establishment of and adherence to standards such as ODBC, COM and even Visual Basic have strengthened their leadership position in the industry. These are all fundamental elements of today’s computing landscape. How about Oracle corporation? Their faithful support of the SQL standard has made them the leading database vendor. Even something as simple as the electrical industry’s standardizing on 120VAC and 60Hz has made life so much simpler for appliance manufacturers. What can the GIS industry learn from these examples?
Standards are a vital part of implementing any successful IT project, and are perhaps even more important in implementing a successful GIS project (which, if successful, has probably been implemented as an IT project). Although the GIS industry is woefully lagging in its pursuit of its own set of standards, it is absolutely vital that standards are given due consideration when implementing a GIS project. Many times in the GIS industry, we see a fascinating demonstration in which two formerly incompatible systems are “transparently” and “seamlessly” interconnected into a unified whole (i.e. bridge building). However, when asked how the integration was accomplished, we frequently meet with a response that is tantamount to: “Such knowledge is reserved for those who know the ways of the Force.” or “I could tell you, but I’d have to kill you afterwards.”
What was demonstrated was impressive, but how it was accomplished is equally, if not more, important. Standards are the “how” part of the equation, particularly as it concerns information abstraction. Building a successful system that embraces spatial information technology requires judicious adherence to standards as much, if not more, than any other project. Adherence to standards builds resiliency into our systems.
Building an Application Abstraction Layer
As discussed earlier, both information and applications have their respective roles in any successful information systems implementation. Just as building an information abstraction layer can greatly assist in managing basic information navigation, retrieval and input, building an application abstraction layer (AAL) can assist in abstracting the information analysis capabilities of different applications.
The AAL does for applications what the IAL does for information. It provides a generic way to work with the functionality that different classes of applications provide, but in a vendor-neutral fashion. That is to say, you can choose the individual applications you need to fulfill a particular set of functionality without having to standardize on a single vendor to provide all of the functionality required.
The Role of Software Components
Software components are to applications what standards are to information. Traditionally, applications have only been accessible as monolithic, impenetrable giants. More and more, applications are being broken down into small, accessible and reusable components. Software components encapsulate the functionality of applications into reusable “chunks” that can be assembled into a fully functional solution. It has been said that software components will do as much to further application development as hardware components have done to advance the state-of-the-art in computer manufacturing.
As applications become less monolithic and more “componentized”, there will be more opportunity to choose best of breed software components from a variety of vendors, instead of having to standardize on a single software vendor, who, despite all his claims to the contrary, cannot possibly be good at everything.
Leveraging Internet/Intranet Technology
It’s all well and good to have universal standards for storing and accessing information and an information abstraction layer (IAL) to connect people with facts. Likewise, it’s fantastic, if perhaps a bit optimistic to have applications serving up their functionality in nicely architected software components, all of them accessible through our ubiquitous application abstraction layer (AAL). However, these elements alone cannot achieve the result that we desire. We need connectivity!
We live in a world where information and applications are distributed across an extraordinarily wide area. Fortunately, we also live in an age where the ultimate (so far) wide area network is already in place and is, coincidentally, revolutionizing the way we live and work. Internet (aka intranet/extranet) technology is the plumbing that connects the appliances (applications) with the water (information) that is flowing freely around the world today.
Bringing IT all Together: The Plateform
How do all of these pieces of the puzzle fit together? Well, it works something like this: the IAL is connected to the information, the AAL is connected to the applications, and the wiring is TCP/IP. When it’s all put together, the result is a platform that is both scalable and flexible and empowers the organizations to choose both information technologies and applications based on their own merits. Moreover, it doesn’t force organizations to choose a homogeneous solution and try to make everyone fit their square pegs into the round holes that we’ve carved out for them in our “solution”. The platform is built around what already exists, not around a vision that a particular GIS vendor has of the world. Users who only want information management functionality get just that and advance analytical users are empowered to accomplish more and increase the overall level of information understanding in an organization.
Call to Action
Every year, GIS companies come out with new, exciting analytical capabilities that are supposed to bring us closer to the Holy Grail of GIS. Instead these innovations often only serve to separate the GIS industry further from other mainstream information technologies, and, of course, make the particular GIS vendor who came up with the idea better than every other GIS vendor. This is not to say that such technological innovation is not commendable, but rather that the effort expended on this innovation is disproportionate to the effort that ought to be expended in bringing GIS into the mainstream, in effect bridging the gap between GIS and other enterprise management information systems.
To put it a bit more bluntly, if GIS vendors spend half the time collaborating to make GIS technology fit better into the mainstream IT world that they currently spend beating each other up for a bigger slice of the existing pie, they would have a much larger pie to carve up. Is that to say that GIS vendors should stop doing the things that make their product suites better than the other guy’s? Of course not; healthy competition is the key to any successful industry being – well – healthy. But the need for the GIS industry to bridge the gap between spatial information management technology and other mainstream information technology has never been greater. Did the size of the database market go down because of standards like SQL and ODBC? No, on the contrary, it grew to where databases are fundamental pillars of the IT operations of virtually every business today. It would be in the best interests of the GIS community to do precisely the same thing.