| GISdevelopment.net ---> GITA 2000 ---> Exploiting Field and Mobile Technologies |
Outstanding in the Field
Mike Carlisle
Cook-Hurlbert Inc.
5222 Thundercreek Rd
Austin, Texas
Mike Carlisle
Cook-Hurlbert Inc.
5222 Thundercreek Rd
Austin, Texas
Introduction
Field users of GIS are not a homogenous group. The functionality that they require is as varied as the tasks that they are asked to perform, so it would be wrong to suggest that there is one enterprise-wide field “solution”. The users can be viewed as being on a sliding scale, from those who simply need to view data through to those who need the full functionality of their master GIS but in the field, with a wide range of requirements in-between. It may seem to be the “top end” users that most require an integrated solution between field and office, but in reality the efficiency of all users is reduced if both systems use different underlying data formats. The question must be, why are the two solutions so different?
Technologies
Let us first look at the various software and hardware technologies that are used by a field GIS and the limitations that these technologies impose.
Software
In terms of the integration of field and office use, the most important aspect of the GIS software is the relationship between the master and field systems. The principal relationship models are:
Thin Client. In this model the field software is basically just a user interface. All (or at least most) processing is done by the master system, which also holds all of the data. This requires that the two systems have a continuous fast communication link between them as each operation on the field unit is actually sending a request for information to the master system.
Thick Client. This is similar to Thin Client, but more work is done by the field system. In a typical Thick Client application, the field system would request data from the master and then perform operations upon it locally. A more intelligent (thicker) client may cache data to improve performance. Again there is a requirement for an open communication link as data is being passed between the two systems as it is used.
Independent Systems. In this model, the field system holds all of its data locally and all of the processing is carried out on the field machine. To be useful, this model requires periodic communication between field and master systems to synchronize data. Unlike the other two models however, this can be done in a batch operation, for example by connecting to the office LAN at the end of the day. Because the data is held locally on the field machine, users who do not know the geographical area that they will be working in need large amounts of storage capacity. This model is often referred to as a disconnected field GIS.
Clearly the choice of model is dependant upon the use to which the field GIS will be put. Client-Server applications rely upon wireless communications, but for particularly data intensive operations the available wireless bandwidth may not be sufficient.
Hardware
Computers
There are three main types of field computing device available: Ruggedized laptops, pen-based computers, and PDAs.
PDAs are becoming increasingly popular, particularly for data capture, but they are restricted to fairly lightweight tasks because of the limitations on available memory and processing power. These limitations are reduced with every model released, but a seamless office/PDA GIS solution is hampered by the fact that PDAs use different Operating Systems from PCs and servers. Windows CE devices will support Java applications, which at least opens the door to interoperability, but the PDA with the greatest market share is the PalmPilot, which does not support Java and uses its own proprietary Operating System.
Pen-based machines are a very popular field choice as they offer a natural method of user interaction and are a lot easier to use while standing up than a conventional laptop. Increasingly the performance of pen-computers is catching up with laptops, though the touch sensitivity of the screen imposes limits on the screen size. Most pen-based computers also suffer from comparatively short battery life, and this is compounded if the unit is also running a wireless modem. Unlike PDAs, pen machines and ruggedized laptops typically run Windows 95/98 or NT.
Communication Technology
Client-Server models require continual communication between the master and field systems. Since the very nature of a field GIS normally precludes connecting to a landline, this communication link must be wireless. Digital wireless communication does not have universal coverage as yet, and its bandwidth (or throughput) is still approximately fifty percent lower than that of a landline.
Radio networks have a comparable bandwidth to digital wireless, but their reliability of service can be worse.
If the data being transmitted is sensitive it may be necessary to encrypt it to maintain security, which adds to the processing time on both sides, though this is normally negligible compared to the transmission time. On a related issue, Client-Server solutions require thorough authentication of users to prevent unauthorized access to proprietary information.
Integrating Field and Office
There are two areas in which field and office GIS systems do not form a seamless system: User Interface and Data Storage. The format that data is stored in imposes performance limitations if conversion between the two systems is necessary, while differences in User Interface, although they appear to be simply cosmetic, impact a user’s productivity because they require the user to become familiar with two applications.
User Interface
The User Interface is the “look and feel” of the application, and it is this aspect which is potentially the most annoying for the user. It is always more convenient if the same operation is carried out in the same way in each application. All Microsoft Office applications share the same standard menu items, toolbars and keyboard shortcuts (e.g. Cut, Copy, Paste) for the simple reason that this makes it easier for the user to move from one program to another.
A User Interface which is uniform between field and office should be the goal, but there are several reasons why differences must exist.
Differences in Functionality. It may not be appropriate for all office functionality to be available in the field. Furthermore it is likely that the field system will need functionality that the office GIS does not have so that it can interact with peripherals such as GPS receivers and laser range-finders.
Field Unit Limitations. The application may need to restrict its User Interface on the field unit as pen computers and PDAs have small screens and so all unnecessary elements must be removed as screen “real estate” is at a premium.
Technology Limitations. Client-Server applications that use web browsers and Internet communication protocols are restricted in the User Interface functionality that they can provide. The thicker the client, the more advanced the User Interface can be. Strictly speaking this does not prevent a uniform User Interface, as the same client can be used on office machines, but in reality it would be unusual to use a browser interface to the GIS from a desktop.
Ordinarily it should not be important which language the GIS was written in (though some will be more customizable than others), but a field solution implemented in Java will be platform independent, allowing a consistent User Interface on CE, pen and desktop machines.
Data Storage
On a Client-Server system, data storage is not really an issue. The real time link with the master GIS means that changes are made to the data as if from a workstation. More intelligent clients may cache data, but they will post CRUD (Create, Update, and Delete) operations as they occur. The same is not true for disconnected field solutions, where the issue of data management is the most significant problem to be overcome.
When a user takes data into the field on a disconnected GIS, the decision must be taken as to whether to lock the map area that they are extracting. Locking means that no other user can make changes to the features in the extracted map area until it is checked back in to the master. If the extracted area is locked, then the user must know the area that they will be working in, and only one field unit can be used in a particular area at any one time. By this method most data management issues can be avoided. Of the three CRUD operations, updates and deletes will not cause problems, but a method must be found to deal with creates.
If extracted areas are not locked the system must be capable of dealing with all of the CRUD operations. The issue is that the master GIS uniquely identifies each element, both spatial and non-spatial. When a new feature is created on the Field GIS, it too must be uniquely identified.
Unfortunately the field system only contains a subset of the total elements in the master, so how do we know that the field generated Id has not already been assigned to an existing feature within the master? The possible solutions to this problem are:
Update and delete operations cause problems when two users carry out operations on the same feature while it is checked out. In its most extreme case, what is the master GIS to do if the field GIS extracts an area, makes changes to a feature, with an Id of say 50, and checks that area back in, but while the data was checked out another user deletes feature 50? Or if two users make different changes to the same attribute of the same feature? Conflict resolution such as this is a significant problem and normally requires functionality to catch the conflicts when the long transaction is completed, and a set of rules to determine how these conflicts are resolved.
These data management issues stem from the long transactional nature of the operations rather than from any characteristic of the field GIS, but other limitations are imposed by the resources of the field unit.
The level of integration of most field and office GIS systems at present is far from satisfactory. There are numerous reasons for this but the principal one must be that in almost every case the office GIS is implemented first and the field component is added at a later date. At that stage the choice is to start from scratch (and re-inventing the wheel is never a popular choice) or to use an existing field solution. Whichever option is chosen, it is unlikely that the field system will properly integrate with the master, as by this stage the task is to adapt the field solution to work with the master.
The way to solve this problem is to regard the office and field systems as different parts of the same solution. Instead of struggling to adapt one system to work with the other we should have the rather more achievable objective of designing a solution which works for the whole enterprise.
This of course is cold comfort to those who already have an established office GIS, but one does not need to look too far into the future to see comprehensive digital wireless coverage and high bandwidth. Those two advances will be enough to enable the replacement of disconnected field systems with Thick Client solutions for all but the most data intensive of tasks, removing many of the associated data management problems.
Field users of GIS are not a homogenous group. The functionality that they require is as varied as the tasks that they are asked to perform, so it would be wrong to suggest that there is one enterprise-wide field “solution”. The users can be viewed as being on a sliding scale, from those who simply need to view data through to those who need the full functionality of their master GIS but in the field, with a wide range of requirements in-between. It may seem to be the “top end” users that most require an integrated solution between field and office, but in reality the efficiency of all users is reduced if both systems use different underlying data formats. The question must be, why are the two solutions so different?
Technologies
Let us first look at the various software and hardware technologies that are used by a field GIS and the limitations that these technologies impose.
Software
In terms of the integration of field and office use, the most important aspect of the GIS software is the relationship between the master and field systems. The principal relationship models are:
Thin Client. In this model the field software is basically just a user interface. All (or at least most) processing is done by the master system, which also holds all of the data. This requires that the two systems have a continuous fast communication link between them as each operation on the field unit is actually sending a request for information to the master system.
Thick Client. This is similar to Thin Client, but more work is done by the field system. In a typical Thick Client application, the field system would request data from the master and then perform operations upon it locally. A more intelligent (thicker) client may cache data to improve performance. Again there is a requirement for an open communication link as data is being passed between the two systems as it is used.
Independent Systems. In this model, the field system holds all of its data locally and all of the processing is carried out on the field machine. To be useful, this model requires periodic communication between field and master systems to synchronize data. Unlike the other two models however, this can be done in a batch operation, for example by connecting to the office LAN at the end of the day. Because the data is held locally on the field machine, users who do not know the geographical area that they will be working in need large amounts of storage capacity. This model is often referred to as a disconnected field GIS.
Clearly the choice of model is dependant upon the use to which the field GIS will be put. Client-Server applications rely upon wireless communications, but for particularly data intensive operations the available wireless bandwidth may not be sufficient.
Hardware
Computers
There are three main types of field computing device available: Ruggedized laptops, pen-based computers, and PDAs.
PDAs are becoming increasingly popular, particularly for data capture, but they are restricted to fairly lightweight tasks because of the limitations on available memory and processing power. These limitations are reduced with every model released, but a seamless office/PDA GIS solution is hampered by the fact that PDAs use different Operating Systems from PCs and servers. Windows CE devices will support Java applications, which at least opens the door to interoperability, but the PDA with the greatest market share is the PalmPilot, which does not support Java and uses its own proprietary Operating System.
Pen-based machines are a very popular field choice as they offer a natural method of user interaction and are a lot easier to use while standing up than a conventional laptop. Increasingly the performance of pen-computers is catching up with laptops, though the touch sensitivity of the screen imposes limits on the screen size. Most pen-based computers also suffer from comparatively short battery life, and this is compounded if the unit is also running a wireless modem. Unlike PDAs, pen machines and ruggedized laptops typically run Windows 95/98 or NT.
Communication Technology
Client-Server models require continual communication between the master and field systems. Since the very nature of a field GIS normally precludes connecting to a landline, this communication link must be wireless. Digital wireless communication does not have universal coverage as yet, and its bandwidth (or throughput) is still approximately fifty percent lower than that of a landline.
Radio networks have a comparable bandwidth to digital wireless, but their reliability of service can be worse.
If the data being transmitted is sensitive it may be necessary to encrypt it to maintain security, which adds to the processing time on both sides, though this is normally negligible compared to the transmission time. On a related issue, Client-Server solutions require thorough authentication of users to prevent unauthorized access to proprietary information.
Integrating Field and Office
There are two areas in which field and office GIS systems do not form a seamless system: User Interface and Data Storage. The format that data is stored in imposes performance limitations if conversion between the two systems is necessary, while differences in User Interface, although they appear to be simply cosmetic, impact a user’s productivity because they require the user to become familiar with two applications.
User Interface
The User Interface is the “look and feel” of the application, and it is this aspect which is potentially the most annoying for the user. It is always more convenient if the same operation is carried out in the same way in each application. All Microsoft Office applications share the same standard menu items, toolbars and keyboard shortcuts (e.g. Cut, Copy, Paste) for the simple reason that this makes it easier for the user to move from one program to another.
A User Interface which is uniform between field and office should be the goal, but there are several reasons why differences must exist.
Differences in Functionality. It may not be appropriate for all office functionality to be available in the field. Furthermore it is likely that the field system will need functionality that the office GIS does not have so that it can interact with peripherals such as GPS receivers and laser range-finders.
Field Unit Limitations. The application may need to restrict its User Interface on the field unit as pen computers and PDAs have small screens and so all unnecessary elements must be removed as screen “real estate” is at a premium.
Technology Limitations. Client-Server applications that use web browsers and Internet communication protocols are restricted in the User Interface functionality that they can provide. The thicker the client, the more advanced the User Interface can be. Strictly speaking this does not prevent a uniform User Interface, as the same client can be used on office machines, but in reality it would be unusual to use a browser interface to the GIS from a desktop.
Ordinarily it should not be important which language the GIS was written in (though some will be more customizable than others), but a field solution implemented in Java will be platform independent, allowing a consistent User Interface on CE, pen and desktop machines.
Data Storage
On a Client-Server system, data storage is not really an issue. The real time link with the master GIS means that changes are made to the data as if from a workstation. More intelligent clients may cache data, but they will post CRUD (Create, Update, and Delete) operations as they occur. The same is not true for disconnected field solutions, where the issue of data management is the most significant problem to be overcome.
When a user takes data into the field on a disconnected GIS, the decision must be taken as to whether to lock the map area that they are extracting. Locking means that no other user can make changes to the features in the extracted map area until it is checked back in to the master. If the extracted area is locked, then the user must know the area that they will be working in, and only one field unit can be used in a particular area at any one time. By this method most data management issues can be avoided. Of the three CRUD operations, updates and deletes will not cause problems, but a method must be found to deal with creates.
If extracted areas are not locked the system must be capable of dealing with all of the CRUD operations. The issue is that the master GIS uniquely identifies each element, both spatial and non-spatial. When a new feature is created on the Field GIS, it too must be uniquely identified.
Unfortunately the field system only contains a subset of the total elements in the master, so how do we know that the field generated Id has not already been assigned to an existing feature within the master? The possible solutions to this problem are:
- Master and field systems both use the same method of generating globally unique Ids. This is easily done by affixing a machine or user Id to each feature Id, but it either restricts the range of possible Ids or makes the Ids larger and more cumbersome than they need to be.
- The master system allocates a block of available Ids to the field system when the data is extracted. New features are assigned Ids from this block, which are guaranteed to be unique within the master. Obviously this places a limit on the number of new features which can be created, but if the block size is sufficiently big then this should not be a problem. The master should ensure that unused Ids from the returned blocks are freed for subsequent use.
- The master checks for newly created features when the area is checked back in. New features are assigned a new Id by the master GIS. Using this method it no longer matters whether the Id allocated by the field GIS is globally unique.
Update and delete operations cause problems when two users carry out operations on the same feature while it is checked out. In its most extreme case, what is the master GIS to do if the field GIS extracts an area, makes changes to a feature, with an Id of say 50, and checks that area back in, but while the data was checked out another user deletes feature 50? Or if two users make different changes to the same attribute of the same feature? Conflict resolution such as this is a significant problem and normally requires functionality to catch the conflicts when the long transaction is completed, and a set of rules to determine how these conflicts are resolved.
These data management issues stem from the long transactional nature of the operations rather than from any characteristic of the field GIS, but other limitations are imposed by the resources of the field unit.
- The database systems used by the master GIS is likely to have a significant footprint, making in impractical to install them on every field machine. Both Oracle and SQL Server have lightweight versions with their own replication tools, but if the GIS uses a proprietary format database then even this may not be an option.
- Existing field systems may simply not support the database format of the master.
- If the master GIS stores its graphical elements in a file format (e.g. DGN, DWG files) the field unit will probably need to perform some conversion to open them. This conversion may not be apparent to the user but behind the scenes the format of the data has changed.
The level of integration of most field and office GIS systems at present is far from satisfactory. There are numerous reasons for this but the principal one must be that in almost every case the office GIS is implemented first and the field component is added at a later date. At that stage the choice is to start from scratch (and re-inventing the wheel is never a popular choice) or to use an existing field solution. Whichever option is chosen, it is unlikely that the field system will properly integrate with the master, as by this stage the task is to adapt the field solution to work with the master.
The way to solve this problem is to regard the office and field systems as different parts of the same solution. Instead of struggling to adapt one system to work with the other we should have the rather more achievable objective of designing a solution which works for the whole enterprise.
This of course is cold comfort to those who already have an established office GIS, but one does not need to look too far into the future to see comprehensive digital wireless coverage and high bandwidth. Those two advances will be enough to enable the replacement of disconnected field systems with Thick Client solutions for all but the most data intensive of tasks, removing many of the associated data management problems.