Providing data to the masses in stages
Defining an Approach
There are two scenarios that will define the approach required for conversion work and for
transitioning the data into the current GIS. If the data being converted merely overlaps in
geographic location but not in connectivity, such as data being converted to a new or unused
GIS layer, there will be no impact to the user during conversion. For example, when
appending wastewater data into a database that holds only clean water asset data,
connectivity is not an issue because each dataset has its own unique layer in the database.
This assumes that white-space management is not an issue between layers. If white-space
management were an issue, then it would be useful to have a copy of the user™s database
available during conversion since the end user could have multiple layers on screen at the
same time. However, if the continuing conversion effort has connectivity with or requires
making edits to the live data, it will be necessary to devise a plan to convert data while
allowing the records group to maintain their live data. This plan should include defining the batches of data to deliver, creating a data locking utility, acquiring a copy of the user™s
database in which to perform the conversion, tracking work being performed, and delivering
data to the user.
Allowing the Records Update Process to Continue
There are several approaches that can be taken for conversion that allow the records update
process to continue. When the conversion effort takes place on-site and directly in the ‚live™
GIS, the application should be configured to ‚lock™ data as it is being updated. Locking data
refers to changing the status of a feature so that it cannot be edited. Even though more than
one user can work in the database there will still need to be some coordination between the
records management group making the updates and the conversion group. The conversion
work will continue to be defined in batches and the work assignments for record updates will
have to avoid areas where the conversion is occurring.
Another option that will let record update work continue is to allow an off-site conversion
group remote access to the live database. Like the above approach, record updates will have
to be performed in areas where the conversion work is not taking place. With remote access
to the database, the business opens itself to the possibility of data corruption; however, this is
more a perceived risk than reality due to the development of several programs that control
data users and system access. These two approaches are the quickest means for getting the
converted data online while keeping it available to the records group and users.
When the business is unable to support one of these approaches and the conversion work
needs to be done off-site, the ‚batches™ of data should be predefined and a routine for locking
data in batches or along edges should be used. Locking the data ensures edits cannot be
made. It can be done spatially by developing a utility that looks for all data within a grid or
other specified area. Locking data allows the conversion to continue in one area at the same
time as the records group updates are being made to another area without fear of overwriting
data within conversion grids. The user™s data is then unlocked as each batch is re-delivered
and appended into the live GIS. The area for future data deliveries will need to remain
locked. In Exhibit 1, below, the darkened area provides a visual look at an area that will be
locked as a batch of data is converted.
Exhibit 1 ΠExample of Locking Data by Grid
If the locked grids were viewed as an image, then the reverse lock creates a negative image
of the data; i.e., data that is unlocked at the utility is locked to the conversion group. This is
depicted in Exhibit 2, below, where the gray area represents assets that are locked either in
the conversion database or the user™s database.
Exhibit 2 ΠExample of Reverse Locking
This prevents the conversion group from inadvertently working in a location that might also
have live update work being done, and reduces the need for extensive validation and version
resolution prior to appending the data back into the live GIS. As with the other two
approaches, the records management group will need to manage the backlog of update
assignments for the locked area. When this method is used, the utility™s users will still be
able to see and ‚read™, or reference, the data within the locked area but the records group will
Conversion
Group has
access to this
data
This data is
locked to the
utility
not be able to perform updates to that data until conversion is complete. Update backlogs
can be kept to a minimum by making deliveries at regularly scheduled intervals.
