PAI-Alchemy A solution for Geo-technical data realignment for OSGB data user Juber Ahmad Executive Engineer (Technical Services) DSM Soft 4 Subramanian Building Promanade Road Cantonment Trichy-620001 INDIA zuber@dsmsoft.com Murali CK Senior Manager (Technical Services) DSM Soft 4 Subramanian Building Promanade Road Cantonment Trichy-620001 INDIA murali@dsmsoft.com Abstract Until quite recent, people involved in developing and using GIS paid little attention to the problems caused by inaccuracy, and imprecision in spatial datasets. Certainly there was an awareness that all data suffers from inaccuracy and imprecision, but the effects on GIS problems and solutions was not considered in great detail. The situation has changed substantially in recent years due to the advancement of data acquisition techniques and GPS technology. Integration of GPS and GIS technology widen the scope for implementation of the GIS technology and enforces to GIS user for having positional accurate spatial data to get the seamless benefit of both the technology. National Data Agencies are working on for data accuracy enhancement and providing the Positional accurate data to the user. Ordinance Survey (OS) of Great Britain has been in process of correcting the data for entire country under Positional Accuracy Improvement (PAI) Programme and now providing the positional accurate data to the user. During this process of data accuracy improvement, most of the feature has been moved. This feature movement presents major problem to utility companies, as now their own in-house data which has been generated with reference of old OS data, does not match with new OS data. To get the benefit of positional accurate data, utility companies and Local Authorities need to correct their in-house data with reference of new OS data. Method of manual correction of the data would be very time consuming and shifting may not be very accurate as well. DSM soft developed a complete solution for the issues raised by Positional Accuracy Improvement Programme (PAI) in UK. Solution includes consultancy, data audit and data repositioning. PAI-Alchemy Introduction Until quite recent, people involved in developing and using GIS paid little attention to the problems caused by inaccuracy, and imprecision in spatial datasets. Certainly there was an awareness that all data suffers from inaccuracy and imprecision, but the effects on GIS problems and solutions was not considered in great detail. The situation has changed substantially in recent years due to the advancement of data acquisition techniques and GPS technology. Integration of GPS and GIS technology widen the scope for implementation of the GIS technology and enforces to GIS user to have positional accurate spatial data to get the seamless benefit of both the technology. National Data Agencies are working on for data accuracy enhancement and providing the Positional accurate data to the user. Ordinance Survey (OS) of Great Britain has been in process of correcting the data for entire country under Positional Accuracy Improvement (PAI) Programme and now providing the positional accurate data to the user. During this process of data accuracy improvement, most of the feature has been moved. This feature movement presents major problem to utility companies, as now their own in-house data which has been generated with reference of old OS data, does not match with new OS data. To get the benefit of positional accurate data, utility companies and Local Authorities need to correct their in-house data with reference of new OS data. Method of manual correction of the data would be very time consuming and shifting may not be very accurate as well. DSM soft developed a complete solution for the issues raised by Positional Accuracy Improvement Programme (PAI) in UK. Solution includes consultancy, data audit and data repositioning. Solution Overview We believe that every organization is unique in its own way therefore their technical data too. We acknowledge the fact that it is essential to understand the data first before coming up with a solution therefore we provides the data audit facility to our customer which enables us to tailored our solution as per their specific requirement and to estimate the cost involves for the job to organization. Topological data cleaning, and data migration is the optional feature of DSM’s PAI solution. DSM soft approach to PAI is to provide complete, flexible and cost effective solution. DSM’s PAI solution will be carried out by using its in-house developed stand-alone desktop application “PAI-Alchemy” specifically developed for PAI. Salient features Batch Processing of the data Accuracy Error markers Error log file Technical Overview PAI-Alchemy has been intimately integrated with the FME object from Safe Software which got the ability to read and write all popular GIS data format. PAI-Alchemy has been developed by using .net frame, according to the state-of-art software development standards. Modular Architecture PAI-Alchemy has been developed on modular architecture which enables us to tailor it for the specific need of the customer’s data set.  PAI-Alchemy: Modular Architecture Batch Processor: Batch processor module enables PAI-Alchemy to process the data into batches and save operational cost of the solution. Batch processor works for link densification as well spatial adjustment process also. Batch Processor generates a minimum boundary rectangle from the extent of link file and the same from user’s geometrical data and process the data based on the links falls in the minimum boundary of spatial data. This is not ensuring only batch processing but seamless adjustment of the spatial data available into the data set. Link densifier: Link densification is the unique feature for PAI-Alchemy which enables it to provide almost 100 % relatively position accurate data. Concept of link densification process has been envisage by studying the various set of Link files supplied by OS along with new Landline data. It has been observed that in some cases distribution of links supplied by OS is not uniform in all the areas. In fact these links tends to follow the distribution of available feature on the landscape. In other words more dense settlement area will have more links than sparse settlement. These links becomes the base for shifting of data by using any transformation method. Accuracy of the most commonly used data transformation technique (TIN based affine transformation and Rubbersheeting) depends on equal distribution of the point over the entire plane to be shifted. To overcome the shortcoming of the above transformation methods, link densification process has been envisaged. This process creates the links on the basis of existing links (supplied by OS) by forming the triangulation using Delaunay Triangulation Method and connected the centeroid of the triangle. This can be understood from the following diagrams.  Constraint Engine Constraint engine is the heart of PAI-Alchemy which is responsible for maintaining the Business rule of an organization in its data during the data processing. It also ensures the topological connectivity of the data. As we discussed earlier also that every organization is unique in its own way therefore its data will also be unique. Even within the organization, different departments might have been using the same data differently for different proposes. In such a scenario a standard solution can not work for every organization without compromising in accuracy of the shifted data. Constraint engine will be tailored as par the requirement of the organization in turn its data. Constraint engine provides the basic frame for accommodating the business rule of an organization. Business rule can be categorized (in general) as following.
Constraint Engine works on Relativity Model based on “Distance and Angle Function”. Vector Warper Vector Warper is the spatial data realignment engine within PAI-Alchemy. It offers two most conventional method of vector shifting e.g. Rubbersheeting and Triangulated Irregular Network (TIN). Both of the methods have got their shortcomings for shifting the data. Choice between these two methods for spatial data shifting depends on several factors. Discussion on those factors is beyond the scope of this article. PAI-Alchemy uses these methods in its own way to minimize the distorted affect of both the methods. Error Marker Error marker is to mark the nonconformity against the business rules enforced by the constraint engine on customer’s spatial data. These nonconformities can occurred due to several reasons among which Real World Changes (RWC) is the most frequent. RWC is nothing but the removal, addition or modification of the features in new OS landline data in comparison to Old OS landline data. These changes are quite natural in land base data over a period of time as during this period many buildings and other features might have been changed. If a feature which has been reference for customer’s spatial data has undergone for changes will be marked as error though the spatial data will be shifted as par the calculated shift in that area. Error marks can be used at the time of manual checking/QC. Error marks are to ensure the accuracy in realignment of spatial data and provide an opportunity to validate the shifted data wherever application is not able to enforce the business rules. Error marker also generates an error log file along with error marks which can be used to know the possible cause of the error. It gives a text massage as “Referenced feature has gone through real world changes” with reference the feature code. Data Process Model From the data processing perspective, PAI-Alchemy processed the data into two steps as
This is the unique approach has been adopted by PAI-Alchemy to realign the customer’s spatial data with high level of accuracy. Data accuracy requirement depends on the purpose for the data is being used. For example, Vehicle navigation and the utility companies as electricity, power, and gas etc, requirement of data accuracy are different. For vehicle navigation 1 or 2 meter deviation is under the tolerance limit but for utility companies it can not be accepted in any case. PAI-Alchemy understands customer’s requirement and offers a choice to have the positional accurate data as per the requirement. We recommend densifying the links for better accuracy for the realigned data though its not compulsory process to follow.  Spatial Adjustment is the realignment process of customer’s data. Realignment is an automized process in PAI-Alchemy. PAI-Alchemy user needs to select the directory of input i.e. Link files, Pre PAI landline data, Post PAI landline and customer’s spatial data. The destination of output directory will also need to be selected by PAI-Alchemy user. At this destination three folders named as Realign Data, Error Marker and Error log Files will be created by the application to store the output data.  Data Process Model: Spatial Adjustment Conclusion: Positional Accuracy Improvement is all about the data accuracy. PAI-Alchemy emphases more on accuracy and it provides upto 100% of relative accuracy in processed data and also ensure that data should be realigned perfectly even if there are some real world changes also. There will always be exceptions that might come across which cannot be neglected. PAI- Alchemy has built in features Value addition that has been envisaged and developed for a PAI solution are two outputs that is delivered along with the features manipulated. The first one is that PAI-Alchemy highlights those points where it finds that data may not be shifted correctly due to some real world changes, for manual check. The second is that PAI-Alchemy also provides a possible cause (Error Log file) on why the data may not be realigned correctly and needs a manual check which can be a starting point for further investigation and building necessary additional rules for elimination of such exceptions in future PAI solutions. |