Quality assurance in data conversion: data end users can trust
Acceptance Criteria
Based on your budget, you must determine what is satisfactory for your business applications
and document your acceptance criteria. Acceptance criteria are achieved by defining standards
for running automated validation tests and by defining a method of computing errors during your
visual inspection. These standards are the rules of the game and should be known up front and
not changed unless everyone is in agreement. As a contractual requirement, you may have the
conversion vendor be responsible for performing the automated validation tests and providing
output reports as part of your deliverable.
During your visual inspection, an Error Tally Sheet is helpful in determining compliance to your
conversion specification. The Error Tally Sheet shows the number of errors by type and
indicates the error value associated with that type of error. An error value for a feature is
associated with the number of attributes (e.g., location, symbology, length, height, etc.) it has.
For example, in Exhibit 1 if an operator missed digitizing a feature with weighted attributes, then
the error value associated with missing the feature would be 16.
Exhibit 1. Extract of Error Tally Sheet for Feature with Weighted Values
Remember that you must consider Error Transmittals (ET's) submitted by the conversion vendor
when computing the number of errors. In order to do this, you must have a procedure for
tracking ET's quickly for referencing. The best way to do this is by creating a database of all the
ET's with their associated source map or file.
Quality acceptance process
You have developed a complete and comprehensive specification, determined your data quality
requirements, and established your acceptance criteria. You must now confirm that your
conversion vendor provides the required data quality. You will need to establish processes to
verify the accuracy and completeness of the new GIS database against your conversion
specification. Verification is achieved through a combination of automated validation testing
and visual inspection procedures. Although you will be able to run automated validation reports
to check a large portion your GIS database, you still need to perform a visual inspection of a
sampling of your data. This can be time consuming and expensive, however, it must be done to
fully confirm data quality. The focus of this discussion is on the importance of acceptance
sampling, during this visual quality assurance process.
Importance of Acceptance Sampling
The common impulse is to do a 100% inspection of your conversion vendor's deliverables, but a
random sample will provide a more efficient and cost effective determination of data quality.
The use of sampling techniques raises several critical questions. How much should be sampled
to ensure your data quality requirement is met? Should re-sampling be performed if certain
criterion is not met? And for a particular sample size, what criteria should be used to accept or
reject the deliverable?
When you employ sampling techniques, you assume a certain risk that the sample may not be
representative of the rest of the work. However, the inspection process for conversion projects
can be particularly tedious or monotonous, and through sampling you will be able to produce
results as good or better than 100% inspection. In order to reduce this risk and minimize the
amount of visual inspection that must be performed, you must develop an acceptance-sampling
plan.
Acceptance sampling is a statistical sampling technique used to test a batch of work that has
already been digitized such as a division, circuit, or wire center. The purpose of acceptance
sampling is to recommend a specific action; it is not an attempt to estimate quality or to control
quality directly. The basic action recommendation is to accept or reject the items represented by
the sample.
Acceptance sampling involves establishing a sampling plan indicating the number of digitized
features that need to be inspected (sample size) and the criteria for determining the acceptance or
rejection of the deliverable. The American National Standard for Sampling Procedures and
Tables for Inspection by Attributes (ANSI/ASQC Z1.4 1993) provides sampling procedures and
reference tables for use in establishing single, double, or multiple sampling plans. A single
sampling plan is a specific type of sampling plan in which a decision to accept or reject a batch
of work is based on a single sample.
