|
|
|
Introducing correctness coefficient
as an accuracy measure for sub pixel classification results
6. Case study
Showing the applicability of the proposed method for the sub pixel
accuracy assessment we applied the proposed accuracy assessment
procedure on a sub window image, a portion of the Airborne Visible/Infrared
Imaging Spectrometer (AVIRIS) of hyperspectral data taken over an
agricultural area of California, USA in 1994. This data has 220
spectral bands about 10 nm apart in the spectral region from 0.4
to 2.45 m with a spatial resolution of 20 m. The image has a size
of 145 rows by 145 columns. Its corresponding ground truth map has
been shown in the figure 3. This map consists of 12 classes including
of Alfalfa, Corn, Corn_min, Corn_notill, Grass_pasture, Grass_pasture_moved,
Grass_Trees, Hay_windrowed, Oatas, Soybeans_clean, Soybeans_notill,
and Wood. Although there are more than 12 classes in the reality
but regarding the radiometric overlap and number of pixels for each
class we merge some of the (spectrally) similar classes and therefore
in this case study the image is classified assuming 12 spectral
classes exist in the study area. This assumption can not affect
the obtained results. Table 1 shows the number of pixels for each
class in the ground truth map.
Figure 3. Color composite image of the study area (RGB bands:
31,19 and 9).
Figure 4. The ground truth map
(includes 12 classes) of the study area.
The basic assumption of the linear spectral unmixing is that the
most of pixels are mixtures of objects. Once all the endmembers
are found in an image (using the mean ROI procedure), all the remaining
pixels are considered to be linear combinations of these endmember
pixels. Some of the hyperspectral image bands are the subject of
absorption and they contain only little signal and more noise. By
applying Minimum Noise Fraction (MNF) transformation on the hyperspectral
image, the correlation and noises are removed from the bands and
they are sorted in respect of their variance (Emami 2002).
Table .1 number of pixels per
class on the ground truth map
In the bands with a high variance, the features can be distinguished
from each other in a better mode, therefore classification accuracy
increased sensibly. Also, applying MNF transformation on the hyperspectral
images reduces the dimensionality of hyperspectral image data for
next processing steps. Hence this method was applied upon MNF images
which lead to the better results than the case which uses the original
images. In this procedure, 20, 50, 87, 120, 155, 175 and 195 number
of bands were tested for classification. In the final classification
120 bands have been used extracted of the MNF produce. Four fraction
images resulted of the LSU have been shown in the figure 5.
Although LSU classification dose not label directly any pixel, however
a classified map, which is a pixel-based representation of classification,
can be generated by choosing the class with the highest pixel value
in the fraction images for the output map (maximum value rule).
By applying this rule on the fraction maps, a thematic map has been
generated in which each pixel has a specific class label (harden
result). This map can be used to compare the proposed sub pixel
and traditional pixel based accuracy assessment methods. Hardening
the LSU results can lead to some lose of the information and accuracy
but this is one of the common approaches to evaluate the accuracy
of the subpixel classification results and also we used it only
for the comparison of the two methods.
Therefore using the ground truth data (figure 4) we calculate the
mentioned accuracy measures for the resulted fraction maps and generated
thematic map. Table 2 shows the confusion matrix of the thematic
map, overall accuracy outputted from this matrix is 86% and the
corresponding Kappa coefficient is about 84%.
Figure 5. Four sample fraction maps resulted from the LSU performed
on the [120] MNF bands.
Table 2. Confusion matrix resulted
from applying the maximum value rule on the fraction maps
Considering the number of pixels for each class in the ground truth
map and the radiometric overlap between classes this can be an optimistic
evaluation of the accuracy and in this level of the information
(hard) we have not any tool to show the error distribution over
the entire image. But using the difference and accuracy maps of
the classes (Equation. 9) we can show the error involved in the
results and spatial distribution relative to the ground truth map.
(Figure 6)
Using the concept presented at Section 5, overall CC and individual
CCs have been calculated. The calculated overall CC using 120 bands
in LSU is about 84.9%. The calculated CCs for individual classes
have been shown at Table 3.
Table 3. The calculated individual
CCs for linear unmixing of the 120 MNF bands.
Using the defined formulas at Section 5, commision and omission
errors for each class are the same as Table 4. As the table 3 and
table 4 show, we can calculate the soft omission error using the
individual CCs:
OEi (%) = 100 - CCi (%)
Table 4. Estimation of Comission and Omission errors in LUM method for MNF images (120 bands)
7. Conclusion
Accuracy assessment is one of the challenging aspects of the subpixel classifiers. Traditional accuracy assessment methods have been designed on the basis of the hard classifiers and therefore there is no straightforward approach for this aim. In the other hand ground truth data that are inherently hard in nature force the accuracy assessment to be hard.
Very few methods have been proposed so far for the sub pixel accuracy assessment. These approaches can not ensure desired flexibility and consistency. Correctness coefficient, recommended by the authors is one of the efforts to ensure the flexibility and consistency of the subpixel accuracy assessment regarding the type of the available data and classification methods.
Obtained results from the case study show that correctness coefficient (soft) and overall accuracy and kappa (hard) are approximately the same or similar in value enough. The proposed method for the accuracy assessment of the subpixel classifiers make possible to inspect the classes individually (individual CCs). Additionally each class can be investigated individually in respect of the corresponding commission and omission errors.
Although in this manner we used hard ground truth data, but this is the common case and there is no any fuzzy ground truth in the traditional image analysis. However, using the correctness coefficient parameter and its family we can take into account the major subpixel properties of such a classifiers and the effect of the inherent limitation of the hard ground truth data can be reduced.
8. References
- Conese C., Maselli F.,1992, Use of error matrices to improve area estimates with maximum likelihood classification procedures, Remote sensing of Environment, 40:113-124
- Emami H., 2002, Evaluation and decomposition of mixed pixels in remotely sensed hyperspectral images for accuracy improvement of classification results, MSc thesis, K.N.Toosi University, Tehran, Iran
- Fisher, P., 1997, The Pixel: a Snare and a Delusion, International Journal of Remote Sensing,
Vol 18, no.3, 679-685.
- Foody G.M., 1996, Approaches for the production and evaluation of fuzzy land cover classifications from remotely sensed data, INT. J. REMOTE SENSING, 1996, VOL. 17, 1317-1340.
- Foody G.M., 2002, Status of land cover classification accuracy assessment, Remote Sensing of Environment 80 (2002) 185-201.
- Gorte B ,1998, Probabilistic segmentation of remotely sensed images, PhD Thesis, Wayeningen Agricultural University (WAU), ITC, Enschlend, The Netherlands, ISBN 90-6164-157-8, ITC Publication 63.
- Maselli F., Rodolfi A., Conese C.,1996 , Fuzzy classification of spatially degraded Thematic mapper data for the estimation of sub-pixel components, Remote sensing of Environment,
- Mather P.M, 1999, Computer processing of remotely sensed images : an introduction, Second edition, John Wiley & Sons
- Richards J.A ,1993 ,Remote Sensing Digital Image Analysis: An Introduction, Second Edition, Springer, ISBN 0-387-5480-8.
|
|
|