Land Cover Classification from Remote Sensing Imagery: Revisiting and Reevaluating Classification Accuracy
Inakwu Odeh Faculty of Agriculture
Food and Natural Resources The University of Sydney
The remote sensing community has long been interested in image classification of aerially sensed and satellite imageries for land cover maps as land use information is the basis for many environmental and socioeconomic applications. However, classifying remote sensing imageries to obtain land use and land cover information still remains a challenge that depends on many factors such as complexity of landscape, selected remote sensing data, image processing and classification methods, etc. In most cases, land cover maps derived from remote sensing imageries are often judged to be insufficient in quality and thus are not considered reliable for quantitative environmental applications. This has also led to the questioning of the suitability of remotely sensed imageries for thematic mapping. Heavy dependence on spectral characteristics or black and white tone is the one of the main reason for poor reliability of classified map. The use of ancillary data before or during or after classification is one way of improving the classification accuracy and reliability of the resulting maps. In this paper we applied the most popular Maximum likelihood classifier for the classification of the land cover of Lower Hunter region of New South Wales, Australia, using Landsat-TM for the year 2005. The major land cover and land use types are Woodland, Pasture and scrubland, Vineyard, Built-up and Water body. For classification purpose seven classes (Woodland, “pasture and scrubland”, Vineyard, Builtup, Water body, “Mine and quarry” and Olive) were identified. For accuracy assessment, only the five major classes are considered while maintaining enough sample size for each land cover class. However, while built-up and vineyard land cover types were found to have a high commission error, the “pasture and scrubland” land type is characterised by high omission error. By applying post-classification sorting using ancillary data, such as DEM, land use boundaries, roads, along with spatial texture and a vegetation index, the overall classification accuracy was improved from 79.5 % to 85.4 % with overall Kappa statistics from 0.74 to 0.81. The individual class user’s accuracy of the post classification corrected map ranged from 73.1 % for vineyard to 96.1 % for water body. Therefore, we conclude that post classification refinement with the use of ancillary data is effective in improving the accuracy of land cover maps.
Remote sensing community has been interested in image classification of remote sensing imageries as classification results are the basis for many environmental and socioeconomic applications and to bring the satellite imageries to usable geographic products (Lu and Weng, 2007; Wilkinson, 2005; Foody, 2002). However, classifying a remote sensing imagery still remains a challenge that depends on many factors such as complexity of landscape in a study area, selected remote sensing data, and image processing and classification approaches etc (Steele et.al, 1998; Lu and Weng, 2007). Most of the time, land cover maps derived from remote sensing are often judged to be insufficient in quality and thus not trusted for quantitative environmental application purpose (Stow et.al., 1990; Foody, 2002; Wilkinson, 2005) thus leading to questioning of the spectral and radiometric suitability of remotely sensed data sets for thematic mapping. This means that fairly specific types of change must be identified using aerial photography and ground reconnaissance (Stow et.al., 1990). Wilkinson (2005) based on a review of 15 years of peer-reviewed experiments on satellite image classification, observed that, there has been no demonstrable improvement in classification performance over the 15 years period though a considerable inventiveness occurred in establishing and testing new classification methods during the period. He even raised doubts about the
value of continued research efforts to improve classification algorithms in remote sensing. Jensen (2005) opined that there is no surprise of low reliability of remote sensing classification as 95 % of the scientists attempt to accomplish classification only using one variable i.e. spectral characteristic (color) or black and white tone. However, some of the researchers have started utilizing ancillary data in combination of remote sensing data to improve classification accuracy (Stefanov et.al., 2001; Watson and Wilcock, 2001; Xiuwan, 2002; Abdul-salem, 2002; Currit, 2005; Yuan et.al., 2005; Judex et.al., 2006). This article is trying to show usefulness of ancillary data in improving the classification of satellite imageries.
TM/ETM+ are the most frequently used data sets at a regional scale (Lu and Weng, 2007; Stefanov et.al., 2001; Yang and Lo, 2002; Judex et.al., 2006; Yuan et.al., 2006) due to their relatively lower cost, longer history and higher frequency of archive. Information regarding the land covers over time and space is a fundamental requirement for environmental monitoring in order to prevent from detrimental environmental impacts before it becomes irreparable. In Australia, little research has been undertaken on land covers change especially in the eastern fringe of Australian continent where changes had occurred more drastically. This study is broadly aimed to fill this gap, and the study site is the Lower Hunter Valley, a well known tourist destination, within the Lower Hunter Region of New South Wales, Australia.
Landsat thematic mapper imagery of the year 2005 were classified with the most widely used parametric classifier, maximum likelihood (ML) decision rule combined with a few ancillary data (e.g. DEM and knowledge of the locality, Land use data, vegetation index and textural analysis of the landsat imagery) through an expert (or hypothesis testing) system to improve the classification accuracy. The aim of this paper is therefore to test the hypothesis that the use of ancillary data could lead to improvement of land use classification. This premise is particularly pertinent because good quality satellite imageries of the study region are not available due to cloud cover and atmospheric haziness- fairly common phenomena in the study region.
Materials and Methods
The study area is located in the lower Hunter region of New South Wales Australia, about 160 km north of Sydney. The key industries of mining, winery and tourism which are the economic engine of the area, are contributing to rapid economic growth. The study area, also called the Hunter Wine Country Private Irrigation District (HWCPID), covers approximately 379 km2. It is located within an undulating plain of the lower Hunter valley, centred on little town of Pokolbin. Geographically it lies between 151°09’43” E to 151°24’58”E Longitude and 32°37’21” S to 32°51’45” S Latitude (Fig 1).
The area has been gaining popularity as a tourist attraction due to winery, stretching grape vineyard beyond the horizon, and golf courses. Pokolbin’s image of a bucolic rural landscape with its varied mosaic of vineyards, pastures, scattered woodlands and wineries, is being threatened by overdevelopment (Holmes and Hartig, 2007). Therefore there is the concern for environmental protection of the region. In spite of the importance of land cover for environmental modelling and planning, our knowledge of land cover/land use and its dynamics for the region is limited.
Fig 1. Map of NSW with the study region
Land user types for the area:
In HWCPID land use ranges from viticulture and dairying to extensive grazing and forestry (Robinson and Helyar, 1996). Pastoral systems have been the dominant agricultural land use in the region for past 100 years and grape vines was started in the 1820s, but has expanded to 3500 hectares of vines today with an annual crush of 35,678 tonnes of grapes (Hunter valley wine country tourism paper distributed by the information centre in Cessnock). The other land uses include livestock production for beef, orchard, and vegetable production. In order to protect the booming wine grape cultivation from drought, the HWCPID was established. The Pokolbin Pipeline Project (PPP) consists of about 128 km of network PVC pipes pumping 5100 million litres annually from Hunter River. The network was designed to supply water to nearly 400 properties spread through out the project area (information collected from Mr. Ken Bray, operational manager of HWCPID, during the field visit to the site).
A 2005 subscene of path/row 89/83 of Landsat 5 Thematic Mapper (TM) was procured from the Australian Centre for Remote Sensing. As it was difficult to obtain cloud-free Landsat imagery for the period between November and March, the actively growing period for vineyard, the TM image was obtained for June 8th, 2005. Orthorectified aerial photographs of the period 2004-06 was also procured from Plateau Images, Alstonville, New South Wales, which were used as reference image for selecting training sites and for validating the classification results.
Land use map of Singleton covering the study area projected to GDA 1994 was also obtained from the Department of Natural Resource; this map contains vector layers of land covers. The meta-data specified the data set belonged to land use between June, 2000 and June, 2007.
The procured landsat TM image of 2005 was projected to WGS 1984 system, which was reprojected to GDA 1994, and then clipped off for the study area. The orthorectified aerial photographs were mosaicked together for displaying as one sheet.
Initial land cover classification based on maximum likelihood algorithm
In this study we adopted the maximum likelihood classifier (MLC), the most widely adopted parametric classification algorithm (Jensen, 2005; Bailly et.al., 2007; Liu et.al., 2002; Currit, 2005; Weng, 2002). It is the optimal choice if the assumption of a normal distribution (in feature space) for each class training area is correct (Liu et. al., 2002; Currit, 2005). The algorithm is based on probability distributions and decision rules which assume the data values to be a set of multivariate normal distributions. MLC assign a particular class to each pixel based on the shortest modified “Mahalanobis distance” of the pixel from the class mean. The algorithm also considers shape, size and orientation of the training samples.
The total number of land cover classes delineated by the classification is seven considering the characteristic of satellite data used, and knowledge of land use of the study region (Table1).
Table1. Land cover classes delineated for the classification.
||Land cover/use class
||Forest covers, and tree covers along the creeks
||Pasture and scrubland
||Natural and cultivated pastures, and scrubs with partial grassland.
||Irrigated and non irrigated vineyards
||Commercial, and residential areas, and with man made structure; road, railway lines.
||Farm dams, sewage ponds.
||Mine and quarry
Jensen (2005) suggested that it is not appropriate to attempt to derive some of level II classes (US Geological Survey Land-use Land-Cover Classification System) using Landsat TM data due to issues of spatial resolution and interpretability. Pastures could not be separated into irrigated and non-irrigated ones. There were very small patches of “Mine and quarry” and olive. Though olive covered very small area, we tried to delineate it due to its growing coverage in recent years. Processing was done as follows:
Post classification refinement using expert system classification
- One thermal band was removed and classification was performed using bands 1, 2, 3, 4, 5 and 7. The aerial photographs were used to identify real land covers on the ground for training.
- In cases where a single pre-defined land cover class has a different spectral signature in different areas, multiple signatures are created and used for the classification, but later merged into one signature for one land cover class.
- Evaluation of collected signatures was performed through viewing histogram, checking contingency matrix and calculating signature separability using a transformed divergence (TD) for a distance between signatures.
- Water body signature was collected from a feature space of 2-5 band combination (non-parametric rule).
- Thresholding was also performed. Thresholding is the process of identifying the pixels in a classified image that are the most likely to be classified incorrectly (Erdas Field Guide, 2005). Distance image file and output thematic raster layer produced by MLC were used for thresholding. The tails of histograms (pixels that are most likely to be misclassified have the higher distance file values at the tail of the histogram of the distance image) were cut off interactively and saved and the removed pixels (blackened ones) were viewed. There were only a few small speckles of removed pixels.
- Finally different signatures of each land cover were merged into one. As the output of this classification had a lot of speckles, we used a 3*3 majority filter to reduce them.
- Accuracy assessment was performed for the imagery with the merged signatures. Interpretation is based on aerial photographs and in field verification.
- Stratified random sampling design was adopted in the accuracy assessment. Only 5 classes; Woodland, “Pasture and scrubland”, Vineyard, Builtup, “Water body” were considered for accuracy assessment meeting the minimum of 50 sample points for each considered class, and the other two classes, “Mine and quary”, and Olive were not considered for accuracy assessment purpose as they were of non significant coverage in the study area. Overall accuracy, user’s and producer’s accuracies, and the Kappa statistics were then derived from the error matrix.
Post classification refinements were applied to reduce the errors caused by similarity of spectral responses of certain classes such as “Pasture and scrubland” and Vineyards and “Pasture and scrubland” and Builtup. Based on the accuracy assessment of the initial classified map, omission and commission errors need to be reduced to improve the accuracy of the map produced. To reduce the commission error of Builtup class, textural analysis of the landsat data was utilized. Additionally in order to reduce the commission error of classifying vineyard, boundary vector of military area, and western state forest were used in the expert system classification.
Builtup areas typically have significant texture resulting from buildings and street grids, whereas homogenous areas such as vineyards have little to no texture. Stefanov et.al. (2001) had successfully utilized texture analysis of landsat imagery for improving the classification accuracy of urban centres. Here, we performed the texture analysis for the TM band 3 using a 3 X 3 moving window and the variance [V; Eq. (1)] (Erdas Field Guide, 2005):
= DN value of pixel (i, j); = n
number of pixels in a window; and Mis the Mean of the moving window which is defined as [Eq.(2)]
Normalized Difference Vegetation Index (NDVI)
Normalized Difference Vegetation Index (NDVI) is the most widely used vegetation index to distinguish healthy vegetation from others or from non-vegetated areas. NDVI was derived using the expression [Eq.(3)]:
NDVI = (NIR-R) / (NIR+R)
where NIR= Near infra red (band 4 of Landsat TM image); R=Red (band 3).
Expert system classification
Results and Discussion
- Builtup class modification: Textural value of band 3 (Red) was used for reducing the commission error of Builtup class. Using the landsat imagery and initial classified map, AOI was drawn around the high built-up patches (Cessnock and Broke road area). These patches were not corrected from the initial classification, i.e. all the built-up areas were retained as such in the final classification. The Builtup class in the rest of the study area was modified using the expert system of classification based on the logic rule: the Builtup class of initial image classification with the texture value = 5 only is classified as new Builtup class in the final classification (Fig 2). Then the remainder of the initial Builtup class that were not included in the new Builtup of final map were reclassified based on their NDVI value, i.e. if NDVI is less than -0.05, then change that to water body, and if the NDVI value is between -0.05 and 0.15, then convert to vineyard, otherwise to pasture. These threshold DN values were determined by detailed inspection of the textural image of band 3 and NDVI image derived from the landsat imagery.
- Vineyard class modification: As the imagery was of the period of vineyard dormancy, there was a high spectral similarity between the vineyards and almost bare ground with scanty grasses in the military area in the north-west part of the study region and rocky open woodland of the west. Therefore the misclassified vineyards in the military area were reclassified into “Pasture and scrubland” and the apparent vineyards classified in the western state forest were reclassified into “Woodland” using a logic rule. For this land use map vector data of the area obtained from the NSW Department of Natural Resources was utilized for assigning the classes.
Additionally, as we do not expect vineyard at higher elevation above 250 m the misclassified vineyards above this elevation were also converted to pasture.
Fig 2. The left white box is the hypothesis being tested, the ellipses represent the conjuctive decision rules and right coloured boxes represent the variables used.
- Apart from builtup and vineyard classes, some corrections were made for “Mine and quary”. The apparent Builtup and Vineyard classified in the real mining patches were converted to “Mine and quarry”. The small patches of the apparent “Mine and Quarry” seen in the non-mining areas were converted to “Water body”. Similarly the apparent olive farms located at high elevation were converted to “Woodland”.
- The corrected map was filtered using 3 x 3 majority filter, to reduce speckles, which is not applied for “Water body” class. The road and railway lines are added to the Builtup class and the map was finalized.
- The corrected imagery is evaluated for accuracy and overall accuracy, user’s and producer’s accuracies, and the Kappa statistics were then derived from the error matrix.
MLC did not produce encouraging result for classifying land covers especially for Builtup and Vineyard classes. The accuracy assessment of the classified map resulting from the initial classification with MLC showed high commission error for the Builtup and Vineyard classes, meaning that there is a probability (proportionate to the error) that pixels classified as builtup and vineyard may not actually exist on the ground. “Pasture and scrubland” had high omission error, meaning that there is a probability (proportionate to the errors) that ground reference points for this class is classified incorrectly (Table 2).
As stated earlier, the imageries were acquired during the dormancy period of vineyard cultivation; it was therefore very difficult to spectrally distinguish land covers of vineyards from that of pasture area with scanty vegetation specifically in the North West of study area (military area) and also vineyards with western rocky mountainous forest (Fig.3.a). Additionally, builtup areas also were overestimated.
Table 2. Classification accuracy of initial map classified with MLC
Lu et.al. (2003) found that most of the time, the traditional approach to classification only distinguishes clearly between forest and non forest land covers. In this study, the classification accuracy for Woodland and Water body were found to be good just using the traditional MLC. But for other land cover classes, the MLC performed quite poorly. Jensen (2005b) also reasoned that the low reliability of remote sensing classification of land use and land cover is because of heavy dependence on only one variable; spectral characteristic (colour) or black and white tone. Although MLC is one of the widely used classifier, it requires input samples to have a normal distribution and heavily dependent on statistics of data. The new thinking is to let the geographical data itself “have a stronger voice” rather that let statistics derived from dataset dictate the analysis (Jensen, 2005). Expert system allows integration of remotely sensed data with other sources of georeferenced information such as previous land use data, spatial texture, and digital elevation models (DEMs), slope, aspect, geology, soils, hydrology, transportation network, vegetation to obtain greater classification accuracy (Stefanov et.al., 2001; Judex et.al., 2002; Jensen, 2005). Therefore expert system classification was constructed for post classification sorting and improvement of accuracy of the initial classification map to reduce the errors of commission and omission.
The accuracy assessment of the final map corrected with the expert system showed an increase in overall accuracy from 79.5% to 85.4% and increased overall Kappa statistics from 0.74 to 0.81 (Table 2 and 3). The commission errors of the Builtup and Vineyard classes (as indicated from improved user’s accuracy) and omission error of the “Pasture and scrubland” class (as indicated by the increased producer’s accuracy of the class) were also reduced. The misclassified patches of vineyards in the western forest as well as in the military reserve have disappeared in addition to the reduction of overly estimated builtup patches resulted from initial classification (Fig 3.b).
Overestimation of builtup was noticed in the area of low builtup areas in the initial classification; therefore they were reduced using the logic that only the builtup of the initial classification map with a textural value = 5 are reclassified as builtup in the final classification. No modification from the initial classification was performed for heavily builtup areas as they also were found to have lesser textural value at some regions due to their homogeneity. Likewise, commission error of Vineyard class was reduced removing
the vineyard patches in the military reserve and western forest. Additionally, vineyards and olives at high elevation were removed utilizing DEM of the region as these classes are not expected at higher elevation. NDVI is another widely used and useful index for the classification and improving the classification. NDVI for water body is usually a negative value, while areas of healthy vegetation have high NDVI. Builtup and vineyard in dormancy stage also are found to have NDVI < 0.15, except in the vineyard areas with grass covers on ground. With the use of various ancillary data of the area, we were able to get more than 70 % user’s accuracy for the individual classes of the final map.
Fig 3. Initial classified map with MLC (a) and the final map after correction using ancillary data (b).
Table 3. Classification accuracy of final map after correction using ancillary data
The expert system of classification allows the integration of remotely sensed data with other sources of georeferenced information, such as land use, spatial texture and DEM to obtain greater accuracy. Here we have used the widely adopted maximum likelihood classifier for initial classification and then attempted to improve the classification accuracy incorporating additional data, such as land use, DEM, spatial texture and NDVI value of the landsat imagery for post classification correction in the expert system classification and conclude that the incorporation of ancillary data along with the spectral classification is beneficial for improvement of land cover maps.
- Abdul-salam, M. (2002). Knowledge Based Object Extraction Technique. 23rd Asian Conference on Remote Sensing, Nov 25-29, 2002. Organized by Asian Association of Remote Sensing (AARS) in collaboration with the Survey Department, Kathmandu at Birendra International Convention Centre, Kathmandu, Nepal.
- Bailly, J.S., Arnaud, M., and C. Puech (2007). Boosting: a Classification Method for Remote Sensing. International Journal of Remote Sensing. Vol 28, No. 7. pp 1687-1710.
- Currit, N. (2005). Development of Remotely Sensed, Historical Land Cover Change Database for Rural Chihuahua, Mexico. International Journal of Applied Earth Observation and Geoinformation 7:232-247.
- Erdas Fild guide (2005). Leica Geosystems Geospatial Imaging, LLC.
- Foody, G.M. (2002). Status of Land Cover Classification Accuracy Assessment. Remote Sensing of Environment. 80: 185-201.
- Holmes, J., and K. Hartig (2007). Metropoliton Colonization and the Reinvention of Place: Class Polarization along the Cessnock- Pokolbin Fault Line. Geographical Research. 45 (1): 54-70.
- Hunter Valley Wine Country Tourism. Welcome to Hunter Valley Wine Country, Australia’s Oldest Wine Region, a paper distributed by the information centre at Cessnock.
- Jensen, J.R. (2005a). Thematic Information Extraction: Pattern Recognition. – A Remote Sensing Perspective. Prentice Hall Series in Geographic Information Science. Series Editor- Keith C. Clarke. 3rd Edition. Chapter 9, pp 337-406.
- Jensen, J.R. (2005b). Thematic Map Accuracy Assessment. Introductory Digital Image Processing – A Remote Sensing Perspective. Prentice Hall Series in Geographic Information Science. Series Editor- Keith C. Clarke. 3rd Edition. Chapter 13, Pp 495-515.
- Judex, M., Thamm, M.J. and G. Menz (2006). Improving Land Cover Classification with a Knowledge Based Approach and Ancillary Data. Proceeding of the workshop of the EARSeL sig on Land Use and Land Cover, dated 28-30 sep, Bonn, 2006.
- Liu, X.H., Skidmore, A.K., and H.V. Oosten (2002). Integration of Classification Methods for improvement of Land cover Map accuracy. ISPRS Journal of Photogrammetry & Remote Sensing. 56: 257-268.
- Lu, D. and Q. Weng (2007). A survey of image classification methods and techniques for improving classification performance. International Journal of Remote Sensing, Vol. 28(5): 823-870.
- Lu, D., Moran, E. And M. Batistella (2003). Linear mixture model applied to Amazonian vegetation classification. Remote Sensing of Environment. 87: 456-469.
- Robinson, J.B., and K.R. Helyar (1996). Simulating the impact of acidifying farming systems on Australian soils. Ecological Modelling 86: 207-211.
- Steele, B.M., Winne, J.C., and R.L. Redmond (1998). Estimation and Mapping of Misclassification Probabilities for Thematic Land Cover Maps. Remote Sensing od Environments. 66: 192-202.
- Stefanov, W.L., Ramsey, M.S., P.R. Christensen (2001). Monitoring Urban Land Cover Change: An Expert System Approach to Land Cover Classification of Semiarid to arid Urban Centers. Remote Sensing of Environment. 77: 173-185.
- Stow, D.A., Collins, D., and D. McKinsey (1990). Land Use Change Detection Based on Multi Date Imagery from Different Satellite Sensor Systems. Geocarto International Vol.5 (3): 3-12.
- Wilkinson, G.G. (2005). Results of Implications of a Study of Fiteen Years of Satellite Classification Experiments. IEEE Transaction on Geoscience and Remote Sensing, Vol. 43(3): 433-440.
- Watson, N., and D. Wilcock (2001). Preclassification as an Aid to the Improvement of Thematic and Spatial Accuracy in Land Cover Maps Derived from Satellite Imagery. Remote Sensing of Environment. 75:267-278.
- Weng, Q. (2002). Land Use Change Analysis in the Zhujiang Delta of China using Remote Sensing, GIS and Stochastic Modelling. Journal of Environmental Manangement. 64:273-284.
- Xiuwan, C. (2002). Using Remote Sensing and GIS to Analyse Land Cover Change and its Impacts on Regional Sustainable Development. International Journal of Remote Sensing, 2002, Vol 23, no-1, 107-124.
- Yang, X. and C.P. Lo (2002). Using a Time Series of Satellite Imagery to Detect Land Use and Land Cover Changes in the Atlanta, Georgia Metropolitan Area. International Journal of Remote Sensing. Vol 23, No. 9, 1775-1798.
- Yuan, F., Sawaya, K.E., Loeffelholz, B.C., and M.E.Bauer (2005). Land Cover Classification and Change Analysis of the Twin Cities (Minnesota) Metropoliton Area by multitemporal Landsat Remote Sensing. Remote Sensing of Environment. 98: 317-328.