| GISdevelopment.net ---> AARS ---> ACRS 2000 ---> Poster Session 1 |
Integration of Multisensor Multitemporal Satellite Data
for Agricultural Vegetation Mapping
L. ZHU and R. TATEISHI
Center for Environment Remote Sensing (CEReS), Chiba University
1-33, Yayoi-cho, Inage-ku, Chiba, 263-8522 Japan
Tel: (81)-43-290-3850 Fax: (81)-43-290-3857
E-mail:zhulin@ceres.cr.chiba-u.ac.jp
L. ZHU and R. TATEISHI
Center for Environment Remote Sensing (CEReS), Chiba University
1-33, Yayoi-cho, Inage-ku, Chiba, 263-8522 Japan
Tel: (81)-43-290-3850 Fax: (81)-43-290-3857
E-mail:zhulin@ceres.cr.chiba-u.ac.jp
Keywords: Data Fusion, Transition Probability, Vegetation Dynamics Indicator
Abstract
Efficient integration of temporal, spectral and spatial resolution information is important for accurate mapping of agricultural areas. In this study, a new temporal fusion classification (TFC) model is presented for classification of agricultural vegetation, based on statistical fusion of single source multitemporal satellite images. In the proposed model, the temporal dependence of multitemporal images is taken into account by estimating transition probabilities from the change pattern of vegetation dynamics indicator (VDI). For integration of multisensor multitemporal satellite data, an extended multisensor temporal fusion classification (MTFC) model is proposed, concerning both temporal attributes and reliability of multiple data sources. The feasibility of the new method is verified by using multitemporal Landsat TM and ERS SAR satellite images, and experimental results show improved performance than the conventional method.
1. Introduction
The effective agricultural mapping and monitoring are required for a variety of applications ranging from general inventory requirements to ecological studies. Remote sensing has shown great potential in agricultural mapping and monitoring due to its advantages over traditional procedures in terms of cost effectiveness and timeliness in the availability of information over larger areas (Murthy et al 1998). Automated interpretation of satellite images for agricultural area mapping is relatively complicated due to spectrum similarity of agricultural crops. Fusion techniques have been adopted for crop discrimination to provide increased interpretation capabilities and more reliable results since data with different characteristics are combined. The efficient fusion measure depends on the better understanding of characteristics of sophisticated multisource data and selecting the optimal interpretation algorithm. Advanced analytical or numerical data fusion techniques are imperative for the integration of temporal, spectral and spatial resolution information.
The aim of this study is to incorporate the temporal dependence of multitemporal image data into the fusion algorithm by estimating transition probabilities theoretically and reasonably from the change pattern of VDI, and consequently enhance the interpretation capabilities; moreover, to integrate multisensor multitemporal satellite data effectively, i.e., both the temporal attribute and the reliability of multiple data sources are concerned.
2. Methodology
2.1 Temporal Data Fusion Based on a Bayesian Formulation
Let us consider that two multispectral remote sensing images acquired at time t1 and t2 on the same area are examined. Let us suppose that a pixel of the multispectral image acquired at time t1 and a spatially corresponding pixel of multispectral image acquired at time t2 . These pixels are characterized by the m-variate observation feature vectors X1 and X2, respectively. Let wi(i=1, 2, ..., n ) and Vk(k=1, 2, ..., n ) be the set of possible land cover classes at time t1 and t2 respectively, if we classify each couple of pixels independently of any other on the basis only of its feature vectors X1 and X1, based on the Bayes rule, it requires that the couple of classes ( wi,Vk) be selected that provides the maximum likelihood L(wi,Vk) , according to Swain (1978) and by applying some transformations, the likelihood function that will be used in the decision rule takes the following form:
2.2 Determination of Transition Probabilities
According to formula (1), we can see that only transition probabilities P(Vk|wi represent the temporal dependence of multitemporal images. Formerly, when we incorporated temporal aspect to the fusion model, transition probabilities were always decided empirically (Solberg et al 1994). It should be more reasonable to decide transition probabilities using the change pattern of VDI, e.g., Normalized difference vegetation index (NDVI) for optical data and backscattering coefficient for SAR data. We can use the change of VDI to represent crop seasonal differences, or land use change of terrain categories. For the consecutive pair of images, let us define the change index (CI) using VDI derived from multitemporal images like following,
CI=DN2-DN1 (2)
Here, DN2 and DN1 are the pixel digital numbers of VDI at time t1 and t2 , respectively. Let us name the change pattern of VDI, which are calculated from the training data set and decided by the predetermined thresholds, as the estimated change pattern (ECP). Thus, the ECP of VDI can be defined as:
if x2£CI
£x1, then ECP=0
if CI>x1, then ECP=1
if CI <x2, then ECP=-1 (3)
Here, x1 and x2 are the predetermined thresholds to decide the estimated change pattern (ECP) of VDI from time t1 to time t1, the determination of x1 and x2 will be discussed afterward. The change pattern of VDI, which are calculated directly from the classification processing of images and defined as the same as formulation (3), are named as the actual change pattern (ACP). By comparing the prepared ECP to the ACP, the transition probabilities P(Vk|wi) are defined as follows:
Here P1(max) and P2(max) are the maximum a posteriori probabilities among the different classes at time t1 and t2, respectively. a and b are user-specified constants, which control the degree of consistency between the multitemporal data.
2.3 Multisensor Data Fusion Modelling
For integration of multisensor multitemporal satellite data, let us consider the m-variate observation feature vectors of Landsat TM data XTt and the set of possible land cover classes Cit (i =1, 2, …, n) related to time t (t =1, 2, …, p), also the d-variate observation feature vectors of ERS SAR data XSg and the set of possible land cover classes Eig(i=1, 2, …, n) related to time g(g=1, 2, ..., q). For each data source, it is suggested to allow different data sources to be weighted differently according to some measure of their reliability (Benediktsson and Swain 1992). Thus, the likelihood function that both the temporal attribute of single-source data and the reliability of multi-source data are concerned can be derived to take the following form:
Here, lT and lS, 0<lT, lS<1 represent the reliability factors associated with the temporal data fusion performance on multitemporal Landsat TM data and ERS SAR data, respectively.
3. Experiment
3.1 Study Area
The study area is located in the region of Zhangwu County, a typical agricultural area of Liaoning province in the northeast part of China. The image size is 1024 x 1024 pixels, corresponding to 25.6 x 25.6 km2, with geographical coordinates from 42°22¢6²N to 42°35¢50²N in latitude and from 122°12¢56²E to 122°31¢50²E in longitude.
Zhangwu's economy is oriented toward agriculture, producing mainly crops such as corn and grain, as well as soybean, wheat, pachyrhizus, peanut, and tobacco. The main crop season in this area is from April to October, including the driest month July. The ground survey trip was carried out in July 1999. The classification of the following land cover classes is considered: corn, paddy, soybean, wheat, meadow, poplar, pine, bare land, urban, water, dried area and deforest area.
3.2 Data Acquisition and Preprocessing
In this study, two Landsat TM images and two ERS-1 SAR images were applied for investigating the performance of multisensor temporal fusion model. The multitemporal Landsat TM images collected on 19 May 1994 and 23 August 1994 were selected mainly because these are the best available cloud-free scenes in the crop season. Relative radiometric correction was performed on multitemporal Landsat TM images so that both the influences of atmospheric conditions and sun elevation angle were removed (Oguma and Yamagata 1997). The two ERS-1 SAR images were acquired on 14 July 1994 and 9 September 1994. For the preprocessing of SAR images, the 16-bit data were firstly converted to 8-bit data. Then, two passes of filtering were performed in order to reduce inherent noise, the first filtering was done by using 3 by 3 median filter, the second iteration using 5 by 5 mean filter. The multitemporal TM and SAR images were co-registered and resampled to 25m pixel size, then geometrically corrected in UTM projection using the nearest neighbor method. The RMS error less than 1 pixel was yielded. Only six channels of TM data were used except the thermal channel 6. NDVI data was extracted and scaled to integer value from 0 to 200 to be used as VDI. For temporal data fusion of ERS SAR data, we used the digital value of SAR backscattering coefficient directly as VDI.
3.3 Parameter Selection
How to decide the thresholds x1 and x2 is very important. It is essential that the specification of x1 and x2 should make the classes to be more easily separated. For different classes to be classified, VDI exhibit different change directions and magnitudes. We firstly derived the VDI values of different classes from the training data set of multitemporal satellite data, then we calculated CI values between all the same classes and different classes, finally, we deliberated the results and selected the most appropriate thresholds to try to separate all CI values into three groups evenly: increasing, decreasing and constant. We can easily decide the thresholds and divide these values into three groups. For Landsat TM data, we decided the thresholds as x1=13 and x2=-1, and for ERS SAR data, we decided the thresholds as x1=12 and x2=-2.
a and b are user-specified parameters which control the degree of consistency between the temporal data, they were determined experimentally with respect to the overall classification accuracy. Through an inspection of these results, the optimal choice (aTM=0.6 and bTM=0.0) was made that the classification performance was best, and the optimal choice of aSAR and bSAR respect to classification accuracy is aSAR=0.7 and bSAR =0.0. For the determination of reliability factors lT and lS, in our MTFC model, lT and lS do not simply represent the reliability of different data sources, but the reliability of the proposed TFC model applied on multiple source data. Thus, the determination of lT and lS rely on classification results of the TFC model for multitemporal Landsat TM and ERS SAR data.
4. Results and Discussion
The performances of the proposed TFC technique are assessed and compared with the conventional Maximum Likelihood Classification (MLC) method. To provide a reference for comparison, the single period images are first analyzed separately, the performances of the MLC method are 73.3% and 73.1% correct for the May 1994 and August 1994 Landsat TM data, respectively, and 24.5% and 22.1% correct for the July 1994 and September 1994 ERS SAR data, respectively. The results of multitemporal classification are substantially better than either of single period performances. The results demonstrated by the proposed TFC method show an overall accuracy of 89.0% and 39.6% for Landsat TM and ERS SAR data, respectively, these are truly better than the MLC method of 85.5% correct performance for TM data and 39.1% correct performance for SAR data. Moreover, 10 of the all 12 classes show improved accuracy using the proposed TFC model compared to the MLC method for Landsat TM data. The results indicate that the proposed TFC method can be applicable for multiple source satellite data which have different characteristics. Furthermore, the computation time using the TFC method is less than half of the time used by the MLC method.
For multiple source classification of multitemporal Landsat TM and ERS SAR images, the proposed multisensor temporal fusion classification (MTFC) model shows improved 89.1% correct performance, compared to an overall accuracy of 88.9% derived by the simple combination temporal fusion classification (SCTFC) model which ignores the reliabilities of different data sources, and much better than the MLC method of 86.4% correct performance.
5. Conclusion
We proposed a new method based on the Bayesian formulation and try to integrate multisensor multitemporal satellite data for accurate mapping of agricultural area. This is a statistical method based on multisensor multitemporal data fusion, which takes into account the temporal dependence of images and the reliabilities of different data sources. In the proposed TFC model, the class-dependent likelihood of multitemporal data are calculated, respectively, and the transition probabilities are estimated from the change pattern of VDI between the same and different classes of images, then the Bayes optimal classification is performed by maximizing the set of the class-dependent likelihood and the transition probabilities. The proposed TFC model was further extended and can be applied for classification of optical and SAR satellite images. This method can also be used as an alternative method for change detection of land cover and land use. Experimental results showed that the fusion model is robust and can improve the classification accuracy, moreover, reduce the dimensionality of the probability functions used and facilitate the computation over time.
References:
Abstract
Efficient integration of temporal, spectral and spatial resolution information is important for accurate mapping of agricultural areas. In this study, a new temporal fusion classification (TFC) model is presented for classification of agricultural vegetation, based on statistical fusion of single source multitemporal satellite images. In the proposed model, the temporal dependence of multitemporal images is taken into account by estimating transition probabilities from the change pattern of vegetation dynamics indicator (VDI). For integration of multisensor multitemporal satellite data, an extended multisensor temporal fusion classification (MTFC) model is proposed, concerning both temporal attributes and reliability of multiple data sources. The feasibility of the new method is verified by using multitemporal Landsat TM and ERS SAR satellite images, and experimental results show improved performance than the conventional method.
1. Introduction
The effective agricultural mapping and monitoring are required for a variety of applications ranging from general inventory requirements to ecological studies. Remote sensing has shown great potential in agricultural mapping and monitoring due to its advantages over traditional procedures in terms of cost effectiveness and timeliness in the availability of information over larger areas (Murthy et al 1998). Automated interpretation of satellite images for agricultural area mapping is relatively complicated due to spectrum similarity of agricultural crops. Fusion techniques have been adopted for crop discrimination to provide increased interpretation capabilities and more reliable results since data with different characteristics are combined. The efficient fusion measure depends on the better understanding of characteristics of sophisticated multisource data and selecting the optimal interpretation algorithm. Advanced analytical or numerical data fusion techniques are imperative for the integration of temporal, spectral and spatial resolution information.
The aim of this study is to incorporate the temporal dependence of multitemporal image data into the fusion algorithm by estimating transition probabilities theoretically and reasonably from the change pattern of VDI, and consequently enhance the interpretation capabilities; moreover, to integrate multisensor multitemporal satellite data effectively, i.e., both the temporal attribute and the reliability of multiple data sources are concerned.
2. Methodology
2.1 Temporal Data Fusion Based on a Bayesian Formulation
Let us consider that two multispectral remote sensing images acquired at time t1 and t2 on the same area are examined. Let us suppose that a pixel of the multispectral image acquired at time t1 and a spatially corresponding pixel of multispectral image acquired at time t2 . These pixels are characterized by the m-variate observation feature vectors X1 and X2, respectively. Let wi(i=1, 2, ..., n ) and Vk(k=1, 2, ..., n ) be the set of possible land cover classes at time t1 and t2 respectively, if we classify each couple of pixels independently of any other on the basis only of its feature vectors X1 and X1, based on the Bayes rule, it requires that the couple of classes ( wi,Vk) be selected that provides the maximum likelihood L(wi,Vk) , according to Swain (1978) and by applying some transformations, the likelihood function that will be used in the decision rule takes the following form:
According to formula (1), we can see that only transition probabilities P(Vk|wi represent the temporal dependence of multitemporal images. Formerly, when we incorporated temporal aspect to the fusion model, transition probabilities were always decided empirically (Solberg et al 1994). It should be more reasonable to decide transition probabilities using the change pattern of VDI, e.g., Normalized difference vegetation index (NDVI) for optical data and backscattering coefficient for SAR data. We can use the change of VDI to represent crop seasonal differences, or land use change of terrain categories. For the consecutive pair of images, let us define the change index (CI) using VDI derived from multitemporal images like following,
Here, DN2 and DN1 are the pixel digital numbers of VDI at time t1 and t2 , respectively. Let us name the change pattern of VDI, which are calculated from the training data set and decided by the predetermined thresholds, as the estimated change pattern (ECP). Thus, the ECP of VDI can be defined as:
if CI>x1, then ECP=1
if CI <x2, then ECP=-1 (3)
Here, x1 and x2 are the predetermined thresholds to decide the estimated change pattern (ECP) of VDI from time t1 to time t1, the determination of x1 and x2 will be discussed afterward. The change pattern of VDI, which are calculated directly from the classification processing of images and defined as the same as formulation (3), are named as the actual change pattern (ACP). By comparing the prepared ECP to the ACP, the transition probabilities P(Vk|wi) are defined as follows:
Here P1(max) and P2(max) are the maximum a posteriori probabilities among the different classes at time t1 and t2, respectively. a and b are user-specified constants, which control the degree of consistency between the multitemporal data.
2.3 Multisensor Data Fusion Modelling
For integration of multisensor multitemporal satellite data, let us consider the m-variate observation feature vectors of Landsat TM data XTt and the set of possible land cover classes Cit (i =1, 2, …, n) related to time t (t =1, 2, …, p), also the d-variate observation feature vectors of ERS SAR data XSg and the set of possible land cover classes Eig(i=1, 2, …, n) related to time g(g=1, 2, ..., q). For each data source, it is suggested to allow different data sources to be weighted differently according to some measure of their reliability (Benediktsson and Swain 1992). Thus, the likelihood function that both the temporal attribute of single-source data and the reliability of multi-source data are concerned can be derived to take the following form:
Here, lT and lS, 0<lT, lS<1 represent the reliability factors associated with the temporal data fusion performance on multitemporal Landsat TM data and ERS SAR data, respectively.
3. Experiment
3.1 Study Area
The study area is located in the region of Zhangwu County, a typical agricultural area of Liaoning province in the northeast part of China. The image size is 1024 x 1024 pixels, corresponding to 25.6 x 25.6 km2, with geographical coordinates from 42°22¢6²N to 42°35¢50²N in latitude and from 122°12¢56²E to 122°31¢50²E in longitude.
Zhangwu's economy is oriented toward agriculture, producing mainly crops such as corn and grain, as well as soybean, wheat, pachyrhizus, peanut, and tobacco. The main crop season in this area is from April to October, including the driest month July. The ground survey trip was carried out in July 1999. The classification of the following land cover classes is considered: corn, paddy, soybean, wheat, meadow, poplar, pine, bare land, urban, water, dried area and deforest area.
3.2 Data Acquisition and Preprocessing
In this study, two Landsat TM images and two ERS-1 SAR images were applied for investigating the performance of multisensor temporal fusion model. The multitemporal Landsat TM images collected on 19 May 1994 and 23 August 1994 were selected mainly because these are the best available cloud-free scenes in the crop season. Relative radiometric correction was performed on multitemporal Landsat TM images so that both the influences of atmospheric conditions and sun elevation angle were removed (Oguma and Yamagata 1997). The two ERS-1 SAR images were acquired on 14 July 1994 and 9 September 1994. For the preprocessing of SAR images, the 16-bit data were firstly converted to 8-bit data. Then, two passes of filtering were performed in order to reduce inherent noise, the first filtering was done by using 3 by 3 median filter, the second iteration using 5 by 5 mean filter. The multitemporal TM and SAR images were co-registered and resampled to 25m pixel size, then geometrically corrected in UTM projection using the nearest neighbor method. The RMS error less than 1 pixel was yielded. Only six channels of TM data were used except the thermal channel 6. NDVI data was extracted and scaled to integer value from 0 to 200 to be used as VDI. For temporal data fusion of ERS SAR data, we used the digital value of SAR backscattering coefficient directly as VDI.
3.3 Parameter Selection
How to decide the thresholds x1 and x2 is very important. It is essential that the specification of x1 and x2 should make the classes to be more easily separated. For different classes to be classified, VDI exhibit different change directions and magnitudes. We firstly derived the VDI values of different classes from the training data set of multitemporal satellite data, then we calculated CI values between all the same classes and different classes, finally, we deliberated the results and selected the most appropriate thresholds to try to separate all CI values into three groups evenly: increasing, decreasing and constant. We can easily decide the thresholds and divide these values into three groups. For Landsat TM data, we decided the thresholds as x1=13 and x2=-1, and for ERS SAR data, we decided the thresholds as x1=12 and x2=-2.
a and b are user-specified parameters which control the degree of consistency between the temporal data, they were determined experimentally with respect to the overall classification accuracy. Through an inspection of these results, the optimal choice (aTM=0.6 and bTM=0.0) was made that the classification performance was best, and the optimal choice of aSAR and bSAR respect to classification accuracy is aSAR=0.7 and bSAR =0.0. For the determination of reliability factors lT and lS, in our MTFC model, lT and lS do not simply represent the reliability of different data sources, but the reliability of the proposed TFC model applied on multiple source data. Thus, the determination of lT and lS rely on classification results of the TFC model for multitemporal Landsat TM and ERS SAR data.
4. Results and Discussion
The performances of the proposed TFC technique are assessed and compared with the conventional Maximum Likelihood Classification (MLC) method. To provide a reference for comparison, the single period images are first analyzed separately, the performances of the MLC method are 73.3% and 73.1% correct for the May 1994 and August 1994 Landsat TM data, respectively, and 24.5% and 22.1% correct for the July 1994 and September 1994 ERS SAR data, respectively. The results of multitemporal classification are substantially better than either of single period performances. The results demonstrated by the proposed TFC method show an overall accuracy of 89.0% and 39.6% for Landsat TM and ERS SAR data, respectively, these are truly better than the MLC method of 85.5% correct performance for TM data and 39.1% correct performance for SAR data. Moreover, 10 of the all 12 classes show improved accuracy using the proposed TFC model compared to the MLC method for Landsat TM data. The results indicate that the proposed TFC method can be applicable for multiple source satellite data which have different characteristics. Furthermore, the computation time using the TFC method is less than half of the time used by the MLC method.
For multiple source classification of multitemporal Landsat TM and ERS SAR images, the proposed multisensor temporal fusion classification (MTFC) model shows improved 89.1% correct performance, compared to an overall accuracy of 88.9% derived by the simple combination temporal fusion classification (SCTFC) model which ignores the reliabilities of different data sources, and much better than the MLC method of 86.4% correct performance.
5. Conclusion
We proposed a new method based on the Bayesian formulation and try to integrate multisensor multitemporal satellite data for accurate mapping of agricultural area. This is a statistical method based on multisensor multitemporal data fusion, which takes into account the temporal dependence of images and the reliabilities of different data sources. In the proposed TFC model, the class-dependent likelihood of multitemporal data are calculated, respectively, and the transition probabilities are estimated from the change pattern of VDI between the same and different classes of images, then the Bayes optimal classification is performed by maximizing the set of the class-dependent likelihood and the transition probabilities. The proposed TFC model was further extended and can be applied for classification of optical and SAR satellite images. This method can also be used as an alternative method for change detection of land cover and land use. Experimental results showed that the fusion model is robust and can improve the classification accuracy, moreover, reduce the dimensionality of the probability functions used and facilitate the computation over time.
References:
- Benediktsson, J. A., and Swain, P. H., 1992, Consensus theoretic classification methods. I.E.E.E. Transactions on Systems, Man, Cybernetics, 22, 688-704.
- Murthy, C. S., Raju, P. V., Jonna, S., Abdul Hakeem, K., and Thiruvengadachari, S., 1998, Satellite derived crop calendar for canal operation schedule in Bhadra project command area, India. International Journal of Remote Sensing, 19, 2865-2876.
- Oguma, H., and Yamagata, Y., 1997, Study on effective observing season selection to produce the wetland vegetation map. Journal of the Japan Society of Photogrammetry and Remote Sensing, 36, 5-16.
- Schistad Solberg, A. H., Jain, A. K., and Taxt, T., 1994, MultiSource classification of remotely sensed data: fusion of Landsat TM and SAR images. I.E.E.E. Transactions on Geoscience and Remote sensing, 32, 768-777.
- Swain, P. H., 1978, Bayesian classification in a time-varying environment. I.E.E.E. Transactions on Systems, Man, Cybernetics, SMC-8, 879-883.