A noise reduction method for portable Lidar Echo data using statistical technique
Hiroshi Okumura, Tradashi Sugita, Hironori Matsumoto, Nobuo Takeuchi
Remote Sensing and Image Research Center, Chiba University
I-22, Yayoi-cho, Inage-ku, Chiba-shi, Chiba 263, Japan
Abstract:
A new noise reduction method for lieder echo data is proposed. This method is based on canonical correlation analysis between two multivariate data groups. GROUP-1 dataset includes moving averaged data of GROUP-1. As the experimental results using both artificial data and actual lidar echo data, the amplitude of noise component in the primary canonical variety of GROUP-1 (result) is reduced more than 30%.
Introduction
Lidar (Leaser Radar) is a powerful tool for monitoring air pollution, stratospheric and boundary layer, plume dispersion, visibility, and studying atmospheric structure and cloud physics. In this research field, a portable lidar system will be more widely used in the future because of its easy handling and wielder portability. However, the signal-to-noise (S/N) ratio of each echo datum which is acquired with such portable lidar systems is not so high because of its low lasing power in comparison with large scale lidar systems.
Lidar echo data includes not only backscatter signal from scatterer but also various kinds of noise components (e.g. shot noise caused by fluctuation of electric current, dark current noise of detector, thermal noise of a resister in the amplifier etc.). The noise reduction methods which are applied for lidar echo data are as follows.
- accumulation of lidar echo data on the assumption of noise randomness;
- moving average method;
- filtering using fast Fourier transform;
- hysteresis smoothing method.
The METHOD (a) is generally using for noise reduction. However, this method is not included in the actual lidar echo data in most cases. Although the METHOD (b) and (c) are very signal. In contrast with this, the METHOD (d) is not so effective against reduction of strong pulse noises.
We developed a new noise reduction method based on statistical technique. NORMALS (Noise Reduction method using Multivariate Analysis technique for Lidar echo Signals), in order to overcome these problems. Canonical correlation analysis is applied for noise reduction by the NOMALS. The NROMALS has the advantage of an effective noise reduction without wave form distortion of backscatter signal.
In the following chapters, the details of the NORMALS are described and its validity is confirmed by numerical simulation and also by the application to actual lidar echo data.
Principle
1 Canonical Correlation Analysis [1], [2]
Canonical correlation analysis (CCA) is one of multivatiate statistical methods. The CCA method converts characteristic variates into uncorrelated integrate variates by the same way as in principal component analysis (PCA). While the PCA is applied to one group of p characteristic
Variates, the CCA is applied to two characteristic variate groups. The first group (GROUMND-1) and the second group (GROUP-2) consists of s and t (p= s + t, t s) characteristic variates, respectively.
Suppose that two characteristic variate groups, X
1..X
s and X
s+i. The mean values of these variates are normalized to 0. Consider following linear compounds;
Coefficients in Eq.1 and Eq. 2 I
pi and m
qi (I, p=1, …., s, j, q=1,….,t), are determined by way of satisfying following conditions;
- The mean values and the variance values of ui and vj are equal to 0 and 1, respectively,
- Ui is uncorrelted with u, (i¹i);
- Vi is uncorrelated with vj (j¹j);
- ui uncorrelated with vj (i¹j);
- correlation coefficients exists between uk and vk (k=1,…., s).
Integrated variates u
i and v
i (i=1…….,t) are called canonical variates, and correlation coefficient r
k is called i-th canonical correlation. Fig. 1 shows the relation between characteristic variates and canonical variates, schematically.
Fig. 1. Characteristic variates and canonical variates(s=t=3)
