Analysis of Four Remote Image Fusion Algorithms for Landsat7 ETM+ PAN and Multi-spectral Imagery

— This study takes the southeastern part of Beijing as an example to compare four remote image fusion algorithms for improving the visualization of Landsat7 ETM+ imagery. This paper introduces four remote image fusion algorithms including the Smoothing Filter Based Intensity Modulation (SFIM), High Pass Filter (HPF) Transform, Brovey Transform, and Multiplication (MLT) Transform. The effectiveness of the four remote image fusion algorithms is evaluated based on different quantitative indexes, including mean, deviation, information entropy, average gradient and correlation. The study reveals that the SFIM transform is the best method to remain spectral information of the original remote image, which does not cause spectral distortion and has highest spatial frequency information. Moreover, the fused remote images from the same sensor system are of high quality and can be used for improving the latter visual interpretation.


I. INTRODUCTION
With the rapid development of information technology, sensor technology and the wide application of different satellite sensors for earth observation using the visible, near infrared, shortwave infrared, thermal infrared, microwave signals, the number of remote sensing images acquired form the same region becomes increasingly huge. Therefore, effective use of massive remote sensing data from different sensors, time phase and resolution images has become a research hot spot in the field of remote image processing. In order to eliminate the parameter differences existing in the remote data from different sensors, different time and image registration errors in the process of integration, some scholars have studied panchromatic and multi-spectral image fusion from multi-resolution sensor, and made some promising results. However efforts about this research should be further strengthened because the existing methods have certain disadvantages and some problems have not been solved quite well.
Landsat7 was launched by the National Aeronautics and Space Bureau and carried the theme: enhanced imaging sensor (ETM+), it maintains the multi-spectral characteristics of Landsat5 TM, the spatial resolution of thermal infrared band cover from 60m to 120m. As a new characteristic, it carries a panchromatic band with 15m spatial resolution. In view of the advantages of Landsat7 ETM+, it has become one of the most commonly used remote sensing data in agriculture, forest and grassland resources survey, land use mapping, geology, hydrology, marine resources survey and environmental monitoring. Researches show that if panchromatic and multi-spectral data can be effectively used, the accuracy of image interpretation, automatic classification and thematic mapping will be significantly improved.
In this paper, four different remote image fusion algorithms, the Smoothing Filter Based Intensity Modulation (SFIM), High Pass Filter (HPF) Transform, Brovey Transform and Multiplication (MLT) Transform are applied to the fusion of panchromatic and multispectral data from landsat7 ETM+. This paper analyzes the four methods respectively. And this paper shows the difference between the four algorithms. Finally, this study aims at finding the best remote image fusion method that can take full advantage of the Landsat7 ETM+.

FUSION
In general, fusion algorithms should maximally remain the important information in the image and add the false information as little as possible in the fusion processing. Meanwhile algorithms must be reliable and stable. Moreover they can suppress the interference maximally. Nowadays multi-source remote sensing image fusion can generally be divided into pixel level, feature level and decision level. In this paper, spatial filtering transform (SFIM, HPF) and the algebraic transform (Brovey, MLT) are used as the main remote images fusion algorithm. They are all based on the pixel level.

A. SFIM Transform
Smoothing Filter-based Intensity Modulation (Liu, 2000) is the transformation of brightness based on smoothing algorithm using the following transform equation: PAN band filtered by low-pass filter ( mean B ) removes high frequency information of the band, but retains its low frequency spectral information. The ratio of mean B and PAN band with high resolution can offset the lowfrequency spectral image and topographic contrast information, and only retain the structure and texture information of high resolution image while high frequency information is integrated into the low resolution image. Therefore, SFIM algorithm can be regarded as an only method adding texture features of high-resolution data into low-resolution images and keeping the spectral characteristics of the original low-resolution images.

B. HPF Transform
HPF is a high-pass filter fusion algorithm proposed by Schowengerdt in 1980, applying for Landsat MSS images. Number footnotes separately in superscripts. Place the actual footnote at the bottom of the column in which it was cited. Do not put footnotes in the reference list. Use letters for table footnotes (see Table 1). Chavez et al. (1991) used this algorithm to merge TM, SPOT and aerial images; the fusion formula is as follows: Where HPF B is the fusion image transformed by HPF algorithm, and high FB is the high spatial resolution image filtered by the high-pass filter that selects 3 ! 3 window as the convolution domain. The result is divided by 2 so as to avoid the excessive expansion of brightness values. This algorithm suppresses the low-frequency spectral information, enhances the high-frequency spatial information and increases the spatial resolution of images through the high-pass filtering firstly, and integration the high spatial resolution image with the low resolution image.

C. Brovey Transform
The Brovey transform, established and promoted by an American scientist-Brovey, is also called the color normalization transform because it involves a red-greenblue (RGB) color transform method, retains the corresponding spectral feature of each pixel, and transforms all the luminance information into a panchromatic image of high spatial resolution. The Brovey transform equation is defined as: where Brovey B is the fusion image created by the algorithm, low B indicates the low-resolution images which are from six multi-spectral bands with 30m of ETM+; high B is the high spatial resolution image which is panchromatic band with 15m resolution; i is the band number.

D. MLT Transform
MLT is a simple multiplication fusion operation, which reflects the mixed messages of the low spatial resolution images and the high spatial resolution images. The MLT mathematical formula is expressed as: B is the low spatial resolution images which are from six multi-spectral bands with 30m of ETM+; high B is the high resolution image, that is panchromatic band with a resolution of 15m; i is band number, j and k is row and column of images respectively. The maximum pixel brightness value acquired by multiplying two bands can reach to 65025 in theory. The operation of square root can avoid overlarge brightness value. The corresponding weights (a and b) of two bands for normalization need to be decided according to the corresponding situations.

III. EVALUATION INDICATOR OF REMOTE IMAGE FUSION
Fusion evaluation plays an important part in the remote image fusion processing. Currently, there are no uniform methods and standards for fusion evaluation of multisource remote sensing image, objective and quantitative evaluation are mainly conducted through a variety of statistical methods. The commonly used evaluation parameters include the mean, deviation, information entropy, average gradient, correlation coefficient. Take an image of size m ! n as an example, the image gray scale range is 0-255, M(x,y) and F(x,y) represent the lowresolution multi-spectral image and fusion image respectively. The computation methods and meanings of each evaluation parameters are listed in the following part 3.1, part 3.2, part 3.3 and part 3.4.

A. Mean
Mean is the average of the pixel gray values, reflecting the average brightness of the image. If the average value is medium, the fusion images have a good visual effect. It is calculated using the following equation:

B. Deviation
Deviation refers to the difference average gray value between the original image and the fused image. Deviation can be expressed as: Deviation reflects the average change level of spectral characteristics between fusion image and original multispectral image. The greater the difference value is, the larger the spectrum distortion is.

C. Entropy
Entropy of a remote image is an important indicator of measuring the abundance of the image, which indicates the average information content of a remote image. In general, the greater the entropy is, the more abundant PAPER ANALYSIS OF FOUR REMOTE IMAGE FUSION ALGORITHMS FOR LANDSAT7 ETM+ PAN AND MULTI-SPECTRAL IMAGERY information a remote image contains. According to the principles of Shannon information theory, information entropy of an image X with 8 bit can be expressed as: Where i P is the probability of the image pixels with the gray value of i .Greater information entropy becomes after fusing the images, the more the information of the fused image increase. The more information abundance in the original images, the higher the quality of the fusion image. So entropy can objectively evaluate changes of information contents.

D. Average gradient
The average gradient is the reflection of the contrast of small details in image, which can be used to evaluate the clarity of an image. The average gradient is as follows: (8) In general, the larger the average gradient is, clearer fusion image is.

E. Correlation
Correlation reflects similarity of the spectral characteristics between the original multi-spectral image and the fusion image. This parameter can be expressed as: The larger the correlation coefficient is, the more the original spectral characteristics of multi-spectral images remain in the fusion image.

IV. REMOTE IMAGE FUSION RESULT AND ANALYSIS
The southeastern part of Beijing City is selected as the study area, which locates in the transition zone between urban and rural, includes complex land use types such as cropland, grassland, water and buildings. The adopting ETM+ PAN and multispectral remote images are obtained on September 6, 2006. The band 3, band 4 and band 5 of the multispectral images are chosen to fuse. The image fusion experiment results of the four fusion algorithms are shown in Fig.1.
The mean, deviation, entropy, average gradient and correlation in each band are separately calculated from the original ETM+ image and the four fused images for objective and quantitative evaluation as shown in Table I.
According to the statistical results in the Table I, the mean value of each band generated by HPF transform is minimum comparing with the other three algorithms, so the brightness of the fused image becomes darker. The largest deviation is produced in HPF transform, the second largest is in Brovey transform, and the lowest is in SFIM transform. Therefore, spectral distortion of HPF is the most serious, that of Brovey is second, that of MLT is third, and that of SFIM possess the best spectral fidelity. The situation of correlation is opposite to the deviation. The entropy represents the information content and the quality of the fused image. Excluding the Brovey transform, the entropy of each band of images generated through other algorithms has increased. The increment of entropy in HPF transform is significant. As shown in the result of average gradient, only the average gradient of the red band of Brovey transform decreases, those of the rest bands increase significantly. Average gradient indicates the involvement of the high-frequency spatial information to some extent. SFIM has highest average gradient, followed by HPF and MLT, Brovey is the lowest.  Comparing with original ETM+ images, visual interpretation, spatial resolution and clarity of four images derived from four remote image fusion algorithms have been greatly improved. The study results show that remote images with different resolutions from the same sensor system can avoid errors due to the seasonal difference, various solar illumination, other environmental condition differences and co-registration errors, which are common in different sensors. The quality of fusion image is affected by spectral fidelity and involvement of highfrequency spatial information. Through comparison of the above methods, it is shown that SFIM algorithm is the best method in retaining the spectral information of the original image, which causes little spectral distortion and remain the highest spatial frequency information from the Landsat ETM+ images of the study area.