1.

Introduction

In many remote sensing and mapping applications that require both high spatial and high spectral resolution, the fusion of panchromatic (Pan) images with a high spatial and low spectral resolution and multispectral (MS) images with a low spatial and high spectral resolution is an important issue.

To date, various techniques for fusing images have been developed. The well-known methods include the intensity-hue-saturation (IHS) transform, Brovey transform, and principal component analysis (PCA). However, a limitation of these methods is that some distortion occurs in the spectral characteristics of the original MS images. Recently, developments in wavelet analysis have provided a potential solution to this problem. The wavelet-based methods are based on the principle of extracting from a Pan image the detailed spatial information that is not presented in the corresponding MS image; this information is later injected into the MS image in a multiresolution framework.^{1, 2}

The wavelet-based method of image fusion provides a high spectral quality in fused satellite images. However, images fused by wavelets have much less spatial resolution because the critical downsampling is included in the wavelet transform. When applying remote sensing to mapping, photogrammetric measurement, or for interpretation purposes, the spatial information of a fused image is just as important as the spectral information. We therefore need to develop an advanced method of image fusion so that the fused image has the high spectral resolution of the MS image and the high spatial resolution of the Pan image at the same time.

Recently, other multiresolution analyses have been developed, including contourlets. Contourlet transform (CT) is expanded to images using basic elements such as contour segments. Contourlet was developed as an improvement over wavelet in terms of this inefficiency. The resulting transform has the multiscale and time, frequency, and localization properties of wavelet, but also offers a high degree of directionality and anisotropy. Specifically, CT involves the basis functions that are oriented at any power-of-two number of directions with flexible aspect ratios. With such a rich set of basis functions, contourlet can represent edges and textures in images better compared with wavelet.^{3, 4}

In this paper, we propose a new image fusion method based on CT and local average gradient (LAG). To the best of our knowledge, this is first time contourlet has been applied to this field. Taking the advantages of contourlet, the proposed fusion method has the higher spatial resolution than the wavelet-based fusion methods. Based on LAG, the proposed fusion method reduces the spectral distortion of the fused image by selecting either the high frequency (HF) coefficients of the MS image or the corresponding HF coefficients of the Pan image conditionally. Our fusion method is able to increase spatial resolution and reduce spectral distortion of the fused image at the same time.

2.

The Proposed Image Fusion Method

2.4.

Outline of Proposed Image Fusion Method

The flowchart of our proposed image fusion method is presented in its entirety as Fig. 1.

Fig. 1

Flowchart of the proposed image fusion method.

3.

Experimental Results

To validate the efficiency of our proposed image fusion method, we conducted the tests on QuickBird images. The Pan image has a typical resolution of $0.7\mathrm{m}$ while the MS image has a resolution of $2.8\mathrm{m}$ . Figure 2a shows the Pan image, and Fig. 2b shows the original MS image of the blue, green, and red bands. The MS image was registered to the Pan image and resampled to $0.7\mathrm{m}$ .

Fig. 2

QuickBird images: (a) Pan image, (b) MS image.

In our fusion method, Pan and MS images are transformed by 5-scale and 4-direction MCT. The filters should be short to keep the computational burden low and avoid smearing image details; therefore, the 5/3 biorthogonal filters are used for LP and the directional decomposition. Because the size of HF subbands decreases with the increase of scale, the size of window $W$ used to compute LAG is $8\times 8$ at the first scale, $4\times 4$ at the second scale, and $2\times 2$ at the other scales. In addition, $T_{s,d}=100$ for each subband at all HF scales in our experiments for simplification.

From the results of the proposed image fusion method presented in Fig. 3b, the fused image contains structural details of the Pan image’s higher spatial resolution and rich spectral information from the MS images. Moreover, compared with the results of image fusion obtained by wavelet shown as Fig. 3a, the results of the proposed image fusion method have better visual effect.

Fig. 3

Comparison of experimental results: (a) wavelet-based fusion method, (b) proposed fusion method.

In addition to visual analysis, we conducted a quantitative analysis. Our analysis of the experimental results is based on the two factors: the standard deviation (SD) and the relative average spectral error (RASE).⁵

Using the two factors, Table 1 compares the experimental results of image fusion for the proposed method, the wavelet-based method, and the IHS method.

Table 1

Comparison of image fusion methods by IHS, wavelet, and contourlet.

Experimental results show that the proposed method of image fusion provides more detailed spatial information than the wavelet-based image fusion method and, simultaneously, preserves the richer spectral content of the MS image than both IHS and the wavelet-base image fusion methods.

4.

Conclusions

We have presented a new method of image fusion based on the concept of LAG by using contourlet transform. Because it represents edges and texture better than wavelet, the contourlet is well suited for image fusion to achieve a fused image with the high spatial and spectral resolution. Based on experimental results, the proposed method provides better visual and quantitative results for remote sensing fusion.