A knowledge-based neural network for fusing edge maps of multi-sensor images

Abstract

With the goal of fusion prescribed as building an edge map that contains as many edges as possible from the given multi-spectral/sensor images, a new fusion scheme, called the knowledge-based neural network fusion (KBNNF), is proposed to fuse edge maps of these images in order to generate a combined edge map that has more complete and reliable edge information than what one can obtain from any single image. The KBNNF is used to fuse edge maps of images having mutually complementary edge information in the following sense: (i) the edges in the images are compatible, i.e., can be interpreted together; and (ii) the edges in the different images reveal different parts of the scene. More complete edge contours of the same object are obtained by linking the edge sections obtained from different images together. The resulting edge map can be used for subsequent study (like object recognition). The proposed scheme bases its confidence and reliability on the analysis of variance (ANOVA)-based edge detector that can address two important issues of edge based image fusion well: (i) the difference in edge position among the images because of the different characteristics of the images and the error in the image registration process; and (ii) the variance existing among the edge test values calculated from different images. The KBNNF has been applied to fuse: (i) radar (SAR)-optical (SPOT), (ii) optical-optical, (iii) infrared-infrared, and (iv) optical-infrared (satellite) image combinations. Comparisons are made with the relevant existing techniques in the literature. The paper concludes with some examples to illustrate the efficacy of the proposed scheme.