Abstract
Abstract— Due to hardware limitations of the imaging sensors,
it is challenging to acquire images of high resolution in both
spatial and spectral domains. Fusing a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral
image (HR-MSI) to obtain an HR-HSI in an unsupervised
manner has drawn considerable attention. Though effective, most
existing fusion methods are limited due to the use of linear
parametric modeling for the spectral mixture process, and even
the deep learning-based methods only focus on deterministic
fully-connected networks without exploiting the spatial correlation and local spectral structures of the images. In this paper,
we propose a novel variational probabilistic autoencoder framework implemented by convolutional neural networks, in order to
fuse the spatial and spectral information contained in the LR-HSI
and HR-MSI, called FusionNet. The FusionNet consists of a
spectral generative network, a spatial-dependent prior network,
and a spatial-spectral variational inference network, which are
jointly optimized in an unsupervised manner, leading to an endto-end fusion system. Further, for fast adaptation to different
observation scenes, we give a meta-learning explanation to the
fusion problem, and combine the FusionNet with meta-learning in
a synergistic manner. Effectiveness and efficiency of the proposed
method are evaluated based on several publicly available datasets,
demonstrating that the proposed FusionNet outperforms the
state-of-the-art fusion methods.
Index Terms— Hyperspectral images, multispectral images,
image fusion, probabilistic generative model, convolutional neural
network, meta-learning.
Modules
Algorithms
Software And Hardware
• Hardware: Processor: i3 ,i5 RAM: 4GB Hard disk: 16 GB • Software: operating System : Windws2000/XP/7/8/10 Anaconda,jupyter,spyder,flask Frontend :-python Backend:- MYSQL