Fusion of Hyperspectral and Multispectral Images Based on a Centralized Non-local Sparsity Model of Abundance Maps

Fusión de imágenes hiperespectrales y multiespectrales basado en un modelo de escacez centralizado no local de mapas de abundancias

  • Edwin Vargas Universidad Industrial de Santander. https://orcid.org/0000-0002-7979-9497
  • Kevin Arias Universidad Industrial de Santander
  • Fernando Rojas Universidad Industrial de Santander
  • Henry Arguello Universidad Industrial de Santander.
Palabras clave: Image fusion, dictionary learning, non-local sparse representation, spectral unmixing, abundance maps (en_US)
Palabras clave: Fusión de imágenes, aprendizaje de diccionarios, representación escasa no-local, desmezclado espectral, mapas de abundancias (es_ES)

Resumen (en_US)

Objective: Hyperspectral (HS) imaging systems are commonly used in a diverse range of applications that involve detection and classification tasks. However, the low spatial resolution of hyperspectral images may limit the performance of the involved tasks in such applications. In the last years, fusing the information of an HS image with high spatial resolution multispectral (MS) or panchromatic (PAN) images has been widely studied to enhance the spatial resolution. Image fusion has been formulated as an inverse problem whose solution is an HS image which assumed to be sparse in an analytic or learned dictionary. This work proposes a non-local centralized sparse representation model on a set of learned dictionaries in order to regularize the conventional fusion problem.
Methodology: The dictionaries are learned from the estimated abundance data taking advantage of the depth correlation between abundance maps and the non-local self- similarity over the spatial domain. Then, conditionally on these dictionaries, the fusion problem is solved by an alternating iterative numerical algorithm.
Results: Experimental results with real data show that the proposed method outperforms the state-of-the-art methods under different quantitative assessments.
Conclusions: In this work, we propose a hyperspectral and multispectral image fusion method based on a non-local centralized sparse representation on abundance maps. This model allows us to include the non-local redundancy of abundance maps in the fusion problem using spectral unmixing and improve the performance of the sparsity-based fusion approaches.

Resumen (es_ES)

Objetivo: Sistemas de adquisición de imagen hiperespectral (HS) son comúnmente usados en un rango diverso de aplicaciones que involucran tareas de detección y clasificación. Sin embargo, la baja resolución de imágenes hiperespectrales podría limitar el rendimiento de las tareas implicadas en dichas aplicaciones. En los últimos años, fusionar la información de una imagen HS con imágenes multiespectrales (MS) o pancromáticas (PAN) de alta resolución espacial ha sido ampliamente usado para mejorar la resolución espacial. La fusión de imágenes ha sido formulada como un problema inverso cuya solución es una imagen HS, la cual se asume escasa en un diccionario analítico o aprendido. Este trabajo propone un modelo de representación escasa centralizado no local sobre un conjunto de diccionarios aprendidos para regularizar el problema de fusión convencional.
Metodología: Los diccionarios son aprendidos a partir los mapas de abundancia estimados para explotar la correlación entre mapas de abundancia y la auto-similitud no local sobre el dominio espacial. Luego, condicionalmente sobre los diccionarios aprendidos, el problema de fusión es solucionado por un algoritmo numérico iterativo y alternante.
Resultados: Los resultados experimentales usando datos reales muestra que el método propuesto supera los métodos del estado del arte bajo diferentes métricas cuantitativas. Conclusiones: En este trabajo nosotros proponemos un método de fusion de imágenes espectrales y multiespectrales basado en un representación no-local centralizada escasa en mapas de abundancias. Este modelo permite incluir la redundancia no local en el problema de fusion usando desmezclado espectral y mejorar los resultados de los métodos de fusión basados en modelo de escaces.

Descargas

La descarga de datos todavía no está disponible.

Biografía del autor/a

Edwin Vargas, Universidad Industrial de Santander.

Ingeniero Electrónico, magister en Ingeniería Electrónica. Estudiante de doctorado en Ingeniería de la Universidad Industrial de Santander. Bucaramanga

Kevin Arias, Universidad Industrial de Santander

Ingeniero de Sistemas e Informática. Estudiante de maestría en Matemática Aplicada de la Universidad Industrial de Santander. Bucaramanga, Colombia

Fernando Rojas, Universidad Industrial de Santander

Ingeniero de Sistemas e Informática, magister en ciencias computacionales. Profesor de la Universidad Industrial de Santander. Bucaramanga, Colombia

Henry Arguello, Universidad Industrial de Santander.

Ingeniero Eléctrico, magister en Ingeniería Eléctrica, Doctor en Ingeniería Eléctrica y Computación. Profesor de la Universidad Industrial de Santander. Bucaramanga, Colombia

Referencias

Arias, K., Vargas, E., & Arguello, H. (2019, September). Hyperspectral and Multispectral Image Fusion based on a Non-locally Centralized Sparse Model and Adaptive Spatial-Spectral Dictionaries. In 2019 27th European Signal Processing Conference (EUSIPCO) (pp. 1-5). IEEE. https://doi.org/10.23919/EUSIPCO.2019.8903001

Beck, A., & Teboulle, M. (2009). A fast-iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2(1), 183-202. https://doi.org/10.1137/080716542

Bioucas-Dias, J. M., Plaza, A., Camps-Valls, G., Scheunders, P., Nasrabadi, N., & Chanussot, J. (2013). Hyperspectral remote sensing data analysis and future challenges. IEEE Geoscience and remote sensing magazine, 1(2), 6-36. https://doi.org/10.1109/MGRS.2013.2244672

Bruckstein, A. M., Donoho, D. L., & Elad, M. (2009). From sparse solutions of systems of equations to sparse modeling of signals and images. SIAM review, 51(1), 34-81. https://doi.org/10.1137/060657704

Chakrabarti, A., & Zickler, T. (2011, June). Statistics of real-world hyperspectral images. In CVPR 2011 (pp. 193-200). IEEE.

https://doi.org/10.1109/CVPR.2011.5995660

Chan, T., Esedoglu, S., Park, F., & Yip, A. (2005). Recent developments in total variation image restoration. Mathematical Models of Computer Vision, 17(2).

Chen, S. S., Donoho, D. L., & Saunders, M. A. (2001). Atomic decomposition by basis pursuit. SIAM review, 43(1), 129-159. https://doi.org/10.1137/S003614450037906X

Daubechies, I., Defrise, M., & De Mol, C. (2004). An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications on Pure and Applied Mathematics: A Journal Issued by the Courant Institute of Mathematical Sciences, 57(11), 1413-1457. https://doi.org/10.1002/cpa.20042

Dian, R., Fang, L., & Li, S. (2017). Hyperspectral image super-resolution via non-local sparse tensor factorization. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 5344-5353). https://doi.org/10.1109/CVPR.2017.411

Dong, W., Zhang, L., Shi, G., & Wu, X. (2011). Image deblurring and super-resolution by adaptive sparse domain selection and adaptive regularization. IEEE Trans. on image process., 20(7), 1838-1857. https://doi.org/10.1109/TIP.2011.2108306

Dong, W., Li, X., Zhang, L., & Shi, G. (2011, June). Sparsity-based image denoising via dictionary learning and structural clustering. In CVPR 2011 (pp. 457-464). IEEE. https://doi.org/10.1109/CVPR.2011.5995478

Dong, W., Zhang, L., & Shi, G. (2011, November). Centralized sparse representation for image restoration. In 2011 IEEE ICCV (pp. 1259-1266). https://doi.org/10.1109/ICCV.2011.6126377

Dong, W., Zhang, L., Shi, G., & Li, X. (2012). Nonlocally centralized sparse representation for image restoration. IEEE transactions on Image Processing, 22(4), 1620-1630. https://doi.org/10.1109/TIP.2012.2235847

Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image processing, 15(12), 3736-3745. https://doi.org/10.1109/TIP.2006.881969

Cardon, H. D. V., Álvarez, M. A., & Gutiérrez, Á. O. (2015). Representación óptima de señales MER aplicada a la identificación de estructuras cerebrales durante la estimulación cerebral profunda. Tecnura, 19(45), 15-27. https://doi.org/10.14483/udistrital.jour.tecnura.2015.3.a01

Medina Rojas, F., Arguello Fuentes, H., & Gómez Santamaría, C. (2017). A quantitative and qualitative performance analysis of compressive spectral imagers. Tecnura, 21(52), 53-67. https://doi.org/10.14483/udistrital.jour.tecnura.2017.2.a04

Velasco, A. C., García, C. A. V., & Fuentes, H. A. (2016). Un estudio comparativo de algoritmos de detección de objetivos en imágenes hiperespectrales aplicados a cultivos agrícolas en Colombia. Revista Tecnura, 20(49), 86-100.

Fu, Y., Lam, A., Sato, I., & Sato, Y. (2017). Adaptive spatial-spectral dictionary learning for hyperspectral image restoration. International Journal of Computer Vision, 122(2), 228-245. https://doi.org/10.1007/s11263-016-0921-6

Ghasrodashti, E. K., Karami, A., Heylen, R., & Scheunders, P. (2017). Spatial resolution enhancement of hyperspectral images using spectral unmixing and bayesian sparse representation. Remote Sensing, 9(6), 541. https://doi.org/10.3390/rs9060541

Hege, E. K., O'Connell, D., Johnson, W., Basty, S., & Dereniak, E. L. (2004, January). Hyperspectral imaging for astronomy and space surveillance. In Imaging Spectrometry IX (Vol. 5159, pp. 380-391). International Society for Optics and Photonics. https://doi.org/10.1117/12.506426

Lu, G., & Fei, B. (2014). Medical hyperspectral imaging: a review. Journal of biomedical optics, 19(1), 010901. https://doi.org/10.1117/1.JBO.19.1.010901

Loncan, L., De Almeida, L. B., Bioucas-Dias, J. M., Briottet, X., Chanussot, J., Dobigeon, N., ... & Tourneret, J. Y. (2015). Hyperspectral pansharpening: A review. IEEE Geoscience and remote sensing magazine, 3(3), 27-46. https://doi.org/10.1109/MGRS.2015.2440094

Mallat, S. (1999). A wavelet tour of signal processing. Elsevier. https://doi.org/10.1016/B978-012466606-1/50008-8

Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2009, September). Non-local sparse models for image restoration. In 2009 IEEE 12th ICCV (pp. 2272-2279) https://doi.org/10.1109/ICCV.2009.5459452

Middleton, E. M., Ungar, S. G., Mandl, D. J., Ong, L., Frye, S. W., Campbell, P. E., ... & Pollack, N. H. (2013). The earth observing one (EO-1) satellite mission: Over a decade in space. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 6(2), 243-256. https://doi.org/10.1109/JSTARS.2013.2249496

Oliveira, J. P., Bioucas-Dias, J. M., & Figueiredo, M. A. (2009). Adaptive total variation image deblurring: a majorization–minimization approach. Signal processing, 1683-1693. https://doi.org/10.1016/j.sigpro.2009.03.018

Panasyuk, S. V., Yang, S., Faller, D. V., Ngo, D., Lew, R. A., Freeman, J. E., & Rogers, A. E. (2007). Medical hyperspectral imaging to facilitate residual tumor identification during surgery. Cancer biology & therapy, 6(3), 439-446. https://doi.org/10.4161/cbt.6.3.4018

Simoes, M., Bioucas‐Dias, J., Almeida, L. B., & Chanussot, J. (2014). A convex formulation for hyperspectral image super-resolution via subspace-based regularization. IEEE Transactions on Geoscience and Remote Sensing, 53(6), 3373-3388. https://doi.org/10.1109/TGRS.2014.2375320

Schaepman, M. E., Ustin, S. L., Plaza, A. J., Painter, T. H., Verrelst, J., & Liang, S. (2009). Earth system science related imaging spectroscopy—An assessment. Remote Sensing of Environment, 113, S123-S137. https://doi.org/10.1016/j.rse.2009.03.001

Schreiber, N. F., Genzel, R., Bouché, N., Cresci, G., Davies, R., Buschkamp, P., ... & Shapley, A. E. (2009). The SINS survey: SINFONI integral field spectroscopy of z∼ 2 star-forming galaxies. The Astrophysical Journal, 706(2), 1364. https://doi.org/10.1088/0004-637X/706/2/1364

Shaw, G. A., & Burke, H. K. (2003). Spectral imaging for remote sensing. Lincoln laboratory journal, 14(1), 3-28.

Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society: Series B (Methodological), 58(1), 267-288. https://doi.org/10.1111/j.2517-6161.1996.tb02080.x

Tosic, I., & Frossard, P. (2011). Dictionary learning. IEEE Signal Processing Magazine, 28(2), 27-38. https://doi.org/10.1109/MSP.2010.939537

Tropp, J. A., & Wright, S. J. (2010). Computational methods for sparse solution of linear inverse problems. Proceedings of the IEEE, 98(6), 948-958. https://doi.org/10.1109/JPROC.2010.2044010

Vargas, E., Espitia, O., Arguello, H., & Tourneret, J. Y. (2018). Spectral image fusion from compressive measurements. IEEE Transactions on Image Processing, 28(5), 2271-2282. https://doi.org/10.1109/TIP.2018.2884081

Vargas, E., Arguello, H., & Tourneret, J. Y. (2019). Spectral image fusion from compressive measurements using spectral unmixing and a sparse representation of abundance maps. IEEE Transactions on Geoscience and Remote Sensing, 57(7), 5043-5053. https://doi.org/10.1109/TGRS.2019.2895822

Wei, Q., Bioucas-Dias, J., Dobigeon, N., & Tourneret, J. Y. (2015). Hyperspectral and multispectral image fusion based on a sparse representation. IEEE Transactions on Geoscience and Remote Sensing, 53(7), 3658-3668. https://doi.org/10.1109/TGRS.2014.2381272

Wei, Q., Dobigeon, N., & Tourneret, J. Y. (2015). Bayesian fusion of multi-band images. IEEE Journal of Selected Topics in Signal Processing, 9(6), 1117-1127. https://doi.org/10.1109/JSTSP.2015.2407855

Wei, Q., Bioucas-Dias, J., Dobigeon, N., Tourneret, J. Y., Chen, M., & Godsill, S. (2016). Multiband image fusion based on spectral unmixing. IEEE Transactions on Geoscience and Remote Sensing, 54(12), 7236-7249. https://doi.org/10.1109/TGRS.2016.2598784

Yang, J., Wright, J., Huang, T. S., & Ma, Y. (2010). Image super-resolution via sparse representation. IEEE transactions on image processing, 19(11), 2861-2873. https://doi.org/10.1109/TIP.2010.2050625

Yin, H., Li, S., & Fang, L. (2013). Simultaneous image fusion and super-resolution using sparse representation. Information Fusion, 14(3), 229-240. https://doi.org/10.1016/j.inffus.2012.01.008

Zhao, Y., Yang, J., & Chan, J. C. W. (2013). Hyperspectral imagery super-resolution by spatial–spectral joint non-local similarity. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(6), 2671-2679. https://doi.org/10.1109/JSTARS.2013.2292824

Zhao, Y., Yi, C., Yang, J., & Chan, J. C. W. (2014, July). Coupled hyperspectral super-resolution and unmixing. In 2014 IEEE Geoscience and Remote Sensing Symposium (pp. 2641-2644). IEEE.

Cómo citar
Vargas, E., Arias, K., Rojas, F., & Arguello, H. (2020). Fusión de imágenes hiperespectrales y multiespectrales basado en un modelo de escacez centralizado no local de mapas de abundancias. Tecnura, 24(66), 62 - 75. https://doi.org/10.14483/22487638.16904
Publicado: 2020-10-01
Sección
Investigación