DOI:

https://doi.org/10.14483/23448393.18145

Published:

2022-11-20

Issue:

Vol. 28 No. 1 (2023): January-April

Section:

Systems Engineering

Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection

Herramienta software para la calibración extrínseca de cámaras infrarrojas y RGBD aplicada a inspección termográfica

Authors

  • Jaimen Junior Aza Taimal Universidad del Valle
  • Bladimir Bacca Cortes Universidad del Valle
  • Andrés David Restrepo Girón Universidad del Valle https://orcid.org/0000-0001-7891-1064

Keywords:

Inspección termográfica, calibración extrínseca, fusión de imágenes, imagen térmica, RGBD (es).

Keywords:

thermographic inspection, extrinsic calibration, image fusion, thermal image, RGBD (en).

Downloads

Abstract (en)

Context:  Thermographic inspections are currently used to assess energy efficiency in electrical equipment and civil structures or to detect failures in cooling systems and electrical or electronic devices. However, thermal images lack texture details, which does not allow for a precise identification of the geometry of the scene or the objects in it.

Method: In this work, the development of the software tool called DepTherm is described. This tool allows performing intrinsic and extrinsic calibration between infrared and RGBD cameras in order to fuse thermal, RGB, and RGBD images, as well as to record thermal and depth data. Additional features include user management, a visualization GUI for all three types of images, database storage, and report generation.

Results: In addition to the integration tests performed to validate the functionality of DepTherm, two quantitative tests were conducted in order to evaluate its accuracy. A maximum re-projection error of 1,47±0,64 pixels was found, and the maximum mean error in registering an 11 cm side cube was 4,15 mm.

Conclusions: The features of the DepTherm software tool are focused on facilitating thermographic inspections by capturing 3D scene models with thermal data.

Abstract (es)

Contexto: Las inspecciones termográficas se utilizan en la actualidad para evaluar la eficiencia energética de equipos eléctricos y estructuras civiles o para detectar fallas en sistemas de enfriamiento y dispositivos eléctricos o electrónicos. Sin embargo, las imágenes térmicas carecen de detalles de textura, lo cual no permite identificar con precisión la geometría de la escena ni los objetos en ella.

Método: En este trabajo se describe el desarrollo de la herramienta de software DepTherm, la cual permite realizar calibraciones intrínsecas y extrínsecas entre cámaras infrarrojas y RGBD para fusionar imágenes térmicas, RGB y RGBD, así como para registrar datos térmicos y de profundidad. Funcionalidades adicionales incluyen el manejo de usuarios, una GUI para visualización de los tres tipos de imágenes, el almacenamiento en una base de datos y la generación de reportes.

Resultados: Además de las pruebas de integración para validar la funcionalidad de DepTherm, se realizaron dos pruebas cuantitativas para evaluar su precisión. Se encontró un error máximo de reproyección de 1,47±0,64 pixeles, mientras que el registro de un cubo con 11 cm de lado tuvo un error promedio máximo de 4,147 mm.

Conclusiones: Las funcionalidades de la herramienta software DepTherm están enfocadas en facilitar las inspecciones termográficas capturando modelos 3D de las escenas con información térmica.

Author Biographies

Jaimen Junior Aza Taimal, Universidad del Valle

He got the Electronic Engineer degree on 2021, at the Universidad del Valle, Cali, Colombia. Nowadays, he works in software development focused on computer vision. He is interested in computer vision, frontend and backend development, and embedded systems.

Andrés David Restrepo Girón, Universidad del Valle

Doctor in Engineering of the Universidad del Valle (2014); Magister in Automation of the Universidad del Valle (2005); and Electronic Engineer of the same University (1999); nowadays, Andrés Restrepo Girón is Associate Professor at Universidad del Valle. He belongs to the Perception and Intelligent Systems research group, of the Universidad del Valle. His research interests include Electronic design, electronic instrumentation, digital signal processing and thermography

References

S. Zhao, Z. Fang, and S. Wen, “A real-time handheld 3D temperature field reconstruction system,” in 2017 IEEE 7th Annual Int. Conf. CYBER Tech. Auto. Cont. Int. Syst., 2017, pp. 289-294. https://doi.org/10.1109/CYBER.2017.8446193

X. Li, M. Ding, D. Wei, X. Wu, and Y. Cao, “Estimate depth information from monocular infrared images based on deep learning,” in 2020 IEEE Int. Conf. Prog. Info. Comp. (PIC), 2020, pp. 149-153. https://doi.org/10.1109/PIC50277.2020.9350792

Z. Zhang, “A flexible new technique for camera calibration,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 22, no. 11, pp. 1330-1334, 2000. https://doi.org/10.1109/34.888718

J.-Y. Bouguet, “Camera calibration toolbox for Matlab,” California Institute of Technology, 2013. [Online]. Available: http://www.vision.caltech.edu/bouguetj/calib_doc/

J. T. Lussier and S. Thrun, “Automatic calibration of RGBD and thermal cameras,” in 2014 IEEE/RSJ Int. Conf. Intell. Rob. Syst-, 2014, pp. 451-458. https://doi.org/10.1109/IROS.2014.6942598

S. Sels et al., “A CAD matching method for 3D thermography of complex objects,” Infrared Phys. Technol., vol. 99, pp. 152-157, Jun. 2019. https://doi.org/10.1016/j.infrared.2019.04.014

P. Aksenov et al., “3D thermography for quantification of heat generation resulting from inflammation,” 2003. [Online]. Available: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.13.4874&rep=rep1&type=pdf

T. P. Truong, M. Yamaguchi, S. Mori, V. Nozick, and H. Saito, “Registration of RGB and thermal point clouds generated by structure from motion,” in 2017 IEEE Int. Conf. Comp. Vision Workshops (ICCVW), 2017, pp. 419-427. https://doi.org/10.1109/ICCVW.2017.57

A. Chromy and O. Klima, “A 3D scan model and thermal image data fusion algorithms for 3D thermography in medicine,” J. Healthc. Eng., vol. 2017, pp. 1-9, 2017. https://doi.org/10.1155/2017/5134021

D. Borrmann et al., “The project ThermalMapper – Thermal 3D mapping of indoor environments for saving energy,” IFAC Proc., vol. 45, no. 22, pp. 31-38, Jan. 2012. https://doi.org/10.3182/20120905-3-HR-2030.00045

C. Wang, “Point clouds and thermal data fusion for automated gbXML-based building geometry model generation,” PhD dissertation, Sch. Civil Environ. ENg., Georgia I. Tech., Atlanta, GA, USA, 2014. [Online]. Available: http://hdl.handle.net/1853/54008

S. Vidas, P. Moghadam, and M. Bosse, “3D thermal mapping of building interiors using an RGB-D and thermal camera,” 2013 IEEE Int. Conf. Robot. Autom., pp. 2311-2318, May 2013. https://doi.org/10.1109/ICRA.2013.6630890

C. Yanpeng et al., “Depth and thermal sensor fusion to enhance 3D thermographic reconstruction,” Opt. Express, vol. 26, no. 7, pp. 8179-8193, 2018. https://doi.org/10.1364/OE.26.008179

P. Moghadam and S. Vidas, “HeatWave: the next generation of thermography devices,” in Proc. SPIE 9105, Thermosense: Thermal Infrared Applications XXXVI, 2014, p. 8. https://doi.org/10.1117/12.2053950

S. Karolj, L. TOMISLAV, S. Ivan, L. Jgenero, and G. Ivan, “4D thermal imaging system for medical applications,” Period. Biol., vol. 113, no. 4, pp. 407-416, 2011.

J. Rangel, M. Soldan, and A. Kroll, “3D thermal imaging: Fusion of thermography and depth cameras,” in 12th Int. Conf. Quant. Infrared Thermog., 2015, pp. 1-10. https://doi.org/10.21611/qirt.2014.035

S. Vidas, P. Moghadam, and S. Sridharan, “Real-time mobile 3D temperature mapping,” IEEE Sens. J., vol. 15, no. 2, pp. 1145-1152, Feb. 2015. https://doi.org/10.1109/JSEN.2014.2360709

G. Chernov, V. Chernov, and M. Barboza Flores, “3D dynamic thermography system for biomedical applications,” in Application of Infrared to Biomedical Sciences, E. Ng and M. Etehadtavakol, eds., Singapore: Springer, 2017, pp. 517-545. https://doi.org/10.1007/978-981-10-3147-2_28

Y. Shi, P. Payeur, M. Frize, and E. Bariciak, “Thermal and RGB-D imaging for necrotizing enterocolitis detection,” in 2020 IEEE Int. Symp. Medical Meas. App. (MeMeA), 2020, pp. 1-6. https://doi.org/10.1109/MeMeA49120.2020.9137344

T. Zhang, L. Hu, L. Li, and D. Navarro-Alarcón, “Towards a multispectral RGB-IR-UV-D vision system – Seeing the invisible in 3D,” presented at 2021 IEEE Int. Conf. Robotics Biomimetics (ROBIO), Sanya, China, December 27-31, 2021. https://doi.org/10.1109/ROBIO54168.2021.9739218

S. Schramm, P. Osterhold, R. Schmoll, and A. Kroll, “Combining modern 3D reconstruction and thermal imaging: generation of large-scale 3D thermograms in real-time,” Quant. Infrared Thermogr. J., pp. 1-17, Oct. 2021. https://doi.org/10.1080/17686733.2021.1991746

J. D. Choi and M. Y. Kim, “A sensor fusion system with thermal infrared camera and LiDAR for autonomous vehicles and deep learning based object detection,” ICT Express, 2022. [Online]. Available: https://doi.org/10.1016/j.icte.2021.12.016

R. E. Ospina, S. D. Cardona, and B. Bacca-Cortés, “Software tool for thermographic inspection using multimodal fusing of thermal and visible images,” Ing. y Compet., vol. 19, no. 1, pp. 50-65, 2017. https://doi.org/10.25100/iyc.v19i1.2130

OpenCV, “OpenCV,” OpenCV Python, 2021. [Online]. Available: https://pypi.org/project/opencv-python/4.1.2.30/

C.-T. Hsieh, “An efficient development of 3D surface registration by Point Cloud Library (PCL),” in 2012 Int. Symp. Intel. Signal Proc. Comm. Syst., 2012, pp. 729-734. https://doi.org/10.1109/ISPACS.2012.6473587

Kitware, “VTK – The Visualization Toolkit,” 2021. [Online]. Available: https://vtk.org/

P. Kruchten, The Rational Unified Process: An Introduction, 3rd ed., Boston, MA, USA: Addison-Wesley Professional, 2003.

A. Zisserman and R. Hartley, Multiple view geometry in computer vision, 2nd ed., Cambridge, UK: Cambridge University Press, 2004.

Cloudcompare.org, “CloudCompare,” 2021. [Online]. Available: https://www.danielgm.net/cc/

W. Nakagawa et al., “Visualization of temperature change using RGB-D camera and thermal camera,” L. Agapito, M. Bronstein, C. Rother, eds., Cham, Germany: Springer, 2015, pp. 386-400. https://doi.org/10.1007/978-3-319-16178-5_27

How to Cite

APA

Aza Taimal, J. J. ., Bacca Cortes, B., & Restrepo Girón, A. D. (2022). Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection. Ingeniería, 28(1), e18145. https://doi.org/10.14483/23448393.18145

ACM

[1]
Aza Taimal, J.J. , Bacca Cortes, B. and Restrepo Girón, A.D. 2022. Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection. Ingeniería. 28, 1 (Nov. 2022), e18145. DOI:https://doi.org/10.14483/23448393.18145.

ACS

(1)
Aza Taimal, J. J. .; Bacca Cortes, B.; Restrepo Girón, A. D. Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection. Ing. 2022, 28, e18145.

ABNT

AZA TAIMAL, J. J. .; BACCA CORTES, B.; RESTREPO GIRÓN, A. D. Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection. Ingeniería, [S. l.], v. 28, n. 1, p. e18145, 2022. DOI: 10.14483/23448393.18145. Disponível em: https://revistas.udistrital.edu.co/index.php/reving/article/view/18145. Acesso em: 7 dec. 2022.

Chicago

Aza Taimal, Jaimen Junior, Bladimir Bacca Cortes, and Andrés David Restrepo Girón. 2022. “Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection”. Ingeniería 28 (1):e18145. https://doi.org/10.14483/23448393.18145.

Harvard

Aza Taimal, J. J. ., Bacca Cortes, B. and Restrepo Girón, A. D. (2022) “Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection”, Ingeniería, 28(1), p. e18145. doi: 10.14483/23448393.18145.

IEEE

[1]
J. J. . Aza Taimal, B. Bacca Cortes, and A. D. Restrepo Girón, “Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection”, Ing., vol. 28, no. 1, p. e18145, Nov. 2022.

MLA

Aza Taimal, J. J. ., B. Bacca Cortes, and A. D. Restrepo Girón. “Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection”. Ingeniería, vol. 28, no. 1, Nov. 2022, p. e18145, doi:10.14483/23448393.18145.

Turabian

Aza Taimal, Jaimen Junior, Bladimir Bacca Cortes, and Andrés David Restrepo Girón. “Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection”. Ingeniería 28, no. 1 (November 20, 2022): e18145. Accessed December 7, 2022. https://revistas.udistrital.edu.co/index.php/reving/article/view/18145.

Vancouver

1.
Aza Taimal JJ, Bacca Cortes B, Restrepo Girón AD. Software Tool for the Extrinsic Calibration of Infrared and Rgbd Cameras Applied to Thermographic Inspection. Ing. [Internet]. 2022Nov.20 [cited 2022Dec.7];28(1):e18145. Available from: https://revistas.udistrital.edu.co/index.php/reving/article/view/18145

Download Citation

Visitas

16

Dimensions


PlumX


Downloads

Download data is not yet available.