Development of Real-Time Object Recognition System Based on Thermal-RGB Camera Fusion; 열화상-RGB 카메라 융합 기반 실시간 객체 인식 시스템

초록

This study developed a multi-image sensor fusion algorithm that utilizes the complementary characteristics of thermal and RGB cameras. To overcome the limitations of low-resolution thermal sensors (FLIR Lepton 3.5) considering economic constraints, FSRCNN-based super-resolution processing and thermal-specific YOLOv5 model retraining were performed, improving object recognition rates from 50% to 90%. A dual-judgment sensor fusion system was designed with RGB-priority detection and temperature-based verification, and homography-based registration correction techniques resolved spatiotemporal misalignment issues between the two sensors. Real-time processing performance was validated on the NVIDIA Jetson Orin Nano Super platform, confirming applicability to various practical fields including autonomous driving. The proposed algorithm provides an effective compromise between expensive multi-sensor approaches and environmentally constrained single-sensor approaches, demonstrating significant potential for expanding thermal camera applications in autonomous driving and related fields. © 2026 The Korean Society of Mechanical Engineers.

키워드