Abstract:
Based on three modalities of visible light images, infrared thermal imaging, and depth images, an intelligent identification model for abnormal operations in substation booths is constructed using multi-modal visual perception, enabling real-time monitoring of all-time operation behaviors under complex lighting and weather conditions. The model targets four typical types of abnormal operations: not wearing safety helmets, personnel entering dangerous areas, operating equipment illegally, and performing high-altitude work without a safety belt. A labeled dataset containing 28 460 multi-modal images is constructed. Based on the YOLOv11s object detection framework, a feature extraction and cross-modal attention fusion module for the visible light-infrared-depth three branches is designed to enhance the model's feature expression ability in adverse conditions such as insufficient lighting, rain and fog, and occlusion, achieving an mAP50 of 0.812. The model is exported in ONNX format and quantized with INT8 static method, reducing the model size by 56% on ARM-based edge computing devices and lowering the single-frame inference time to 78.4ms. Experimental results show that this model can significantly reduce resource consumption while maintaining high detection accuracy, and can be effectively deployed on edge computing terminals of substation booths, meeting the requirements for intelligent identification and early warning of abnormal operations in all-weather, real-time, and precise scenarios, providing a feasible technical solution for the intelligent transformation of power safety production.