基于多模态视觉感知的变电所亭违规作业智能识别模型构建

Construction of an Intelligent Identification Model for Illegal Operations in Substation Booths Based on Multimodal Visual Perception

  • 摘要: 基于可见光图像、红外热成像与深度图像三类模态信息,构建基于多模态视觉感知的变电所亭违规作业智能识别模型,实现复杂光照与气象条件下的全时段作业行为监控。以不戴安全帽、人员闯入危险区域、违规操作设备、高处作业未系安全带4类典型违规作业为识别对象,构建包含28 460张多模态图像的标注数据集。在YOLOv11s目标检测框架的基础上,设计可见光—红外—深度三分支特征提取与跨模态注意力融合模块,强化模型在光照不足、雨雾、遮挡等恶劣工况下的特征表达能力,使模型mAP50达到0.812。采用ONNX格式导出与INT8静态量化技术,在ARM架构边缘计算设备上将模型体积压缩56%,单帧推理时间降至78.4ms。实验结果表明,该模型在保持较高检测精度的同时能够显著降低资源消耗,可有效部署于变电所亭边缘计算终端,满足全天候、实时、精准的违规作业智能识别与预警需求,为电力安全生产的智能化转型提供可行的技术方案。

     

    Abstract: Based on three modalities of visible light images, infrared thermal imaging, and depth images, an intelligent identification model for abnormal operations in substation booths is constructed using multi-modal visual perception, enabling real-time monitoring of all-time operation behaviors under complex lighting and weather conditions. The model targets four typical types of abnormal operations: not wearing safety helmets, personnel entering dangerous areas, operating equipment illegally, and performing high-altitude work without a safety belt. A labeled dataset containing 28 460 multi-modal images is constructed. Based on the YOLOv11s object detection framework, a feature extraction and cross-modal attention fusion module for the visible light-infrared-depth three branches is designed to enhance the model's feature expression ability in adverse conditions such as insufficient lighting, rain and fog, and occlusion, achieving an mAP50 of 0.812. The model is exported in ONNX format and quantized with INT8 static method, reducing the model size by 56% on ARM-based edge computing devices and lowering the single-frame inference time to 78.4ms. Experimental results show that this model can significantly reduce resource consumption while maintaining high detection accuracy, and can be effectively deployed on edge computing terminals of substation booths, meeting the requirements for intelligent identification and early warning of abnormal operations in all-weather, real-time, and precise scenarios, providing a feasible technical solution for the intelligent transformation of power safety production.

     

/

返回文章
返回