Smart vision-based monitoring system for heavy construction and surface mining jobsites

与其他工业和服务流程一样，建筑业务需要实时反馈系统来衡量关键绩效指标，如生产率，并采取措施纠正干扰高效运营的问题。目前，自动化系统广泛应用于许多行业和服务，并大大提高了生产率。然而，建筑行业在采用自动化方面进展缓慢。为了开发有效的自动化反馈系统，研究人员测试了不同的传感技术，以监测建筑工地独特的恶劣环境中的工人和设备。土方工程，如公路建设和露天采矿作业，一直是工业开发自动传感系统的主要重点。然而，在动态环境中监测不同类型的设备是一项挑战；系统面临着理解建筑工地环境的挑战。例如，液压挖掘机可以采取任意数量的形状，这使软件对设备动作进行分类的能力变得紧张。低成本的数码相机与计算机视觉技术的进步相结合，使得基于视觉的系统有可能成为建筑作业解决方案的候选者。然而，目前使用基于视觉的算法来监控重型作业的系统并不理想，需要一定程度的人为干预才能成功运行。例如，他们无法理解场景，用户必须确定要监控的动作类型；此外，用户还必须将相机的取景器设置在感兴趣的操作上。本研究计划的目标是弥合理论计算机视觉和机器学习算法与实际应用之间的差距，开发一种通用的基于智能视觉的系统，以测量建筑和露天采矿工地土方过程的生产力。除了目标识别和跟踪之外，拟议的研究计划将研究基于已识别动作的动作识别和理解场景的算法的开发，相机的平移和缩放功能的自动控制，相机网络的主动控制（包括在网络中相机之间传输跟踪的能力），生产力估计，偏差检测和开发模块的集成。最终的结果将是一个自动化框架，能够在不同类型的建筑环境中跟踪不同类型的机械。***这项研究计划可能会改变土方监测，使其成为一种基于知识的自动化实践，从而提高生产率，节省成本，提高加拿大采矿和重型民用部门的竞争力，并减少这些业务的温室气体排放。**

Construction operations, like other industrial and service processes, require real-time feedback systems to measure key performance indicators, such as productivity, and to take steps to correct problems that interfere with efficient operations. Currently, automated systems are broadly used in many industries and services and have improved productivity substantially. However, the construction industry has been slow to adopt automation. To develop effective automated feedback systems, researchers have tested different sensing technologies to monitor workers and equipment in the uniquely rugged environment of construction jobsites. Earthmoving projects, such as highway construction and surface mining operations, have been a primary initial focus in the industry for developing automated sensing systems. However, monitoring different types of equipment in a dynamic environment is a challenge; systems are challenged to understand the context of a construction site. For example, hydraulic excavators can take any number of shapes, which strains the ability of software to classify the actions of the equipment. Low-cost digital cameras, combined with promising advances in computer vision, make vision-based systems likely candidates for solutions in construction operations. The current systems using vision-based algorithms to monitor heavy operations, however, aren't ideal and require some level of human intervention to operate successfully. For instance, they are not able to understand the scene and the user must determine the action type to be monitored; also, the user has to set the viewfinder of the camera on the operation of interest. The objective of this research program is to bridge the gap between theoretical computer vision and machine learning algorithms on one hand, and practical applications on the other, to develop a generic smart vision-based system to measure the productivity of earthmoving processes in construction and surface mining jobsites. Beyond object recognition and tracking, the proposed research program will investigate development of algorithms for action recognition and understanding the scene based on identified actions, automated control of panning and zooming features of a camera, proactive control of a network of cameras (including the ability to transfer tracking among cameras in the network), productivity estimation, deviation detection, and integration of developed modules. The final result will be an automated framework capable of tracking different kinds of machinery in different types of construction environments. ***This research plan could potentially transform earthmoving monitoring, making it an automated knowledge-based practice and leading to productivity increases, cost savings, improvement in the competitiveness of Canadian mining and heavy civil sectors, and reduction of greenhouse gas emissions of these operations. **