Smart vision-based monitoring system for heavy construction and surface mining jobsites

与其他工业和服务流程一样，建筑作业需要实时反馈系统来衡量生产率等关键绩效指标，并采取措施纠正干扰高效运营的问题。目前，自动化系统在许多行业和服务中得到广泛应用，并显著提高了生产率。然而，建筑业在采用自动化方面进展缓慢。为了开发有效的自动反馈系统，研究人员测试了不同的传感技术，以监控建筑工地独特崎岖环境中的工人和设备。土方工程，如公路建设和露天采矿作业，一直是该行业开发自动传感系统的主要初始重点。然而，在动态环境中监控不同类型的设备是一项挑战；系统需要了解建筑工地的环境。例如，液压挖掘机可以采取任意数量的形状，这使得软件无法对设备的动作进行分类。低成本的数码相机，再加上计算机视觉的进步，使得基于视觉的系统可能成为建筑作业解决方案的候选者。然而，目前使用基于视觉的算法来监控繁重操作的系统并不理想，需要一定程度的人工干预才能成功操作。例如，他们不能理解场景，用户必须确定要监视的动作类型；此外，用户必须将相机的取景器设置在感兴趣的操作上。这项研究的目标是一方面弥合理论计算机视觉和机器学习算法与实际应用之间的差距，另一方面开发一个通用的基于智能视觉的系统来测量建筑和露天采矿工地的土方工序的生产率。除了物体识别和跟踪，拟议的研究计划还将研究基于识别的动作识别和理解场景的算法的开发，摄像机平移和变焦功能的自动控制，摄像机网络的主动控制(包括在网络中的摄像机之间传输跟踪的能力)，生产率估计，偏差检测，以及开发模块的集成。最终结果将是一个自动化框架，能够在不同类型的施工环境中跟踪不同类型的机械。

Construction operations, like other industrial and service processes, require real-time feedback systems to measure key performance indicators, such as productivity, and to take steps to correct problems that interfere with efficient operations. Currently, automated systems are broadly used in many industries and services and have improved productivity substantially. However, the construction industry has been slow to adopt automation. To develop effective automated feedback systems, researchers have tested different sensing technologies to monitor workers and equipment in the uniquely rugged environment of construction jobsites. Earthmoving projects, such as highway construction and surface mining operations, have been a primary initial focus in the industry for developing automated sensing systems. However, monitoring different types of equipment in a dynamic environment is a challenge; systems are challenged to understand the context of a construction site. For example, hydraulic excavators can take any number of shapes, which strains the ability of software to classify the actions of the equipment. Low-cost digital cameras, combined with promising advances in computer vision, make vision-based systems likely candidates for solutions in construction operations. The current systems using vision-based algorithms to monitor heavy operations, however, aren’t ideal and require some level of human intervention to operate successfully. For instance, they are not able to understand the scene and the user must determine the action type to be monitored; also, the user has to set the viewfinder of the camera on the operation of interest. The objective of this research program is to bridge the gap between theoretical computer vision and machine learning algorithms on one hand, and practical applications on the other, to develop a generic smart vision-based system to measure the productivity of earthmoving processes in construction and surface mining jobsites. Beyond object recognition and tracking, the proposed research program will investigate development of algorithms for action recognition and understanding the scene based on identified actions, automated control of panning and zooming features of a camera, proactive control of a network of cameras (including the ability to transfer tracking among cameras in the network), productivity estimation, deviation detection, and integration of developed modules. The final result will be an automated framework capable of tracking different kinds of machinery in different types of construction environments.