CAREER: Recognition of Dynamic Activities in Unstructured Environments

摘要：本项目致力于研究和开发能够在动态变化的环境中以最低限度的监督识别动作的视觉机。重点在于识别面部、手势和手臂手势、人类步态、人群中个体行为的变化(例如，行走人群中的人开始奔跑)、以及由于相机运动、动态背景(例如，水、雾、火、烟、蒸汽)以及多个移动对象和人在动态变化的环境中发生的人群行为变化(例如，行走人群中的一群人突然开始奔跑)。拟议研究的智力价值将是一个统一的理论框架，用于识别结合了几何学、动力学和聚类学的人类和人群活动。识别任务将被视为混合动力学模型(例如，刚性运动、非刚性运动、线性动力学模型)的推断，表现出在空间(由于单个帧中发生的多个活动)和时间(由于在一段时间内发生的多个活动)的突然变化。这一框架的发展将需要在动态场景重建、时空视频分析、几何空间上的聚类和动力系统上的核等方面取得重大进展。拟议研究的广泛影响包括在计算机视觉、机器学习、控制理论、机器人学和生物医学工程方面的应用。识别人类和人群活动的技术直接适用于监视和安全。用于动态场景重建和时空建模的技术在交通监控、体育报道/广播、人机交互、图像检索和搜索、视频运动捕获和基于图像的绘制中是有用的。几何空间上的聚类技术可应用于控制理论中的混合系统识别、多机器人系统中的侦察和映射问题以及生物医学工程中的基因表达数据和生物网络的建模。

Abstract: This project is concerned with the study and development of seeing machines capable of recognizing actions in dynamically changing environments with a minimal level of supervision. The emphasis will be on the recognition of face, hand and arm gestures, human gaits, changes in individual behaviors in a crowd (e.g., a person in the middle of a walking crowd starts running), and changes in crowd behavior (e.g., a group of individuals in a walking crowd suddenly start running) occurring in a dynamically changing environment due to camera motion, dynamic backgrounds (e.g., water, fog, fire, smoke, steam) and multiple moving objects and people. The intellectual merit of the proposed research will be a unifying theoretical framework for the recognition of human and crowd activities that combines geometry, dynamics and clustering. The recognition task will be viewed as the inference of a mixture of dynamical models (e.g., rigid motions, non-rigid motions, linear dynamical models) exhibiting abrupt changes both in space (due to multiple activities occurring in a single frame) and in time (due to multiple activities occurring over a period of time). The development of this framework will require significant advances on dynamic scene reconstruction, spatio-temporal video analysis, clustering on geometric spaces, and kernels on dynamical systems. The broad impact of the proposed research includes applications in computer vision, machine learning, control theory, robotics, and biomedical engineering. Techniques for recognition of human and crowd activities are directly applicable to surveillance and security. Techniques for dynamic scene reconstruction and spatio-temporal modeling are useful in traffic monitoring, sports coverage/broadcast, human-computer interaction, image retrieval and search, video motion capture, and image-based rendering. Techniques for clustering on geometric spaces can be applied to identification of hybrid systems in control theory, reconnaissance and mapping problems in multiple robot systems, and modeling of gene expression data and biological networks in biomedical engineering.