HCC: Large: Intelligent Tracking Systems that Reason about Group Behavior

“这项奖励是根据2009年美国复苏和再投资法案（公法111-5）资助的。”对不同环境中生物体的复杂性进行推理的能力是智力的标志之一。在这个项目中，PI和她的跨学科研究团队将设计计算机视觉算法，用于在三维空间中智能跟踪大量活着的个体。她将开发用于跟踪微生物、蝙蝠、鸟类和人类群体的特定系统。她将制定机器学习方法来分析群体行为，特别是群体形成和分散的条件，以及群体内个体的相互作用。本研究的一个重要创新方面是系统和全面的方法来推理在视频数据中观察到的大量生物体的运动，而不管它们是人类、动物还是细胞。该领域以前的研究主要集中在研究单一类型生物的行为，以及主要基于模拟的行为理论的测试，没有适当的分析工具来自动探索和量化大量的视觉数据集。另一方面，这个项目将基于对在三维空间中移动的个体群体成员的数千个轨迹的分析得出研究结果。为此，PI和她的团队将在现场和公共场所收集视频数据，以确保最佳的数据捕获条件。他们将利用这些数据为数百只蝙蝠、鸟类或人的画面匹配问题开发强大的解决方案。他们将基于多个校准相机生成运动轨迹的立体重建，并使用机器学习来模拟群体行为并挖掘轨迹数据。最后，他们将把他们的推理系统的发现与当前关于群体形成和群体内个体相互作用的理论进行比较。一个类似的，系统的研究策略将被用来解决对单细胞行为的理解。该团队将设计显微镜成像协议，开发单个细胞分割和跟踪的解决方案，并使用统计学习技术来发现细胞在生理相关基质上行为的模式和相关性。更广泛的影响：了解动物和微生物群体的行为过程对有效保护种群和生态系统以及管理细胞环境至关重要。项目成果将推进计算机视觉、人工智能、行为生态学和生物工程等领域的知识，并将为回答紧迫的经济和伦理问题提供新的工具，例如关于风能设施中鸟类和蝙蝠的死亡率。

"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."The ability to reason about the complexity of living organisms in diverse environments is one of the hallmarks of intelligence. In this project the PI and her interdisciplinary team of investigators will design computer vision algorithms for intelligent tracking of large groups of living individuals in three-dimensional space. She will develop specific systems for tracking groups of microorganisms, bats, birds, and humans. And she will formulate machine learning methods for analyzing group behavior, specifically the conditions for formation and dispersal of groups, and the interactions of individuals within a group. An important innovative aspect of this research is the systematic and comprehensive approach to reasoning about the motion of large groups of living organisms observed in video data, independently of whether they happen to be humans, animals, or cells. Previous efforts in this area have typically focused on studying the behavior of a single type of organism, and on testing theories of behavior based predominately on simulations, without the appropriate analytical tools to automatically explore and quantify the vast number of visual data sets. This project, on the other hand, will base research findings on the analysis of thousands of trajectories of individual group members moving in 3D space. To this end, the PI and her team will collect video data in the field and in public spaces to ensure optimal data capture conditions. They will use these data to develop robust solutions for the problem of matching hundreds of individual bats, birds, or people from frame to frame. They will generate stereoscopic reconstructions of movement trajectories based on multiple calibrated cameras, and use machine learning to model group behavior and mine the trajectory data. Finally, they will compare the findings of their reasoning system against current theories about the formation of groups and the interactions of individuals within a group. A similar, systematic research strategy will be employed to address understanding of the behavior of single cells. The team will design microscope imaging protocols, develop solutions for the segmentation and tracking of individual cells, and use statistical learning techniques to discover patterns and correlations in the behavior of the cells on physiologically relevant substrates.Broader Impacts: Understanding the processes by which groups of animals and microorganisms behave is crucial to the effective conservation of populations and ecosystems and the management of cellular environments. Project outcomes will advance knowledge across the fields of computer vision, artificial intelligence, behavioral ecology, and biological engineering, and will provide new tools for answering urgent economic and ethical questions, for example about the mortality of birds and bats in wind energy facilities.