HCC: Large: Intelligent Tracking Systems that Reason about Group Behavior

“该奖项是根据2009年美国复苏和再投资法案（公法111-5）资助的。“在不同环境中推理生物体复杂性的能力是智能的标志之一。 在这个项目中，PI和她的跨学科研究团队将设计计算机视觉算法，用于在三维空间中智能跟踪大群活体个体。 她将开发用于跟踪微生物，蝙蝠，鸟类和人类群体的特定系统。 她将制定机器学习方法来分析群体行为，特别是群体形成和分散的条件，以及群体内个体的相互作用。 这项研究的一个重要创新方面是系统和全面的方法来推理在视频数据中观察到的大量生物体的运动，无论它们是人类，动物还是细胞。 以前在这一领域的努力通常集中在研究单一类型生物体的行为，以及主要基于模拟的行为理论的测试，而没有适当的分析工具来自动探索和量化大量的视觉数据集。 另一方面，该项目将基于对个体群体成员在3D空间中移动的数千条轨迹的分析得出研究结果。 为此，PI及其团队将在现场和公共场所收集视频数据，以确保最佳的数据采集条件。 他们将使用这些数据来开发强大的解决方案，以解决数百只蝙蝠，鸟类或人的帧与帧之间的匹配问题。 他们将基于多个校准的相机生成运动轨迹的立体重建，并使用机器学习来建模群体行为并挖掘轨迹数据。 最后，他们将把他们的推理系统的发现与当前关于群体形成和群体内个体相互作用的理论进行比较。 将采用类似的系统研究策略来解决对单细胞行为的理解。 该团队将设计显微镜成像方案，开发用于分割和跟踪单个细胞的解决方案，并使用统计学习技术来发现细胞在生理相关基质上的行为模式和相关性。了解动物和微生物群体的行为过程对于有效保护种群和生态系统以及管理细胞内的微生物是至关重要的。环境. 项目成果将推动计算机视觉、人工智能、行为生态学和生物工程领域的知识发展，并将为回答紧迫的经济和伦理问题提供新的工具，例如关于风能设施中鸟类和蝙蝠的死亡率。

"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."The ability to reason about the complexity of living organisms in diverse environments is one of the hallmarks of intelligence. In this project the PI and her interdisciplinary team of investigators will design computer vision algorithms for intelligent tracking of large groups of living individuals in three-dimensional space. She will develop specific systems for tracking groups of microorganisms, bats, birds, and humans. And she will formulate machine learning methods for analyzing group behavior, specifically the conditions for formation and dispersal of groups, and the interactions of individuals within a group. An important innovative aspect of this research is the systematic and comprehensive approach to reasoning about the motion of large groups of living organisms observed in video data, independently of whether they happen to be humans, animals, or cells. Previous efforts in this area have typically focused on studying the behavior of a single type of organism, and on testing theories of behavior based predominately on simulations, without the appropriate analytical tools to automatically explore and quantify the vast number of visual data sets. This project, on the other hand, will base research findings on the analysis of thousands of trajectories of individual group members moving in 3D space. To this end, the PI and her team will collect video data in the field and in public spaces to ensure optimal data capture conditions. They will use these data to develop robust solutions for the problem of matching hundreds of individual bats, birds, or people from frame to frame. They will generate stereoscopic reconstructions of movement trajectories based on multiple calibrated cameras, and use machine learning to model group behavior and mine the trajectory data. Finally, they will compare the findings of their reasoning system against current theories about the formation of groups and the interactions of individuals within a group. A similar, systematic research strategy will be employed to address understanding of the behavior of single cells. The team will design microscope imaging protocols, develop solutions for the segmentation and tracking of individual cells, and use statistical learning techniques to discover patterns and correlations in the behavior of the cells on physiologically relevant substrates.Broader Impacts: Understanding the processes by which groups of animals and microorganisms behave is crucial to the effective conservation of populations and ecosystems and the management of cellular environments. Project outcomes will advance knowledge across the fields of computer vision, artificial intelligence, behavioral ecology, and biological engineering, and will provide new tools for answering urgent economic and ethical questions, for example about the mortality of birds and bats in wind energy facilities.