HCC: Large: Intelligent Tracking Systems that Reason about Group Behavior

这个奖项是根据2009年《美国复苏和再投资法案》(公法111-5)资助的。在这个项目中，PI和她的跨学科研究团队将设计计算机视觉算法，以在三维空间中智能跟踪大量活着的个人。她将开发特定的系统来跟踪微生物、蝙蝠、鸟类和人类的群体。她还将制定机器学习方法来分析群体行为，特别是群体形成和扩散的条件，以及群体中个人之间的互动。这项研究的一个重要的创新方面是系统和全面的方法来推理在视频数据中观察到的大群生物的运动，而无论他们是人类、动物还是细胞。以前在这一领域的努力通常集中在研究单一类型生物体的行为，以及测试主要基于模拟的行为理论，而没有适当的分析工具来自动探索和量化大量的可视数据集。另一方面，该项目将基于对数千个群体成员在3D空间中移动的轨迹的分析得出研究成果。为此，PI和她的团队将在现场和公共场所收集视频数据，以确保最佳数据捕获条件。他们将利用这些数据为数百只蝙蝠、鸟类或人从一帧到另一帧的匹配问题开发强大的解决方案。他们将基于多个校准的摄像头生成运动轨迹的立体重建，并使用机器学习来建模群体行为并挖掘轨迹数据。最后，他们将把他们的推理系统的发现与当前关于群体形成和群体内个人相互作用的理论进行比较。将采用类似的、系统的研究策略来解决对单个细胞行为的理解。该团队将设计显微镜成像方案，开发单个细胞的分割和跟踪解决方案，并使用统计学习技术来发现细胞在生理相关底物上的行为模式和相关性。广泛的影响：了解动物和微生物群体的行为过程对于有效保护种群和生态系统以及管理细胞环境至关重要。项目成果将推动计算机视觉、人工智能、行为生态学和生物工程等领域的知识进步，并将为回答紧迫的经济和伦理问题提供新的工具，例如关于风能设施中鸟类和蝙蝠的死亡问题。

"This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5)."The ability to reason about the complexity of living organisms in diverse environments is one of the hallmarks of intelligence. In this project the PI and her interdisciplinary team of investigators will design computer vision algorithms for intelligent tracking of large groups of living individuals in three-dimensional space. She will develop specific systems for tracking groups of microorganisms, bats, birds, and humans. And she will formulate machine learning methods for analyzing group behavior, specifically the conditions for formation and dispersal of groups, and the interactions of individuals within a group. An important innovative aspect of this research is the systematic and comprehensive approach to reasoning about the motion of large groups of living organisms observed in video data, independently of whether they happen to be humans, animals, or cells. Previous efforts in this area have typically focused on studying the behavior of a single type of organism, and on testing theories of behavior based predominately on simulations, without the appropriate analytical tools to automatically explore and quantify the vast number of visual data sets. This project, on the other hand, will base research findings on the analysis of thousands of trajectories of individual group members moving in 3D space. To this end, the PI and her team will collect video data in the field and in public spaces to ensure optimal data capture conditions. They will use these data to develop robust solutions for the problem of matching hundreds of individual bats, birds, or people from frame to frame. They will generate stereoscopic reconstructions of movement trajectories based on multiple calibrated cameras, and use machine learning to model group behavior and mine the trajectory data. Finally, they will compare the findings of their reasoning system against current theories about the formation of groups and the interactions of individuals within a group. A similar, systematic research strategy will be employed to address understanding of the behavior of single cells. The team will design microscope imaging protocols, develop solutions for the segmentation and tracking of individual cells, and use statistical learning techniques to discover patterns and correlations in the behavior of the cells on physiologically relevant substrates.Broader Impacts: Understanding the processes by which groups of animals and microorganisms behave is crucial to the effective conservation of populations and ecosystems and the management of cellular environments. Project outcomes will advance knowledge across the fields of computer vision, artificial intelligence, behavioral ecology, and biological engineering, and will provide new tools for answering urgent economic and ethical questions, for example about the mortality of birds and bats in wind energy facilities.