III: Medium: Efficient Collaborative Perception over Controllable Agent Networks

当今的许多应用程序提供不同格式的数据（例如，文本，视频）从不同种类的传感器（例如，推特、摄像头、安装在机器人上的传感器）。有关于如何分别搜索每个数据源的工作，但这忽略了跨源数据之间的隐藏连接。例如，在灾害管理中，协作感知将允许通过联合分析来自社交帖子的文本和图像内容、来自固定或移动的电话摄像头的视频以及UAV记录的图像或视频来搜索语义概念“火灾”或“人群”可能存在的位置。该项目将研究如何通过将来自所有数据源的信息映射到一个公共信息空间来跨数据源联合搜索。该项目有可能提高社会和视频监控数据在需要态势感知的任务中的效用。在救灾领域，救灾人员将对局势有更全面的综合看法，从而能够更好地分配资源。该项目将利用加州大学滨江，这是一个西班牙裔服务机构的学生多样性，从而扩大在研究过程中代表性不足的群体的参与。这个项目将加强和扩大正在进行的高中和大学的PI的推广活动。这个项目的目标是创造知识，以促进有效和高效的协作感知上的一套独立和多模式的数据生成代理。该项目将研究如何联合建模社会和传感器数据，并使用这种建模来有效地支持联合嵌入空间上的时空查询。除了将信息从多模态的不同来源映射到一个公共信息空间之外，该项目还将研究如何优化无人机等可控代理的注意力路由，以最大限度地提高收集信息的可靠性和覆盖范围。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Many of today's applications provide data in different formats (e.g., text, videos) from different kinds of sensors (e.g., tweets, cameras, sensors mounted on robots). There is work on how to search each source of data separately, but this misses the hidden connections between the data across the sources. As an example, in disaster management, collaborative perception would allow searching for locations where the semantic concepts "fire" or "crowds" may be present, by jointly analyzing text and image content from social posts, video from stationary or mobile phone cameras, and the images or videos recorded by UAVs. This project will study how to jointly search across data sources by mapping the information coming from all data sources to a common information space. The project has the potential to increase the utility of social and video surveillance data for tasks that require situational awareness. In the area of disaster response, responders will have a more integrated and holistic view of the situation, so they can better allocate their resources. The project will capitalize on the student diversity at UC Riverside, which is a Hispanic Serving Institution, and thus broaden the participation of under-represented groups in the research process. This project will strengthen and extend the ongoing high school and college outreach activities of the PIs.The goal of this project is to create the knowledge to facilitate effective and efficient collaborative perception on top of a set of independent and multi-modal data generating agents. The project will study how to jointly model social and sensor data and use this modeling to efficiently support spatio-temporal queries on the joint embedding space. In addition to mapping information from multi-modal disparate sources to a common information space, this project will study how to optimize the attention routing of controllable agents like UAVs to maximize the reliability and coverage of the collected information.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.