CGV: Medium: Collaborative Research: A Heterogeneous Inference Framework for 3D Modeling and Rendering of Sites

组织和使用与物理站点相关的3D数据在许多应用中都很重要，例如历史重建、建筑设计和城市规划。但是，还没有开发出一种方法来利用这些站点可用的全部数据类型。有用的数据通常来自历史来源，并且需要大量的处理才能有用。其中一些处理可以自动化，但有些必须由人工完成。一个尚未解决的问题是如何协调人力来有效地执行这一过程。在当前的项目中，pi将解决重建3D站点的定量和定性准确性问题，以便允许不同人群在构建数据集时输入和参与，并将展示使用异构3D站点表示的各种应用。具体来说，这项工作将做出以下贡献：将开发用于注释异构输入的新技术，平衡自动化和人工输入；将设计协调数字计算、人类计算和机器学习的新技术；建筑分析和设计以及材料风化分析的新工具将基于新的3D表示开发；并将展示利用3D数据讲故事的新想法。项目成果将包括3D站点异构数据的新组织，对3D站点数据领域自动化技术和人工计算的相对贡献的新见解（这将适用于涉及大型复杂数据集的其他具有挑战性的问题），重建3D模型的新算法，以及进行建筑和文化遗产研究的新技术。更广泛的影响：这项研究将对建筑设计和文化遗产的文献、解释和交流产生重大影响。该项目的各个阶段将涉及研究生和本科生，以及不同学科的学生，包括计算机科学、建筑和艺术史。pi将在此基础上制作针对计算机科学、建筑和文化遗产的教学模块。

Organizing and using 3D data related to physical sites is important in many applications such as historical reconstruction, architectural design, and urban planning. However, no method has been developed that exploits the full range of data types available for such sites. Useful data often comes from historical sources, and requires substantial processing to be useful. Some of this processing can be automated, but some of it must be done by humans. An as-yet unsolved problem is how to coordinate human effort to efficiently carry out this process. In the current project the PIs will address quantitative and qualitative accuracy issues in reconstructing 3D sites so as to allow for input and participation by different populations in building data sets, and will demonstrate a variety of applications using a heterogeneous 3D site representation. Specifically, the work will make the following contributions: new techniques for annotating heterogeneous input will be developed, balancing automated and human input; new techniques for coordinating digital computation, human computation, and machine learning will be devised; new tools for architectural analysis and design, and for material weathering analysis, will be developed based on the new 3D representation; and new ideas for storytelling from 3D data will be demonstrated. Project outcomes will include a new organization of heterogeneous data for 3D sites, new insights into the relative contributions of automated techniques and human computation in the domain of 3D site data (which will be applicable to other challenging problems involving large complex data sets), new algorithms for reconstructing 3D models, and new techniques for conducting studies in architecture and in cultural heritage.Broader Impacts: This research will have a strong impact on architectural-design and cultural heritage documentation, interpretation and communication. The various phases of the project will involve students at both the graduate and undergraduate levels, and in diverse disciplines including computer science, architecture, and art history. The PIs will produce teaching modules based on this work targeted at computer science, architecture, and cultural heritage.