Mental Canvas: Exploring the Middle Ground Between Sketch and Object

这个项目的目标是开发一种使用草图和照片设计新物体的替代技术；要探索的特殊应用是建筑设计。而不是“自上而下”的方法，在3D中定义大块，并对这些大块进行提炼和装饰，设计将从用户在3D环境中安排的不同细节级别的草图和照片中开发出来。与任何当前的设计系统不同，在设计阶段，草图和图像不需要在几何上表示结构的一致表示。设计师可以在视图中尝试不同的形式和细节变化，而不必与自动更改其他视图的系统作斗争。这种自由是通过仅用笔画和像素表示设计而实现的。该系统将允许用户以一种方式浏览2D草图和照片，让模型通过用户自己的视觉系统融合图像而出现。在设计经过多次迭代后，草图收敛到几乎一致的表示，计算机视觉技术可以从用户输入中提取3D模型。该方法的优点是，用户在熟悉的2D模式下定义细节，没有在过程中的每一步定义完整3D模型的限制和时间延迟。智力优势：在探索草图和对象之间的中间地带时，提出的工作解决了交互式3D图形中如何在2D和3D之间无缝移动的“圣杯”问题之一。这项研究将促进对人类如何构思和设计几何形式的理解（包括形式的抽象和复杂问题的简化），这是一种塑造我们生活和工作中大多数物体的活动。这种理解不仅包括计算机图形学视角（算法和表示），还包括建筑视角，其中实践者对形式创造有深入的工作知识，但缺乏严格分析和描述它的计算或数学背景。更广泛的影响：提议的工作将通过提供创建和编辑3D形式的新方法，对创意设计过程产生直接影响，也将影响2D表示在很大程度上占主导地位的思想交流。这项工作不仅涉及计算机图形学和建筑学，还涉及与人类如何思考（如计算机视觉、认知科学、心理学）和建造（工业设计、土木工程、机械工程）相关的其他研究领域。PI与耶鲁大学建筑学院的教师、学生和实践者的密切关系将提供机会，通过应用于实际设计问题来探索和测试新方法。

The goal of this project is to develop an alternative technique for designing new objects using sketches and photographs; the particular application to be explored is architectural design. Rather than a "top down" approach, defining large masses in 3D and refining and decorating these masses, designs will be developed from sketches and photographs of varying levels of detail arranged by the user in a 3D environment. Unlike any current design system, the sketches and images do not need to represent geometrically- consistent representations of the structure during the design phase. The designer can experiment with different variations in form and detail in a view without struggling with a system that automatically changes other views. This freedom is made possible by representing the design as strokes and pixels only. The system will allow the user to tour the 2D sketches and photographs in a manner that lets the model emerge by the fusion of images by the user's own visual system.After the design is refined by multiple iterations of sketches that converge to a nearly consistent representation, computer vision techniques could be used to extract a 3D model from the users input. The advantage of the approach is that the user works in a familiar 2D mode for defining details, without the limitations and time delays of a full 3D model being defined at each step in the process.Intellectual Merit: In exploring the middle ground between sketch and object, the proposed work addresses one of the "holy grail" problems in interactive 3D graphics how to move seamlessly between 2D and 3D. This investigation will advance the understanding of how humans conceive of and design geometric form (including the abstraction of form and simplification of complex problems), an activity that shapes most of the objects with which we live and work. This understanding includes not only a computer graphics perspective (algorithms and representations) but also an architectural perspective, where practitioners have a deep working knowledge of form creation but lack a computational or mathematical background to rigorously analyze and describe it.Broader Impacts: The proposed work will have a direct impact on the creative design process by offering a novel approach for creating and editing 3D form, and will also impact the communication of ideas, where 2D representation has largely dominated. This work crosses and links disciplines, not only computer graphics and architecture, but other research areas related to how humans think (such as computer vision, cognitive science, psychology) and build (industrial design, civil engineering, mechanical engineering). The PI's close relationship with faculty, students, and practitioners at the Yale School of Architecture will provide opportunity to explore and test the new approach through application to real design problems.