Mental Canvas: Exploring the Middle Ground Between Sketch and Object

这个项目的目标是开发一种使用草图和照片来设计新物体的替代技术；要探索的特殊应用是建筑设计。设计将从用户在3D环境中安排的不同细节级别的草图和照片中发展而来，而不是以3D方式定义大质量并改进和装饰这些质量。与任何当前的设计系统不同，草图和图像不需要在设计阶段表示结构的几何一致性表示。设计师可以在一个视图中尝试不同的形式和细节，而不需要与自动更改其他视图的系统作斗争。这种自由是通过仅将设计表示为笔画和像素来实现的。该系统将允许用户浏览2D草图和照片，其方式是让模型通过用户自己的视觉系统融合图像而显现。在通过多次草图迭代来改进设计之后，可以使用计算机视觉技术从用户输入中提取3D模型。这种方法的优点是，用户在熟悉的2D模式下定义细节，而不会在过程的每个步骤中定义完整的3D模型的限制和时间延迟。智力优势：在探索草图和对象之间的中间地带时，所提出的工作解决了交互式3D图形中如何在2D和3D之间无缝移动的圣杯问题之一。这项研究将促进对人类如何构思和设计几何形状(包括形式的抽象和复杂问题的简化)的理解，几何形状是一种塑造我们生活和工作中大多数物体的活动。这种理解不仅包括计算机图形学角度(算法和表示法)，还包括建筑角度，从业者拥有深厚的形式创建工作知识，但缺乏计算或数学背景来严格分析和描述形式。广泛影响：拟议的工作将通过提供一种创建和编辑3D形式的新方法，直接影响创意设计过程，并将影响思想的交流，其中2D表示在很大程度上占主导地位。这项工作跨越和联系了多个学科，不仅是计算机图形学和建筑学，而且还有其他与人类思维方式(如计算机视觉、认知科学、心理学)和建筑(工业设计、土木工程、机械工程)相关的研究领域。PI与耶鲁大学建筑学院的教职员工、学生和从业者的密切关系将提供机会，通过将新方法应用于实际设计问题来探索和测试新方法。

The goal of this project is to develop an alternative technique for designing new objects using sketches and photographs; the particular application to be explored is architectural design. Rather than a "top down" approach, defining large masses in 3D and refining and decorating these masses, designs will be developed from sketches and photographs of varying levels of detail arranged by the user in a 3D environment. Unlike any current design system, the sketches and images do not need to represent geometrically- consistent representations of the structure during the design phase. The designer can experiment with different variations in form and detail in a view without struggling with a system that automatically changes other views. This freedom is made possible by representing the design as strokes and pixels only. The system will allow the user to tour the 2D sketches and photographs in a manner that lets the model emerge by the fusion of images by the user's own visual system.After the design is refined by multiple iterations of sketches that converge to a nearly consistent representation, computer vision techniques could be used to extract a 3D model from the users input. The advantage of the approach is that the user works in a familiar 2D mode for defining details, without the limitations and time delays of a full 3D model being defined at each step in the process.Intellectual Merit: In exploring the middle ground between sketch and object, the proposed work addresses one of the "holy grail" problems in interactive 3D graphics how to move seamlessly between 2D and 3D. This investigation will advance the understanding of how humans conceive of and design geometric form (including the abstraction of form and simplification of complex problems), an activity that shapes most of the objects with which we live and work. This understanding includes not only a computer graphics perspective (algorithms and representations) but also an architectural perspective, where practitioners have a deep working knowledge of form creation but lack a computational or mathematical background to rigorously analyze and describe it.Broader Impacts: The proposed work will have a direct impact on the creative design process by offering a novel approach for creating and editing 3D form, and will also impact the communication of ideas, where 2D representation has largely dominated. This work crosses and links disciplines, not only computer graphics and architecture, but other research areas related to how humans think (such as computer vision, cognitive science, psychology) and build (industrial design, civil engineering, mechanical engineering). The PI's close relationship with faculty, students, and practitioners at the Yale School of Architecture will provide opportunity to explore and test the new approach through application to real design problems.