CGV: Medium: Collaborative Research: Developing conceptual models for navigation, marking, and inspection in the context of 3D image segmentation

三维图像分割是以图像为导向的科学领域中一项重要且普遍存在的任务，特别是在生物医学领域，图像是生物发现的基础。虽然成像技术揭示了整个对象的空间内容和活动，但生物学家最终对特定的解剖结构(如器官、组织、细胞等)感兴趣。因此，在一组给定的图像中描绘感兴趣的结构是数据到知识管道中典型的第一步，分割的效率和准确性都至关重要地影响着数据在研究和临床实践中的利用方式。创建准确的分割，特别是3D生物医学图像，是一项需要人类和计算机合作的艰巨任务。虽然人类专家凭借其出众的视觉感知技能以及多年培训获得的丰富知识和经验，最终决定了什么构成了准确的分割，但他们缺乏计算算法的客观性或效率。另一方面，在没有专家指导的情况下，分割算法很容易在生物医学图像中不可避免的噪声和歧义的存在下失败。在这项研究中，PI将研究3D图像分割作为一种人机交互范式，以更好地理解当前分割过程中涉及的人为因素，目标是使该过程更高效、更准确和更具可重复性。该团队的假设是，通过更深入地了解人们如何在3D分割环境中执行低级感知和认知任务(例如，视觉线索、通过标记描绘结构以及局部准确性或质量标准)，以及领域专家希望如何指定高级分割约束(例如，连通性、拓扑和形状)，分割过程可以得到显著改善。为了验证这一假设，PI将分析领域专家的分割过程，这些分割过程跨越生物学和临床实践中实际分割者和分割任务的合理子空间，以定义一个概念框架，该框架捕捉分割的低级别感知和认知元素以及与导航、标记和检查相关的高级信息。在该框架的基础上并实例化，该团队将与专家合作开发一个原型分割工具，该工具探索新的交互和可视化范例及其支持算法。原型工具将用于验证概念框架和创建更有效的实际分割解决方案。广泛的影响：通过将分割作为人类感知和认知任务来制定和研究，这项工作与现有的分割算法或工具研究大相径庭。由此产生的概念框架将成为两个社区之间的桥梁，导致双方更好地设计当前和未来的分割工具，并为分割算法制定新的问题框架。对于最终用户，工作原型将支持由底层概念框架支持的更有效的细分体验。此外，将用户在处理分割问题时拥有的感知线索和概念模型的种类形式化，将作为一个有用的测试用例，以理解更一般的问题，即当感知和认知被重新映射以解决它们从未设计过的问题时，它们如何相互作用。为了传播这项研究的结果，PI将以开源项目的形式发布他们的工作原型，然后可以作为算法开发人员、工具开发人员和最终用户之间的共享交流平台。

3D image segmentation is an important and ubiquitous task in image-oriented scientific disciplines, particularly biomedicine, where images provide the basis for biological discovery. While imaging techniques reveal spatial content and activities within an entire subject, ultimately biologists are interested in specific anatomical structures (e.g., organs, tissues, cells, etc.). Delineation of the structures of interest within a given set of images is therefore a typical first-step in the data-to-knowledge pipeline, with both the efficiency and accuracy of segmentation critically affecting how the data is utilized in research and clinical practice. Creating accurate segmentations, particularly for 3D biomedical images, is a non-trivial task that calls for cooperation between humans and computers. While human experts, with their superior visual perception skills and vast knowledge and experience acquired from years of training, ultimately decide what constitutes an accurate segmentation, they lack the objectivity or efficiency of computational algorithms. On the other hand, without expert guidance, segmentation algorithms easily fail in the presence of the noise and ambiguity that are inevitable in biomedical images. In this research the PIs will investigate 3D image segmentation as a human-computer interaction paradigm to better understand the human factors that are involved in the current segmentation process, with the goal of making the process more efficient, accurate and repeatable. The team's hypothesis is that the segmentation process could be significantly improved through a deeper understanding of how people perform low-level perception and cognition tasks in the context of 3D segmentation (e.g., visual cues, delineation of structures by marks, and local accuracy or quality criteria), and how domain experts wish to specify high-level segmentation constraints (e.g., connectivity, topology, and shape). To test this hypothesis the PIs will analyze the segmentation process by domain experts that span a reasonable subspace of the actual segmentors and segmentation tasks in biology and clinical practice, to define a conceptual framework that captures the low-level perception and cognitive elements of segmentation as well as the higher-level information related to navigation, marking, and inspection. Building upon and instantiating the framework, the team will work with experts to develop a prototype segmentation tool that explores novel interaction and visualization paradigms as well as their supporting algorithms. The prototype tool will be used to both verify the conceptual framework and to create a more effective practical solution to segmentation.Broader Impacts: By formulating and studying segmentation as a human perception and cognitive task, this work represents a major departure from existing research on either segmentation algorithms or tools. The resulting conceptual framework will serve as a bridge between the two communities, leading both to better designs for current and future segmentation tools and the framing of new problems for segmentation algorithms. For end users, the working prototype will support a more effective segmentation experience that is powered by the underlying conceptual framework. Furthermore, formalizing the kinds of perceptual cues and conceptual models users have when approaching the segmentation problem will serve as a useful test case for understanding the more general question of how perception and cognition interact when they are re-mapped to solve a problem they were never designed for. To disseminate the findings of this research, the PIs will release their working prototype as an open-source project, which can then serve as a shared communication platform between algorithm developers, tool developers, and end users.