CGV: Medium: Collaborative Research: Developing conceptual models for navigation, marking, and inspection in the context of 3D image segmentation

在以图像为导向的科学学科中，三维图像分割是一项重要而普遍的任务，特别是生物医学，图像为生物发现提供了基础。虽然成像技术揭示了整个主题的空间内容和活动，但最终生物学家对特定的解剖结构（例如，器官，组织，细胞等）感兴趣。因此，在给定的一组图像中描述感兴趣的结构是数据到知识管道的典型第一步，分割的效率和准确性严重影响数据在研究和临床实践中的利用方式。创建准确的分割，特别是3D生物医学图像，是一项需要人类和计算机之间合作的重要任务。虽然人类专家凭借其卓越的视觉感知技能和多年训练中获得的丰富知识和经验，最终决定了什么是准确的分割，但他们缺乏计算算法的客观性和效率。另一方面，如果没有专家的指导，分割算法很容易在生物医学图像中不可避免的噪声和歧义的存在下失败。在本研究中，pi将研究3D图像分割作为人机交互范例，以更好地了解当前分割过程中涉及的人为因素，目标是使该过程更高效、准确和可重复。该团队的假设是，通过更深入地了解人们如何在3D分割的背景下执行低级感知和认知任务（例如，视觉线索，通过标记描绘结构，以及局部精度或质量标准），以及领域专家希望如何指定高级分割约束（例如，连通性，拓扑结构和形状），可以显著改善分割过程。为了验证这一假设，pi将由领域专家分析分割过程，这些领域专家跨越了生物学和临床实践中实际分割器和分割任务的合理子空间，以定义一个概念框架，该框架捕获了分割的低级感知和认知元素以及与导航，标记和检查相关的高级信息。在此框架的基础上，团队将与专家合作开发一个原型分割工具，探索新的交互和可视化范例及其支持算法。原型工具将用于验证概念框架并创建更有效的实际分割解决方案。更广泛的影响：通过将分割作为人类感知和认知任务进行制定和研究，这项工作代表了对分割算法或工具的现有研究的重大背离。由此产生的概念框架将成为两个社区之间的桥梁，既可以为当前和未来的分割工具提供更好的设计，也可以为分割算法提出新的问题。对于最终用户，工作原型将支持由底层概念框架提供动力的更有效的分段体验。此外，形式化用户在处理分割问题时所拥有的感知线索和概念模型的种类，将作为一个有用的测试案例，用于理解更普遍的问题，即当感知和认知被重新映射以解决它们从未设计过的问题时，它们是如何相互作用的。为了传播这项研究的结果，pi将把他们的工作原型作为开源项目发布，然后作为算法开发人员、工具开发人员和最终用户之间的共享通信平台。

3D image segmentation is an important and ubiquitous task in image-oriented scientific disciplines, particularly biomedicine, where images provide the basis for biological discovery. While imaging techniques reveal spatial content and activities within an entire subject, ultimately biologists are interested in specific anatomical structures (e.g., organs, tissues, cells, etc.). Delineation of the structures of interest within a given set of images is therefore a typical first-step in the data-to-knowledge pipeline, with both the efficiency and accuracy of segmentation critically affecting how the data is utilized in research and clinical practice. Creating accurate segmentations, particularly for 3D biomedical images, is a non-trivial task that calls for cooperation between humans and computers. While human experts, with their superior visual perception skills and vast knowledge and experience acquired from years of training, ultimately decide what constitutes an accurate segmentation, they lack the objectivity or efficiency of computational algorithms. On the other hand, without expert guidance, segmentation algorithms easily fail in the presence of the noise and ambiguity that are inevitable in biomedical images. In this research the PIs will investigate 3D image segmentation as a human-computer interaction paradigm to better understand the human factors that are involved in the current segmentation process, with the goal of making the process more efficient, accurate and repeatable. The team's hypothesis is that the segmentation process could be significantly improved through a deeper understanding of how people perform low-level perception and cognition tasks in the context of 3D segmentation (e.g., visual cues, delineation of structures by marks, and local accuracy or quality criteria), and how domain experts wish to specify high-level segmentation constraints (e.g., connectivity, topology, and shape). To test this hypothesis the PIs will analyze the segmentation process by domain experts that span a reasonable subspace of the actual segmentors and segmentation tasks in biology and clinical practice, to define a conceptual framework that captures the low-level perception and cognitive elements of segmentation as well as the higher-level information related to navigation, marking, and inspection. Building upon and instantiating the framework, the team will work with experts to develop a prototype segmentation tool that explores novel interaction and visualization paradigms as well as their supporting algorithms. The prototype tool will be used to both verify the conceptual framework and to create a more effective practical solution to segmentation.Broader Impacts: By formulating and studying segmentation as a human perception and cognitive task, this work represents a major departure from existing research on either segmentation algorithms or tools. The resulting conceptual framework will serve as a bridge between the two communities, leading both to better designs for current and future segmentation tools and the framing of new problems for segmentation algorithms. For end users, the working prototype will support a more effective segmentation experience that is powered by the underlying conceptual framework. Furthermore, formalizing the kinds of perceptual cues and conceptual models users have when approaching the segmentation problem will serve as a useful test case for understanding the more general question of how perception and cognition interact when they are re-mapped to solve a problem they were never designed for. To disseminate the findings of this research, the PIs will release their working prototype as an open-source project, which can then serve as a shared communication platform between algorithm developers, tool developers, and end users.