An Interactive Web-Based Game for Collaborative Labeling of Medical Images

描述(由申请人提供)：脑局部解剖的统计图谱已被证明在描述人类神经系统结构和功能之间的关系方面非常有用。通常情况下，专家人类评分员手动检查三维体积的每一片。这种方法可能非常耗费时间和资源，因此成本严重限制了特定对象标记可行的临床研究。提高人工标记的效率和可靠性的方法将为临床研究脑功能的形态相关性提供极大的帮助。这项拟议工作的目标是通过对许多受过最低培训的评分员的合作努力进行统计分析，为医学图像标签提供专家评分员的替代方案。拟议的研究调查了体积标记和基于网络的协作的既定做法的扩展，以创建标记的创新基础设施。将开发一个用户界面，作为基于网络的互动游戏，用户将在其中解决小谜题。每个谜题将代表整个标签挑战的一部分。来自许多评分者的重叠和互补的结果将在一个统计框架内重新组合，以估计并最大限度地减少标记区域的不确定性。考虑到不同的培训经验、工作环境、计算机硬件和动机，评分员在现实世界中的表现很难预测。为了考察这一系统的实用价值，我们将为给定三个激励范式的统计重新组合结果建立评分员内、评分员间标签以及测试-重测指标：1)将招募有偿实习生进行试点研究；2)将鼓励公众参与竞争性的在线游戏；以及3)将进行有限的试验，1基于1个绩效1在1个标签1游戏1中1。这种标签方法的优势将在1阿尔茨海默病1的小脑和海马区的试点研究中得到展示。1该系统将免费提供。拟议的系统将通过实现社区合作参与，同时提供对结果的准确性和不确定性的主动监测和控制，极大地降低创建手动标签的成本。我们将为标签语言奠定基础，这是一种新颖的、一致的框架，用于以分层、程序化的方式描述标签目标。增加高可靠性手动标签的可用性将提高健康和疾病大脑中定量MRI分析的质量、一致性和效率。公共卫生相关性：这项拟议的研究通过对许多受过最低培训的评分员的合作努力的统计分析，调查了一种替代专家评分员的手动标记医学图像的方法。这个基于网络的系统将极大地降低研究感兴趣的解剖区域的成本，因为它允许许多个人之间进行合作，同时提供对结果的准确性和不确定性的主动监测和控制。

DESCRIPTION (provided by applicant): Statistical atlases of regional brain anatomy have proven to be extremely useful in characterizing the relationship between the structure and function of the human nervous system. Typically, an expert human rater manually examines each slice of a three-dimensional volume. This approach can be exceptionally time and resource intensive, so cost severely limits the clinical studies where subject-specific labeling is feasible. Methods for improved efficiency and reliability of manual labeling would be of immense benefit for clinical investigation into morphological correlates of brain function. The goal of the proposed work is to enable an alternative to expert raters for medical image labeling through statistical analysis of the collaborative efforts of many, minimally-trained raters. The proposed research investigates extension of established practices for volumetric labeling and web- based collaboration to create an innovative infrastructure for labeling. A user interface will be developed as an interactive web-based game in which users will solve small puzzles. Each puzzle will represent a portion of the overall labeling challenge. Overlapping and complementary results from many raters will be recombined within a statistical framework that estimates and minimize the uncertainty of labeled regions. Real world performance of raters is difficult to predict given varied training experience, work environment, computer hardware, and motivation. To investigate the practical utility of this system, we will establish reliability measures for intra-rater, inter-rater labels as well as test-retest metrics for statistically recombined results given three motivational paradigms: 1) Paid interns will be recruited for a pilot study; 2) The public will be encourage to participate in a competitive, online game; and 3) A limited trial will be conducted1 with1 payments1 based1 on1 performance1 in1 the1 labeling1 game .1 The advantages of this labeling approach will be demonstrated in pilot studies of the cerebellum in spinocerebellar ataxia and of the hippocampus1in1Alzheimer s1Disease.1The system will be made freely available. The proposed system will dramatically reduce the cost of creating manual labels by enabling collaborative community involvement, while providing active monitoring and control of the accuracy and uncertainty of the results. We will lay the foundation for a language of labeling, which is a novel, consistent framework for describing labeling objectives in a hierarchical, programmatic manner. Increased availability of high reliability manual labels will improve the quality, consistency, and efficiency of quantitative MRI analyses in the brain in health and disease. PUBLIC HEALTH RELEVANCE: The proposed research investigates an alternative to expert raters for manual labeling of medical images through statistical analysis of the collaborative efforts of many, minimally- trained raters. The web-based system will dramatically reduce the cost of studying anatomical regions of interest by enabling collaboration among many individuals, while providing active monitoring and control of the accuracy and uncertainty of the results.