An Interactive Web-Based Game for Collaborative Labeling of Medical Images

描述（由申请人提供）：区域脑解剖学统计图谱已被证明对于表征人类神经系统的结构和功能之间的关系非常有用。通常，专家评估员手动检查三维体积的每个切片。这种方法可能非常耗费时间和资源，因此成本严重限制了可以进行特定主题标记的临床研究。提高手动标记效率和可靠性的方法将为脑功能形态相关性的临床研究带来巨大好处。拟议工作的目标是通过对许多经过最低限度培训的评估员的协作工作进行统计分析，为医学图像标记提供专家评估员的替代方案。拟议的研究调查了体积标签和基于网络的协作的既定实践的扩展，以创建创新的标签基础设施。用户界面将被开发为基于交互式网络的游戏，用户将在其中解决小难题。每个谜题将代表整个标签挑战的一部分。来自许多评估者的重叠和互补结果将在统计框架内重新组合，以估计和最小化标记区域的不确定性。鉴于不同的培训经验、工作环境、计算机硬件和动机，评估者的真实表现很难预测。为了研究该系统的实际效用，我们将在给定三种激励范式的情况下，建立评估者内部、评估者间标签的可靠性度量以及统计重组结果的重测指标：1）将招募带薪实习生进行试点研究； 2）鼓励公众参与竞争性在线游戏； 3) 将进行有限的试验1，并根据1标记1游戏中的1表现1进行1付费1。1这种标记方法的优点将在脊髓小脑共济失调小脑和阿尔茨海默病1疾病海马的试点研究中得到证明。1该系统将免费提供。所提出的系统将通过支持社区协作参与来显着降低创建手动标签的成本，同时提供对结果准确性和不确定性的主动监控和控制。我们将为标签语言奠定基础，这是一种新颖、一致的框架，用于以分层、程序化的方式描述标签目标。高可靠性手动标签可用性的增加将提高健康和疾病大脑中定量 MRI 分析的质量、一致性和效率。公共卫生相关性：拟议的研究通过对许多经过最低限度培训的评估者的协作努力进行统计分析，调查了专家评估者手动标记医学图像的替代方案。基于网络的系统将通过实现许多人之间的协作来显着降低研究感兴趣的解剖区域的成本，同时提供对结果的准确性和不确定性的主动监测和控制。

DESCRIPTION (provided by applicant): Statistical atlases of regional brain anatomy have proven to be extremely useful in characterizing the relationship between the structure and function of the human nervous system. Typically, an expert human rater manually examines each slice of a three-dimensional volume. This approach can be exceptionally time and resource intensive, so cost severely limits the clinical studies where subject-specific labeling is feasible. Methods for improved efficiency and reliability of manual labeling would be of immense benefit for clinical investigation into morphological correlates of brain function. The goal of the proposed work is to enable an alternative to expert raters for medical image labeling through statistical analysis of the collaborative efforts of many, minimally-trained raters. The proposed research investigates extension of established practices for volumetric labeling and web- based collaboration to create an innovative infrastructure for labeling. A user interface will be developed as an interactive web-based game in which users will solve small puzzles. Each puzzle will represent a portion of the overall labeling challenge. Overlapping and complementary results from many raters will be recombined within a statistical framework that estimates and minimize the uncertainty of labeled regions. Real world performance of raters is difficult to predict given varied training experience, work environment, computer hardware, and motivation. To investigate the practical utility of this system, we will establish reliability measures for intra-rater, inter-rater labels as well as test-retest metrics for statistically recombined results given three motivational paradigms: 1) Paid interns will be recruited for a pilot study; 2) The public will be encourage to participate in a competitive, online game; and 3) A limited trial will be conducted1 with1 payments1 based1 on1 performance1 in1 the1 labeling1 game .1 The advantages of this labeling approach will be demonstrated in pilot studies of the cerebellum in spinocerebellar ataxia and of the hippocampus1in1Alzheimer s1Disease.1The system will be made freely available. The proposed system will dramatically reduce the cost of creating manual labels by enabling collaborative community involvement, while providing active monitoring and control of the accuracy and uncertainty of the results. We will lay the foundation for a language of labeling, which is a novel, consistent framework for describing labeling objectives in a hierarchical, programmatic manner. Increased availability of high reliability manual labels will improve the quality, consistency, and efficiency of quantitative MRI analyses in the brain in health and disease. PUBLIC HEALTH RELEVANCE: The proposed research investigates an alternative to expert raters for manual labeling of medical images through statistical analysis of the collaborative efforts of many, minimally- trained raters. The web-based system will dramatically reduce the cost of studying anatomical regions of interest by enabling collaboration among many individuals, while providing active monitoring and control of the accuracy and uncertainty of the results.