Web-Based Infrastructure for Comparison and Validation of Image Computing Methods

摘要 我们建议为算法评估开发基础设施并部署商业安装 服务(AES)。这项服务将有助于弥合算法研究人员和商业产品之间的差距 开发商。它将(1)帮助商业公司确定哪些医学图像分析和信息学 他们应该将算法集成到他们的产品中，以及(2)为算法研究人员提供更好的访问权限 与临床相关的数据量，并更好地了解临床和商业需求。 具体地说，AES将提供一项服务，商业组织可以通过该服务发布临床数据分析 挑战(与预期产品相关的业绩里程碑相关的数据、指标和奖励)以及 研究人员可以很容易地将这些挑战纳入他们的算法开发工作流程中。 我们成功的第一阶段资助达到了顶峰，我们的原型系统成熟并作为在线 多模式脑瘤分割(BRAT)的基础设施在MICCAI 2012和 ISBI 2013年的前列腺分割大挑战。 在此，我们建议(目标1)扩展我们的系统以支持一种新的算法提交机制 基于解决临床集成的虚拟机技术(即，多步骤数据处理， 包括人工交互)、安全性和计算资源可伸缩性，以支持广泛的测试。我们 将(目标2)扩展现有的软件开发工具(即我们流行的CMake构建系统)以使 向AES提交算法挑战了算法开发的一个固有且毫不费力的部分 研究人员。我们将(目标3)使用额外的重大挑战来验证生成的系统，并且我们将 作为拟议工作的一部分，将其交付给我们的第一个商业客户并从他们那里接收反馈。具体来说， 两个学术团体(俄亥俄州立大学和犹他大学)已经同意进行 挑战使用我们的系统。此外，一个商业团体(成像终端)已同意担任 我们的第一个商业客户。他们是一个成像核心实验室，提供算法解决方案 制药公司和临床研究机构。他们将使用我们的AES来发布客户的数据 和指标，提供奖励，从而吸引算法开发人员为他们的客户生成解决方案 有问题。 人们普遍认为，算法研究人员之间存在着鸿沟，医学能力 设备，以及临床实践的需要。拟议中的工作将有助于弥合这一鸿沟，并将作为 一个可行的商业模式。

Abstract We propose to develop the infrastructure for and deploy a commercial installation of an Algorithm Evaluation Service (AES). The service will help bridge the gap between algorithm researchers and commercial product developers. It will (1) assist commercial company in determining which medical image analysis and informatics algorithms they should integrate into their products, and (2) provide algorithm researchers with better access to clinically relevant amounts of data and with a better understanding of clinical and commercial needs. Specifically, the AES will provide a service whereby commercial organizations can post clinical data analysis challenges (data, metrics, and awards tied to performance milestones related to their intended products) and researchers can easily incorporate those challenges into their algorithm development workflows. Our successful Phase I grant culminated with our prototype system maturing and serving as the online infrastructure for the Multimodal Brain Tumor Segmentation (BRATS) Grand Challenge at MICCAI 2012 and the Prostate Segmentation Grand Challenge at ISBI 2013. Herein, we propose to (Aim 1) extend our system to support a novel mechanism for algorithm submission based on virtual machine technology that addresses clinical integration (i.e., multi-step data processing, including human interaction), security, and computational resource scalability to support extensive testing. We will (Aim 2) extend existing software development tools (i.e., our popular CMake build system) to make the submission of algorithms to AES challenges an inherent and effortless part of algorithm development for researchers. We will (Aim 3) validate the resulting system using additional grand challenges, and we will deliver it to and receive feedback from our first commercial customer as part of the proposed work. Specifically, two academic groups (Ohio State University and The University of Utah) have agreed to conduct grand challenges using our systems. Additionally, a commercial group (Imaging Endpoints) has agreed to serve as our first commercial customer. They are an imaging core lab that provides algorithmic solutions to pharmaceutical companies and clinical research organizations. They will use our AES to post a client's data and metrics, offer a prize, and thereby attract algorithm developers to generate solutions to their client's problem. It is generally accepted that a chasm exists between algorithm researchers, the capabilities of medical devices, and the needs of clinical practice. The proposed work will help bridge that chasm and will operate as a viable business model.