Web-Based Infrastructure for Comparison and Validation of Image Computing Methods

点击翻译按钮获取中文摘要

Abstract We propose to develop the infrastructure for and deploy a commercial installation of an Algorithm Evaluation Service (AES). The service will help bridge the gap between algorithm researchers and commercial product developers. It will (1) assist commercial company in determining which medical image analysis and informatics algorithms they should integrate into their products, and (2) provide algorithm researchers with better access to clinically relevant amounts of data and with a better understanding of clinical and commercial needs. Specifically, the AES will provide a service whereby commercial organizations can post clinical data analysis challenges (data, metrics, and awards tied to performance milestones related to their intended products) and researchers can easily incorporate those challenges into their algorithm development workflows. Our successful Phase I grant culminated with our prototype system maturing and serving as the online infrastructure for the Multimodal Brain Tumor Segmentation (BRATS) Grand Challenge at MICCAI 2012 and the Prostate Segmentation Grand Challenge at ISBI 2013. Herein, we propose to (Aim 1) extend our system to support a novel mechanism for algorithm submission based on virtual machine technology that addresses clinical integration (i.e., multi-step data processing, including human interaction), security, and computational resource scalability to support extensive testing. We will (Aim 2) extend existing software development tools (i.e., our popular CMake build system) to make the submission of algorithms to AES challenges an inherent and effortless part of algorithm development for researchers. We will (Aim 3) validate the resulting system using additional grand challenges, and we will deliver it to and receive feedback from our first commercial customer as part of the proposed work. Specifically, two academic groups (Ohio State University and The University of Utah) have agreed to conduct grand challenges using our systems. Additionally, a commercial group (Imaging Endpoints) has agreed to serve as our first commercial customer. They are an imaging core lab that provides algorithmic solutions to pharmaceutical companies and clinical research organizations. They will use our AES to post a client's data and metrics, offer a prize, and thereby attract algorithm developers to generate solutions to their client's problem. It is generally accepted that a chasm exists between algorithm researchers, the capabilities of medical devices, and the needs of clinical practice. The proposed work will help bridge that chasm and will operate as a viable business model.