BIGDATA: Small DCM: ESCA DA Computational infrastructure for massive neurosci

描述（由申请人提供）：理想情况下，当神经科学家收集数TB的图像堆栈时，数据会自动处理以供开放访问和分析。然而，虽然世界各地的几个实验室正在以前所未有的速度收集数据-每天高达TB-但仍然缺乏促进流数据密集型计算的计算技术。此外，部署数据密集型计算集群超出了大多数实验室的手段和能力。该项目将扩展、开发和部署这些技术。为了展示这些工具，我们将利用它们来支持正在进行的小鼠大脑结构（MBA）项目，该项目已经积累了超过0.5 PB的图像数据。这些数据集带来的主要计算挑战是规模问题。接下来的任务在不同的习得方式中仍然相对固定。到目前为止，收集这种规模数据的实验室几乎完全孤立，让他们为每个问题“重新发明轮子”。此外，现有的解决方案是不够的，原因有很多：它们通常包括许多依赖于手动数据输入的Excel电子表格，它们缺乏可扩展的科学数据库后端，并且它们运行在不是专门为手头的计算任务设计的临时集群上。我们的目标是通过在MBA项目管道中实施以下技术进步来增强当前的技术水平：（1）数据管理将由一个统一的系统组成，该系统可以自动捕获元数据、启动处理管道并在几分钟内而不是几小时内提供质量控制反馈。(2)数据处理任务将运行“核外”算法，适合其计算要求，包括细胞体和过程的配准，对齐和语义分割。(3)数据存储将自动构建数据库，用于存储多模态图像数据和从机器视觉算法中提取的注释。这些数据库将在空间上共同注册并存储在优化的异构计算集群上。(4)数据访问将自动提供给每个人，包括所有图像数据和数据衍生产品，通过Web服务，包括3D查看，下载和进一步处理。(5)数据分析将扩展适用于多尺度电路图的随机图模型。相关性（见说明）：在美国，神经系统疾病约占疾病总负担的30%。全脑神经解剖学可从大量的神经科学图像堆栈中获得被广泛认为是我们预防和治疗此类疾病的能力中的关键缺失环节。因此，该项目旨在通过开发和应用BIGDATA工具进行管理，存储，访问和分析来缩小这一差距。

DESCRIPTION (provided by applicant): Ideally, as neuroscientists collect terabytes of image stacks, the data are automatically processed for open access and analysis. Yet, while several labs around the world are collecting data at unprecedented rates- up to terabytes per day-the computational technologies that facilitate streaming data-intensive computing remain absent. Also deploying data-intensive compute clusters is beyond the means and abilities of most experimental labs. This project will extend, develop, and deploy such technologies. To demonstrate these tools, we will utilize them in support of the ongoing mouse brain architecture (MBA) project, which already has amassed over 0.5 petabytes (PBs) of image data. The main computational challenges posed by these datasets are ones of scale. The tasks that follow remain relatively stereotyped across acquisition modalities. Until now, labs collecting data on this scale have been almost entirely isolated, left to "reinvent the wheel" for each of these problems. Moreover, the extant solutions are insufficient for a number of reasons: they often include numerous excel spreadsheets that rely on manual data entry, they lack scalable scientific database backends, and they run on ad hoc clusters not specifically designed for the computational tasks at hand. We aim to augment the current state of the art by implementing the following technological advancements into the MBA project pipeline: (1) Data Management will consist of a unified system that automatically captures metadata, launches processing pipelines, and provides quality control feedback in minutes instead of hours. (2) Data Processing tasks will run algorithms "out-of-core", appropriate for their computational requirements, including registration, alignment, and semantic segmentation of cell bodies and processes. (3) Data Storage will automatically build databases for storing multimodal image data and extracted annotations learned from the machine vision algorithms. These databases will be spatially co-registered and stored on an optimized heterogeneous compute cluster. (4) Data Access will be automatically available to everyone-including all the image data and data derived products-via Web-services, including 3D viewing, downloading, and further processing. (5) Data Analytics will extend random graph models suitable for multiscale circuit graphs. RELEVANCE (See instructions): Nervous system disorders are responsible for approximately 30% of the total burden of illness in the United States. Whole brain neuroanatomy-available from massive neuroscientific image stacks-is widely believed to be a key missing link in our ability to prevent and treat such illnesses. Thus, this project aims to close this gap via the development and application of BIGDATA tools for management, storage, access, and analytics.