The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience.
The Open Connectome Project Data Cluster: Scalable Analysis and Vision for High-Throughput Neuroscience.
复制标题
DOI：
10.1145/2484838.2484870
复制
发表时间：
2013
期刊：
影响因子：
--
通讯作者：
Vogelstein RJ
中科院分区：
文献类型：
--
作者：
Burns R;Roncal WG;Kleissas D;Lillaney K;Manavalan P;Perlman E;Berger DR;Bock DD;Chung K;Grosenick L;Kasthuri N;Weiler NC;Deisseroth K;Kazhdan M;Lichtman J;Reid RC;Smith SJ;Szalay AS;Vogelstein JT;Vogelstein RJ
我们描述了一个可扩展的数据库集群，用于高通量脑成像数据的空间分析和注释，最初用于 3D 电子显微镜图像堆栈，但也适用于时间序列和多通道数据。该系统主要针对使用高性能计算集群上并行执行计算机视觉算法来构建连接体（大脑的神经连接图）的工作负载而设计。这些服务和开放科学数据集可在 openconnecto.me 上公开获取。系统设计继承了很多NoSQL横向扩展和数据密集型计算架构。我们通过对空间索引进行分区来将数据分发到集群节点。我们将 I/O 定向到不同的系统（读取并行磁盘阵列并写入固态存储），以避免 I/O 干扰并最大化吞吐量。所有编程接口都是RESTful Web服务，简单且无状态，提高了可扩展性和可用性。我们包括对生产系统的性能评估，强调空间数据组织的有效性。
We describe a scalable database cluster for the spatial analysis and annotation of high-throughput brain imaging data, initially for 3-d electron microscopy image stacks, but for time-series and multi-channel data as well. The system was designed primarily for workloads that build connectomes— neural connectivity maps of the brain—using the parallel execution of computer vision algorithms on high-performance compute clusters. These services and open-science data sets are publicly available at openconnecto.me. The system design inherits much from NoSQL scale-out and data-intensive computing architectures. We distribute data to cluster nodes by partitioning a spatial index. We direct I/O to different systems—reads to parallel disk arrays and writes to solid-state storage—to avoid I/O interference and maximize throughput. All programming interfaces are RESTful Web services, which are simple and stateless, improving scalability and usability. We include a performance evaluation of the production system, highlighting the effec-tiveness of spatial data organization.