BIGDATA: IA: DKA: Collaborative Research: High-Throughput Connectomics

连接组学是绘制神经元结构之间连接的科学，以帮助我们了解大脑如何工作。使用天文学的类比，连接组学研究人员希望建立“望远镜”，使科学家能够准确地观察大脑。然而，在天文学中，显微切片机和电子显微镜收集的原始数据，连接组学的仪器，太大了，无法有效存储，必须以非常高的计算速度进行分析。我们的目标是研究，开发和部署一个软件架构，使连接组学数据的高吞吐量分析的速度，它被收购。我们将开发第一个计算基础设施，以支持高通量连接组学，而无需人工干预。如果成功，该系统将首次允许绘制小型哺乳动物大脑（1立方毫米）的皮质柱，并有望在几年内绘制哺乳动物皮质的重要部分。连接组学大数据问题的解决方案是一种新的高吞吐量连接组学软件架构，我们称之为MapRecurse。MapRecurse之所以这样命名，是因为它与广泛使用的MapReduce框架有一些相似之处，它将提供一种统一的方式来指定要应用于收集的数据的计算步骤和验证测试的序列。MapRecurse的关键是能够以结构化的方式布局数据和计算，并保持局部性。使用它，程序员将能够将快速，不太准确的分割算法应用于低分辨率的数据，以便快速计算输出神经网络图的第一个版本。然后，特定领域的图论方法将检查图的正确性，并确定需要进一步改进的不一致区域。然后，MapRecurse将使用更慢，更准确的分割和重建算法进行自下而上的本地处理，以更高的数据分辨率，验证和纠正任何错误。迭代递归地进行，并在多个核心上并行进行，这就是该方法的名称。我们相信MapRecurse和这里开发的数据结构和算法将在其他高吞吐量应用中找到应用，例如天文学，生物学，社交媒体应用或经济学。

High-Throughput Connectomics Connectomics is the science of mapping the connectivity between neuronal structures to help us understand how brains work. Using the analogy of astronomy, connectomics researchers wish to build 'telescopes' that will allow scientists to accurately view the brain. However, as in astronomy, the raw data collected by microtomes and electron microscopes, the instruments of connectomics, is too large to store effectively, and must be analyzed at very high computation rates. Our goal is to research, develop, and deploy a software architecture that enables high-throughput analysis of connectomics data at the speed at which it is being acquired. We will develop the first computational infrastructure to support high-throughput connectomics without human intervention. If successful, this system will allow for the first time the mapping of a cortical column of a small mammalian brain (1 cubic millimeter), and hopefully within a few years the mapping of significant sections of a mammalian cortex. The solution to the big data problem of connectomics is a new high-throughput connectomics software architecture that we call MapRecurse. MapRecurse, named so because it bears some resemblance to the widely used MapReduce framework, will provide a unified way of specifying sequences of computational steps and validation tests to be applied to the collected data. Key to MapRecurse will be the ability to layout data and computation in a structured way that preserves locality. Using it, programmers will be able to apply fast, less accurate segmentation algorithms to low resolutions of the data in order to quickly compute a first version of the output neural network graph. Domain-specific graph theoretical methods will then check for correctness of the graph and identify areas of inconsistencies that are in need of further refinement. MapRecurse will then apply bottom-up, local processing with slower, more accurate segmentation and reconstruction algorithms to higher resolutions of the data, verifying and correcting any errors. The iterations progress recursively and in parallel across multiple cores, giving the approach its name. We believe that MapRecurse and the data structures and algorithms developed here will find applications in other high-throughput applications, such as, in astronomy, biology, social media applications, or economics.