MRI: Development of a System for Interactive Analysis and Visualization of Multi-Terabyte Datasets

该项目创建了一种新的集群计算机设计，旨在以交互方式探索数十兆兆字节的数据集，支持对海量数据集的分析和可视化。研究工作主要集中在基因组学和生物信息学、宇宙学和地球物理学以及计算流体力学(湍流和恒星对流)上。为了近乎实时地渲染其渲染信息需要高达2TB的同时活动存储的数据集，将使用一个由80个具有大量磁盘空间的双处理器节点组成的大型系统来执行可视化任务。可视化系统将为在多个时间片上探索性地查看多个变量之间的相互作用提供略大的回旋余地。出于交互探索海量数据集的需求，该系统支持交互分析和可视化，该交互分析和可视化沿着任何交互指定和/或修改的路径穿过4个维度来检查沿该路径的所有数据，方法是以与节点对数据执行重要分析和/或可视化操作的能力匹配的速率将这些数据以与LCSE的13M像素Power Wall一样大的观看屏幕上的高分辨率显示给用户。体积可视化以及各种模式匹配和互相关都是这些数据的目标。设计目标需要在科学可视化、数据管理、数据存储和集群数据共享以及基于优先级的作业调度方面进行大量研究。广泛的影响：该系统的开发涉及合作学科的许多学生和博士后。该系统将用于教学和研究，从而获得更高的学位。明尼苏达州克洛奎特的Fond Du Lac部落和社区学院的计算机科学系也将从这个项目中受益，这是因为计算科学与工程实验室(LCSE)与这所美洲原住民服务学院现有的合作。LCSE还分享了其技术开发和软件。再一次，开发的软件将以开放源码格式在网上提供。

This project, creating a new cluster computer design targeted at interactive exploration of datasets tens of terabytes in scale, supports analysis and visualization of massive datasets. Research efforts concentrate mainly on genomics and bioinformatics, cosmology and geophysics, and computational fluid dynamic (turbulence and stellar convections). For the purpose of rendering, in near real-time, datasets whose render information requires up to 2TB of simultaneously active storage, a large system consisting of 80 dual processor nodes with a massive amount of disk space will be used for the visualization task. The visualization system will allow slightly more leeway for exploratory viewing of interactions among multiple variables at multiple time slices. Motivated by the need to interactively explore voluminous datasets, the system supports interactive analysis and visualization that follows any interactively specified and/or modified path through the 4 dimensions examining all the data along that path by streaming this data, potentially replicated at each of 80 network nodes, into the memories at a rate that matches the node's ability to perform significant analysis and/or visualization operations on the data and to display the results for the user at high resolution on viewing screens as large as the LCSE's 13 Mpixels Power Wall. Volume visualization and various kinds of pattern matching and cross correlation target this data. The design goals require significant research in scientific visualization, data management, data storage and cluster data sharing, as well as prority-based job scheduling.Broader Impacts: The development of this system involves many students and postdocs in the collaborating disciplines. The system will be used in instruction as well as in research leading to advance degrees. The computer science department at Fond du Lac Tribal and community College in Cloquet Minnesota will also benefit from this project due to the existing collaboration of the Laboratory for Computational Science and Engineering (LCSE) with this Native American serving college. LCSE has also shared its technology developments and software. Once again, the developed software will be made available over the Web in Open source format.