Flash Gordon: A Data Intensive Computer

摘要UCSD 0910847 Norman，Michael L.该项目支持一个适合于数据密集型应用的新超级计算系统的获取、部署和操作。 该系统被称为Flash Gordon，将由位于圣地亚哥的加州大学超级计算机中心部署，并集成到TeraGrid中。 该系统由Appro International Incorporated与合作伙伴Intel和ScaleMP设计，旨在弥合现代计算系统中主内存和旋转磁盘存储之间不断扩大的延迟差距。 它使用闪存来提供高密度、经济实惠、低延迟的存储，可以将其配置为扩展交换空间或非常快的文件系统。 该系统将由非常大的共享虚拟内存，高速缓存一致的“超级节点”，以支持一套通用的编程范式。 Flash Gordon的大容量可寻址虚拟内存、低延迟闪存和用户友好的编程环境将为数据密集型应用程序提供更高的性能，这些应用程序在当前的大规模架构上可扩展性较差，为许多研究领域的变革性研究提供了资源。 即使是顺序代码也能够寻址高达TB的快速虚拟内存。如提案所述，这一资源将使研究人员能够应对的科学挑战包括以下方面。从头基因组组装：基因测序仪产生有关基因组许多小片段的信息。 一些最近的汇编算法使用基于图的方法，在共享内存系统上更容易执行。 使用Flash Gordon，研究人员将能够快速组装复杂的基因组，如哺乳动物基因组。天文学：现代天文学数据库可能很大;例如，斯隆数字巡天的大小约为6 TB。 通常，研究人员用来执行复杂的天文现象搜索的分析算法可以更容易地在共享内存系统上实现。 Flash Gordon将使研究人员能够将斯隆数字巡天的副本加载到与超级节点相关联的闪存中，极大地扩展了天文学家可以进行的分析类型。天体物理学：宇宙学模拟产生描述模拟宇宙的许多TB的输出。 对这些模拟结果的详细分析，以发现诸如坍缩晕、星系合并、矮星系和星系团等特征，通常需要基于密度的聚类分析，而这些聚类分析不能很好地并行化。 使用Flash Gordon，这些分析可以通过利用大型SMP分区和快速闪存来加速。交互网络：交互网络，表示对象之间关系的图形，用于流行病学，遗传学，系统生物学和种群生物学等领域的研究。 这些交互网络可以表示存储在不同数据库中的数据类型之间的关系;例如，将社交网络数据库与医疗记录和基因组图谱相结合，以探索遗传抗病性等问题。 Flash Gordon将加速大型交互网络的分析，因为数据库可以存储在固态磁盘上，大大减少访问时间，并允许更复杂的分析类型。项目团队将利用UCSD正在进行的一些教育活动来扩大和多样化可以利用这种计算资源的用户社区，其中包括为科学和工程领域代表性不足的妇女和少数民族成功开展的外展计划。 该项目还将为本科生创建一个暑期培训计划。

ABSTRACT UCSD 0910847 Norman, Michael L.This project supports the acquisition, deployment and operation of a new supercomputing system suitable for data-intensive applications. The system, to be known as Flash Gordon, will be deployed by the University of California at San Diego at the San Diego Supercomputer Center and integrated into the TeraGrid. The system, which has been designed by Appro International Incorporated, with partners Intel and ScaleMP, seeks to bridge the widening latency gap between main memory and rotating disk storage in modern computing systems. It uses flash memory to provide a level of dense, affordable, low-latency storage that can be configured as either extended swap space or a very fast file system. The system will consist of very large shared virtual-memory, cache-coherent "super-nodes" to support a versatile set of programming paradigms. Peak performance will exceed 200 teraflops/s in double precision.Flash Gordon's large addressable virtual memory, low-latency flash memory, and user-friendly programming environment will provide a step-up in capability for data-intensive applications that scale poorly on current large-scale architectures, providing a resource that will enable transformative research in many research domains. Even sequential codes will be able to address up to terabytes of fast virtual memory. Examples of scientific challenges, as described in the proposal, that this resource will allow researchers to tackle, include the following. De Novo Genome Assembly: Gene sequencers produce information about many small fragments of a genome. Some recent assembly algorithms use a graph-based approach, much more readily executed on a shared-memory system. Using Flash Gordon, researchers will be able to rapidly assemble complex genomes such as mammalian genomes.Astronomy: Modern astronomy databases can be large; for example, the Sloan Digital Sky Survey is approximately six terabytes in size. Typically, the analysis algorithms that researchers use to perform complex searches for astronomical phenomena can be implemented more easily on shared-memory systems. Flash Gordon will enable researchers to load a copy of the Sloan Digital Sky Survey into the flash memory associated with a super-node, greatly extending the types of analyses astronomers can make.Astrophysics: Cosmological simulations produce many terabytes of output describing the simulated universe. Detailed analysis of the results of these simulations, to find features such as collapsed halos, galaxy mergers, dwarf galaxies, and galaxy clusters, often requires density-based cluster analysis that does not parallelize well. With Flash Gordon, these analyses can be accelerated by exploiting the large SMP partitions and fast flash memory.Interaction Networks: Interaction networks, graphs representing the relationships between objects, are used in research in areas such as epidemiology, phylogenetics, systems biology, and population biology. These interaction networks can represent relationships between types of data stored in different databases; for example, the combination of social network databases with medical records and genomic profiles to explore questions such as genetic resistance to disease. Flash Gordon will speed analysis of large interaction networks because the databases can be stored on the solid-state disks, greatly reducing access time and permitting more complex types of analysis.The project team will leverage a number of ongoing educational activities at UCSD to expand and diversify the community of users that can utilize this computational resource, including successful outreach programs for women and minorities from underrepresented groups in science and engineering. The project will also create a summer training program for undergraduates.