Autonomic Data Management for Very Large Dataset Visualization

适用于超大型数据集可视化的自主数据管理

• 批准号: EP/D059674/1
• 负责人: Min Chen
• 金额: $ 1.96万
• 依托单位: Swansea University
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2006
• 资助国家: 英国
• 起止时间: 2006 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FD059674%2F1
• 关键词: Autonomic; Data; Management; Very; Large

Never before in history have we had such capability for generating, collecting and storing digital data. Data repositories at terabyte level are becoming commonplace in many applications, including bioinformatics, medicine, remote sensing and nano-technology. In some applications, such as network traffic visualization and video visualization, we are encountering the scenario that dynamic data streams are almost temporally unbounded. Many visualization tasks are evolving into visual data mining processes.The main scientific challenge to be tackled in this proposed collaborative project is how to manage visualization tasks that involve very large datasets which must reside on one or a few remote data servers for various reasons, including excessive space requirements, data sharing provision, dynamic data generation, and data security. Because of the rapid developments of data capture and data generation technologies (e.g., web-based data collection and Grid computing), the management of such visualization over a complex computing and communication infrastructure is becoming a bottleneck to interactive visual data mining. It is not possible to device a uniform approach to suit various requirements nor cost-effective to place the burden of complex data management upon users.One emerging strategy for developing complex computing infrastructure is autonomic computing, which seeks inspiration in self-adaptive biological systems and self-governing social and economic systems. In a recent survey coordinated by the PI, the concept of autonomic computing was first brought into the context of visualization, and it was suggested that the concept can be deployed in a visualization infrastructure in an evolutionary manner. It is desirable to investigate into an autonomic approach to the management of very large datasets in visualization, to design and prototype knowledge-based framework for capturing dynamic system information and users' experiences, and to develop adaptive algorithms that can utilize such information dynamically for data management in visualization.The aim of this research is to develop an autonomic approach to the management of very large dataset visualization. The joint research programme will be carried out primarily by the PI, Professor Min Chen, and his PhD student, Mr. David Chisnall, in collaboration with Professor Charles Hansen and his research team in University of Utah.The main resources for conducting this joint programme is in place with a current PhD studentship held by Mr. Chisnall (January 2004 - December 2007). Some work on O1, O2, and O3 has been carried out. We therefore do not seek any funding for staffing and equipment costs. The funding sought for this project is for covering travelling and subsistence cost, as face-to-face meetings and short-term visits are indispensable in such close collaboration involving intensive algorithm design and software development, as well as direct access to computing infrastructure, datasets, and scientific users in Utah.

在历史上，我们从未有过这样的能力来生成、收集和存储数字数据。在生物信息学、医学、遥感和纳米技术等许多应用中，tb级别的数据存储库正变得越来越普遍。在一些应用中，例如网络流量可视化和视频可视化，我们会遇到动态数据流在时间上几乎是无界的情况。许多可视化任务正在演变为可视化数据挖掘过程。在这个拟议的协作项目中，需要解决的主要科学挑战是如何管理涉及非常大的数据集的可视化任务，这些数据集由于各种原因必须驻留在一个或几个远程数据服务器上，包括过多的空间需求、数据共享提供、动态数据生成和数据安全。由于数据捕获和数据生成技术的快速发展（例如，基于web的数据收集和网格计算），在复杂的计算和通信基础设施上对这种可视化的管理正在成为交互式可视化数据挖掘的瓶颈。不可能采用统一的方法来满足各种需求，也不可能将复杂数据管理的负担放在用户身上，从而节省成本。开发复杂计算基础设施的一个新兴策略是自主计算，它从自适应生物系统和自治的社会和经济系统中寻求灵感。在PI最近协调的一项调查中，自主计算的概念首次被引入到可视化的上下文中，并建议该概念可以以一种渐进的方式部署在可视化基础设施中。需要研究一种自治的方法来管理可视化中的大型数据集，设计和原型基于知识的框架来捕获动态系统信息和用户体验，并开发自适应算法，可以动态地利用这些信息进行可视化中的数据管理。本研究的目的是开发一种自治的方法来管理非常大的数据集可视化。联合研究计划将主要由首席研究员陈敏教授和他的博士生David Chisnall先生与犹他大学的Charles Hansen教授及其研究团队合作进行。开展这一联合项目的主要资源是目前由奇斯纳尔先生（2004年1月至2007年12月）持有的博士学位。对O1、O2和O3进行了一些研究。因此，我们不为人员和设备费用寻求任何资金。为该项目寻求的资金用于支付旅行和生活费用，因为面对面的会议和短期访问在这种涉及密集算法设计和软件开发的密切合作中是必不可少的，以及直接访问犹他州的计算基础设施、数据集和科学用户。

项目成果

Ray-driven dynamic working set rendering For complex volume scene graphs involving large point clouds

光线驱动的动态工作集渲染适用于涉及大型点云的复杂体积场景图

• DOI: 10.1007/s00371-006-0091-6
• 发表时间: 2007
• 期刊: The Visual Computer
• 作者: Chisnall D