Performance Evaluation of On-Demand Provisioning of Data Intensive Applications

该项目正在研究数据密集型应用程序配置的动态策略的有效性，方法是使用NSF CLUE工具和ApacheHadoop编程环境对替代配置策略进行彻底的性能评估。当前的系统采用“一刀切”和相对静态的解决方案方法，即使它们服务于具有广泛的访问和处理需求的应用程序。研究中的参考数据集是来自机载激光雷达测量的高分辨率地形数据集。正在评估使用并行数据库技术的替代解决方案与Hadoop环境的性能。将并行数据库方法与基于Hadoop的方法相结合的混合策略--例如基于用户指令和/或工作负载分析--也在测试和评估中。这项研究将有助于理解数据密集型应用程序的动态预配置策略中的性能权衡。潜在的影响是，根据按需和动态办法提供数据集，而不是目前的静态办法，重新评估如何实施数据档案和向广大用户提供数据集。这项研究的结果将包括全面的性能评估和关于最佳利用大型集群计算环境来支持数据密集型应用程序的建议，以及对动态、混合的数据管理方法的评估。结果将通过专业会议和期刊传播。推荐的方法将在真实的数据密集型环境中实施，例如OpenTopgraph y.org门户网站。

This project is studying the effectiveness of dynamic strategies for provisioning data intensive applications by conducting a thorough performance evaluation of alternative provisioning strategies using the NSF CluE facility and the Apache Hadoop programming environment. Current systems adopt a "one size fits all" and relatively static solution approach even as they serve applications with a wide range of access and processing needs. The reference data sets in the study are high-resolution topographic data sets from airborne LiDAR surveys. The performance of alternative solutions using parallel database technology versus the Hadoop environment is being evaluated. Hybrid strategies, which blend the parallel database approach with the Hadoop-based approach?based, for example, on user directives and/or workload analysis?are also being tested and evaluated. The research will contribute to an understanding of the performance tradeoffs in dynamic provisioning strategies for data intensive applications. The potential impact is a reassessment of how data archives are implemented and data sets served to a broad user community based on on-demand and dynamic approaches to provisioning data sets, as opposed to the current static approaches. The results from this study will include a thorough performance evaluation and recommendations on the best use of large cluster computing environments for supporting data intensive applications, and an evaluation of a dynamic, blended approach to data management. Results will be disseminated via professional conferences and journals. The recommended approaches will be implemented in real data intensive environments, such as the OpenTopography.org portal.