Performance Evaluation of On-Demand Provisioning of Data Intensive Applications

该项目正在研究动态策略的有效性，通过使用NSF CluE设施和Apache Hadoop编程环境对替代供应策略进行全面的性能评估来提供数据密集型应用程序。当前的系统采用“一刀切”和相对静态的解决方案方法，即使它们服务于具有广泛访问和处理需求的应用。研究中的参考数据集是来自机载激光雷达测量的高分辨率地形数据集。正在评估使用并行数据库技术与Hadoop环境的替代解决方案的性能。混合策略，将并行数据库方法与基于Hadoop的方法相结合？例如，基于用户指令和/或工作负载分析？也在测试和评估中。该研究将有助于了解数据密集型应用程序的动态配置策略的性能权衡。潜在的影响是重新评估如何实施数据档案，以及如何根据按需和动态方法提供数据集，而不是目前的静态方法，向广大用户提供数据集。这项研究的结果将包括一个全面的性能评估和建议，最好地利用大型集群计算环境，以支持数据密集型应用程序，并评估一个动态的，混合的方法来管理数据。研究结果将通过专业会议和期刊传播。建议的方法将在真实的数据密集型环境中实施，例如OpenTopography.org门户网站。

This project is studying the effectiveness of dynamic strategies for provisioning data intensive applications by conducting a thorough performance evaluation of alternative provisioning strategies using the NSF CluE facility and the Apache Hadoop programming environment. Current systems adopt a "one size fits all" and relatively static solution approach even as they serve applications with a wide range of access and processing needs. The reference data sets in the study are high-resolution topographic data sets from airborne LiDAR surveys. The performance of alternative solutions using parallel database technology versus the Hadoop environment is being evaluated. Hybrid strategies, which blend the parallel database approach with the Hadoop-based approach?based, for example, on user directives and/or workload analysis?are also being tested and evaluated. The research will contribute to an understanding of the performance tradeoffs in dynamic provisioning strategies for data intensive applications. The potential impact is a reassessment of how data archives are implemented and data sets served to a broad user community based on on-demand and dynamic approaches to provisioning data sets, as opposed to the current static approaches. The results from this study will include a thorough performance evaluation and recommendations on the best use of large cluster computing environments for supporting data intensive applications, and an evaluation of a dynamic, blended approach to data management. Results will be disseminated via professional conferences and journals. The recommended approaches will be implemented in real data intensive environments, such as the OpenTopography.org portal.