EAGER: Scalable Big Data Analytics

大数据分析需要弥合数据密集型计算和数据驱动计算之间的差距，以获得可操作的见解。前者主要侧重于优化数据移动、重用、组织和存储，而后者侧重于假设驱动的、自下而上的数据发现，这两个领域在某种程度上是独立发展的。这个探索性项目旨在研究一个整体生态系统，该生态系统可以优化来自模拟、传感器或业务流程的数据生成（事务步骤）；组织该数据（可能与其他数据结合），以便对下游数据分析进行简化和预处理（组织步骤）；从这些数据执行知识发现、学习和挖掘模型（预测步骤）；并导致行动（例如，改进模型、新实验、建议）（反馈步骤）。智力优势：与目前孤立地考虑每个步骤的优化不同，该项目将大数据分析的整个生态系统的可扩展性和优化作为设计策略的一部分。该项目旨在考虑大数据在设计算法、软件、分析和数据管理方面的挑战。这种策略与传统方法形成对比，传统方法首先为小数据设计算法，然后将其扩展。该项目旨在在设计和实现算法、软件和应用程序时，将数据复杂性、计算需求和数据访问模式作为一个整体来对待。更广泛的影响：该项目可以在气候信息学和社交媒体分析等许多关键应用中推进大数据分析的最新技术。该项目产生的软件将在开源许可下提供给更广泛的科学界。该项目为西北大学研究生和博士后研究人员的教育和培训提供了更多的机会。

Big Data analytics requires bridging the gap between data-intensive computing and data-driven computing to obtain actionable insights. The former has primarily focused on optimizing data movement, reuse, organization and storage, while the latter has focused on hypothesis-driven, bottom-up data-to-discovery and the two fields have evolved somewhat independently. This exploratory project aims to investigate a holistic Ecosystem that optimizes data generation from simulations, sensors, or business processes (Transaction Step); organizes this data (possibly combining with other data) to enable reduction, pre-processing for downstream data analysis (Organization Step); performs knowledge discovery, learning and mining models from this data (Prediction Step); and leads to actions (e.g., refining models, new experiments, recommendation) (Feedback Step). Intellectual Merit: As opposed to the current practice of considering optimizations in each step in isolation, the project considers scalability and optimizations of the entire Ecosystem for big data analytics as part of the design strategy. The project aims to consider big data challenges in designing algorithms, software, analytics, and data management. This strategy contrasts with traditional approaches that first design algorithms for small data sizes and then scale them up. The project aims to treat data complexity, computational requirement, and data access patterns as a whole when designing and implementing algorithms, software and applications. Broader Impacts: The project could advance the state of the art in big data analytics across a number of key applications such as Climate Informatics and Social Media Analytics. The software resulting from the project is being made available to the broder scientific community under open source license. The project offers enhanced opportunities for education and training of graduate students and postdoctoral researchers at Northwestern University.