Eager Maps and Lazy Folds for Graph-Structured Applications

点击翻译按钮获取中文摘要

This project is investigating linguistic extensions to map/reduce abstractions for programming large-scale distributed systems, with special focus on applications that manipulate large, unstructured graphs. It targets real-world graph analysis tasks found in comparative analysis of biological networks as an important case study.The project is investigating the following specific questions: (i) how can highly unstructured graph-based formalisms be cast in the map/reduce framework? (ii) how effectively can these specifications leverage existing map/reduce infrastructure? (iii) how can these abstractions and their execution environments be enhanced to provide the semantic expressiveness necessary for programmability and scalable performance? (iv) how can these analysis tasks be integrated into comprehensive scientific resources usable by the wider applications community? Answers to these questions entails exploring linguistic extensions to existing map/reduce abstractions, defining new implementations on wide-area multicore/SMP platforms, and crafting an expressive graph analysis toolkit suitable for realistic deployment in important domains such as systems biology.Results that arise from this project advance the state-of-the-art in analysis of large sparse unstructured graphs and directly impact a very broad class of scientific applications. Beyond specific target applications in biology, graph-based formalisms find direct applications in social sciences (social networks), recommender systems, and commerce (networks of transactions).