III: Small: RanKloud: Data Partitioning and Resource Allocation Strategies for Scalable Multimedia and Social Media Analysis

今天，多媒体数据在电子商务、监控、教育、网络服务和社交媒体等广泛的应用中被大量生产和使用，具有显著的经济效益和社会效益。因此，迫切需要系统来提供对大型媒体数据集合的高度可扩展的处理和高效的分析。在本研究项目中开发的RanKout原型系统，专注于在基于云的可扩展环境中处理大量多媒体数据的应用的需求和要求。大多数多媒体应用共享一些核心操作，包括集成/融合、分类、聚类、图分析、近邻搜索和相似性搜索。然而，当以幼稚的方式执行时，这些核心操作的成本往往非常高，因为需要考虑的对象数量和对象特征可能会令人望而却步。避免这一成本需要避免多余的工作。这项研究的重点是下一代基于云的海量媒体处理和分析系统，其中支配其设计的基本原则包括对特定分析任务的数据和功能的效用的认识。合并用于执行特定实用程序任务的数据和特征实用程序预计将显著降低分析任务的总体成本。RanKout项目的研究计划包括：(1)数据模型和查询语言，以规范多媒体数据处理工作流程；(2)适应性强、秩知晓的并行多媒体数据处理原语；(3)运行时数据采样策略，以支持数据和资源的适配；(4)避免浪费和不平衡的策略，用于效用感知的数据划分、资源分配和增量批处理。RanKout弥合了我们对基于云计算的总体理解，特别是多媒体数据的高效处理的重要差距。预计这些结果将使新的工具和系统能够支持可伸缩性，以解决内容感知多媒体和社交媒体分析中的一大类问题，并对网络智能、商业智能以及科学和传感器应用产生影响，所有这些应用都需要处理不精确的多媒体数据，以便更有效地做出决策。该项目为研究生和本科生提供研究经验的机会，包括课程中的研究成果和挑战，包括Capstone项目。亚利桑那州立大学(亚利桑那州立大学)通过国家认可的住宿荣誉学院和少数族裔进入研究职业计划招收高质量的本科生。项目成果的国内和国际传播包括重要的会议和期刊出版物，以及兰克劳网站(http://aria.asu.edu/rankloud).)上的开放源码软件许可证

Today, multimedia data are produced and consumed in massive quantities in a broad range of applications with significant economic and societal benefit, including e-commerce, surveillance, education, web services, and social media. Hence, there is an urgent need for systems to provide highly scalable processing and efficient analysis of large media data collections. The RanKloud prototype system, developed in this research project, focuses on the needs and requirements of applications that deal with large quantities of multimedia data in a cloud-based scalable environment. Most multimedia applications share a few core operations, including integration/fusion, classification, clustering, graph analysis, near-neighbor search, and similarity search. When performed naively, however, these core operations are often very costly, because the number of objects and object features that need to be considered can be prohibitive. Avoiding this cost requires that redundant work is avoided. This research focuses on the next generation cloud-based massive media processing and analysis systems where the fundamental principles that govern their design include an awareness of the utilities of data and features to a particular analysis task. Incorporating data and feature utilities for performing a particular utility task is expected to significantly reduce the overall cost of the analysis task. The RanKloud project research plan includes: (1) data model and query language to specify multimedia data processing workflows; (2) adaptable, rank-aware parallel multimedia data processing primitives; (3) run-time data sampling strategies to support adaptation to data and resource; and (4) waste- and unbalance-avoidance strategies for utility-aware data partitioning, resource allocation, and for incremental batched processing.RanKloud bridges an important gap in our understanding of cloud-based computing in general, and efficient processing of multimedia data in particular. The results are expected to enable new tools and systems supporting scalability in a large class of problems in content-aware multimedia and social media analysis with impact in web intelligence, business intelligence, and scientific and sensor applications all of which need to handle imprecise multimedia data for more effective decision making. This project provides research experience opportunities for graduate and undergraduate students and includes research results and challenges in courses, including Capstone projects. Arizona State University (ASU) recruits top-quality undergraduates through a nationally recognized residential Honors College and the Minority Access to Research Careers program. The national and international dissemination of the project results includes premier conference and journal publications, as well as open source software licenses at the RanKloud Web site (http://aria.asu.edu/rankloud).