SHF: Small: Developing a Highly Efficient and Accurate Approximation System for Warehouse-Scale Computers with the Sub-dataset Distribution Aware Approach

尽管今天的仓库级计算机提供了巨大的数据处理能力，但从一个大数据集中获得一个特别的查询答案仍然具有挑战性。为了解决这个问题，近年来出现了一种趋势，即利用近似计算在更小的原始数据样本上实现更快的执行，方法是在合理程度上牺牲结果的准确性。基于离线的采样方法和基于在线的聚类采样解决方案已逐步应用于现实世界，以加速大数据查询。教育方面的好处来自于拓宽学生的经验，从一个排名靠前的西班牙裔博士学位授予机构和加强计算机科学/工程课程活动。这门关于仓库规模计算机和大数据的在线跨机构本科选修课程将有助于提供一种重新想象的学习体验，这种学习体验将最佳地利用当今的技术，并辅以来自三所不同大学的学生提供的广泛的富含媒体的学习材料。在开发一个集成硬件和软件，可扩展的近似系统有很大的困难。主要的挑战是在给定的错误范围内最小化访问数据的总大小及其相关的I/O开销。现有流行的等概率聚类抽样解决方案不能很好地处理许多遵循非均匀分布的实际应用。为了解决这些问题，本研究通过探索新的子数据分布感知方法来捕获非均匀类型的子数据分布，采用不等概率聚类采样来解决非均匀子数据分布导致的采样效率低下和方差大的问题，并考虑采样过程的独特性来匹配计算机硬件特征。例如SSD阵列，以释放其全部潜力。该研究将确保未来的大数据近似系统能够实现高速度的大数据分析，从而彻底改变人们与世界互动的方式；并通过高效、有效的数据处理提高高生产率对经济的影响。

Despite the fact that today's warehouse-scale computers supply enormous data processing capacity, getting an ad-hoc query answer from a big dataset remains challenging. To attack the problem, recent years have seen one trend to exploit approximate computing to achieve faster execution on a much smaller sample of the original data by sacrificing result accuracy to a reasonable extent. Both offline based sampling approaches and online cluster sampling solutions have been gradually deployed in a real world to accelerate big data query. Educational benefits arise from broadening the experience of students from a top ranked Hispanic Ph.D. degree awarding institution and enhanced computer science/engineering curriculum activities. The online cross-institution undergraduate elective course about warehouse-scale computer and big data will be helpful in providing a re-imagined learning experience that makes optimum use of today's technologies supplemented by a broad range of media-rich study materials that students from three different universities. There are major difficulties in developing an integrated hardware and software, scalable approximation system.  The main challenge is to minimize the total size of accessed data and its associative I/O overhead subject to a given error bound. Existing popular cluster sampling with equal probability solutions do not deal well with many real-world applications following a non-uniform distribution. This research aims to tackle those challenges by investigating new sub-dataset distribution aware methods to capture sub-dataset distributions especially for non-uniform types, applying cluster sampling with unequal probability to address the inefficient sampling and large variance problem caused by non-uniform sub-dataset distribution, and taking into account the unique properties of sampling process to match with the computer hardware features, such as SSD arrays to unleash their full potential. The research will ensure future big data approximation system enables high velocity of big-data analytics to revolutionize the way that people interact with the world; and high productivity improvement of the economic impact through the efficient and effective data processing.