SHF: Medium: Scalable Holistic Autotuning for Software Analytics

软件分析将大量的低价值数据提炼成更小的高价值数据集，这些数据集为软件质量增强提供了重要的见解。它对于软件工作量估计、风险管理、缺陷预测、项目资源管理和许多其他任务都是必不可少的。 软件分析是一个复杂、耗时的过程。 最近的研究试图通过更好地利用现有计算资源的智能优化器来缓解这个问题。优化的可能选项空间非常大，并且跨越多个层：算法、编译器和执行时间选项的所有可能设置。更复杂的是，有许多相互竞争的目标可以用来指导这种调整;例如，减少CPU使用，同时提高学习模型的预测能力。现有的研究主要集中在有限的优化器，探索只有几个选项，在大多数一个级别，而试图改善只有一个或两个目标，留下的巨大潜力的优化untapped.This研究提出了先进的整体可扩展的智能优化软件分析（SHASA）的状态。SHASA同时为多个优化目标调整所有级别的选项。它通过开发一套新颖的技术来实现这一雄心勃勃的目标，这些技术可以有效地处理巨大的调谐空间。这些技术利用所有这些选项和目标之间的协同作用，通过利用相关性过滤（快速处理无用的选项），推理的局部性（使过时的调优能够更快地更新）和冗余减少（减少搜索空间以获得更好的调优）。 这项研究将产生算法和工具，这些算法和工具对软件分析研究来说更加有用和有效。这些技术可以推广到软件分析之外，用于计算科学和工程。 一个重要的更广泛的影响是最大限度地减少CPU和内存的使用，最终降低数据中心的能源消耗，因为数据分析计算的规模显着增长，计算要求越来越高。

Software analytics distills large quantities of low-value data down to smaller sets of higher value data that shed important insights for software quality enhancement. It is essential for software effort estimation, risk management, defect prediction, project resource management and many other tasks.  Software analytics is a  complex, time-consuming process.  Recent research has tried to alleviate the issue through intelligent optimizers  that make better use of existing computational resources. The space of possible options for optimization is very large, and spans over multiple layers: all possible settings for algorithms, compilers, and execution time options. To complicate matters, there are many competing goals that could be used to guide that tuning; e.g. reducing CPU usage while increasing the predictive power of the learned model. Existing research has mainly focused on limited optimizers that explore just a few options at mostly one level while trying to improve on just one or two goals, leaving the large potential of optimizations untapped.This research proposes to advance the state of the art to holistic scalable intelligent optimization for software analytics (SHASA). SHASA tunes all levels of options for multiple optimization objectives at the same time. It achieves this ambitious goal through the development of a set of novel techniques that efficiently handle the tremendous tuning space. These techniques take advantage of the synergies between all those options and goals by exploiting relevancy filtering (to quickly dispose of unhelpful options), locality of inference (that enables faster updates to outdated tunings) and redundancy reduction (that reduces the search space for better tunings).  This research will produce algorithms and tools that are demonstrably more useful and efficient for software analytics research. Those techniques are generalizable beyond software analytics for use in computational science and engineering at large.  An important broader impact is minimizing CPU and memory usage, ultimately reducing energy consumption in data centers, as data analytics computations grown significantly in scale and become computationally more demanding.