SHF: Small: Enabling New Machine-Learning Usage Scenarios with Software-Defined Hardware for Symbolic Regression

点击翻译按钮获取中文摘要

Despite the widespread success of machine learning, existing techniques have limitations and/or unattractive trade-offs that prohibit important usage scenarios, particularly in embedded and real-time systems. For example, artificial neural nets provide sufficient accuracy for many applications, but can be too computationally expensive for embedded usage and may require large training data sets that are impractical to collect for some applications. Even when executed with cloud computing, neural nets often require graphics-processing unit acceleration, which greatly increases power costs that can already dominate the total cost of ownership in large-scale data centers and supercomputers. Similarly, linear regression is a widely used machine-learning technique, but generally requires model specification or guidance by the user, which is prohibitive for difficult-to-understand phenomena and/or many-dimensional problems. This project shows that symbolic regression complements existing machine-learning techniques by providing attractive Pareto-optimal trade-offs that enable new machine-learning usage scenarios where existing technologies are prohibitive. These symbolic-regression benefits come from three key advantages: 1) automatic model discovery, 2) computational efficiency with minimal loss in capability compared to existing techniques, and 3) lower sensitivity to training set size. Despite being studied for decades, symbolic regression is generally limited to toy examples due to the challenge of searching an infinite solution space with numerous local optima. This project presents a solution that significantly advances the state-of-the-art via two primary contributions: 1) 1,000,000x acceleration of the symbolic-regression exploration process, and 2) fundamentally new exploration algorithms that are only possible with such significant acceleration. To accelerate the symbolic-regression exploration process, the investigators introduce software-defined hardware that re-configures every cycle to provide a solution-specific pipeline implemented as a virtual hardware overlay on field-programmable gate arrays. Although this acceleration by itself improves upon the state-of-the-art in symbolic regression considerably, the more important contribution is the enabling of new exploration algorithms that are not feasible without massive increases in performance. The investigators use this performance improvement to introduce a new hybrid exploration algorithm that performs multiple concurrent searches using different configurations of genetic programming and deterministic heuristics, combined with two new prediction mechanisms to avoid local optima: sub-tree look-ahead prediction and operator correlation.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.