CIF: Small: An Algebraic, Convex, and Scalable Framework for Kernel Learning with Activation Functions

近年来，公众对机器学习的兴趣显著增加，从医疗诊断到语音识别，从自动驾驶到广告等各个领域都有应用。然而，维持这种兴趣的能力将取决于机器学习算法是否在可靠性、可扩展性和可解释性方面继续进步。随着越来越多的数据可用，自动驾驶汽车会变得更安全吗？Siri能更好地理解你吗？医生能更好地了解疾病的原因和治疗方法吗？虽然近年来神经网络和深度学习得到了广泛的应用，但这些方法背后的算法在20多年里并没有发生实质性的变化。因此，这个项目重新审视了机器学习算法背后的基础数学——将经典结果与流行的基于神经网络的方法相结合。然后，这个数学框架被用来提出新的方法，以提高机器学习的准确性，增加处理大数据集的能力，并允许机器学习算法的结果更容易用可测量的物理量来解释。为了达到准确性、可扩展性和可解释性的目标，该项目提出了一个经典核学习问题的代数重新表述。具体来说，对于任何给定的核代数，该代数中的正核及其相关的特征映射可以由正矩阵表示，从而导致一个凸优化问题，其解决方案产生一个显式的特征映射，可以用可测量的物理量来解释。基于该框架，激活函数用于定义核代数，这些核代数是通用的，但在所有核集合中是密集的，其特征映射模拟了定义神经网络的神经切线核的特征映射，从而提高了算法的准确性。其次，使用鞍点表示和原始对偶方法将核学习问题转换为二次规划-从而产生更具可扩展性的核学习算法。最后，通过求解相关的偏微分方程，得到特征映射的奇异值分解。这种分解用于识别数据中的关键特征，此外，还产生了与样本数量线性扩展的简化算法-这意味着具有数万个样本的数据集的可扩展性。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Public interest in machine learning has increased significantly in recent years, with application in a diversity of fields, from medical diagnosis to speech recognition to autonomous driving to advertising. The ability to sustain this interest, however, will depend on whether machine learning algorithms continue to advance in terms of both reliability, scalability, and interpretability. As more data becomes available, Will self-driving cars become safer? Will Siri understand you better? Will doctors be able to better understand the causes and treatments for diseases? While neural networks and deep learning have seen widespread adoption in recent years, the algorithms which underly these methods have not changed substantially in over 20 years. This project, therefore, revisits the fundamental mathematics which underly machine learning algorithms – integrating classical results with the popular neural-network based approaches. This mathematical framework is then used to propose new methods for improving the accuracy of machine learning, for increasing the ability to process large data sets, and for allowing the results of machine learning algorithms to be more readily interpreted in terms of measurable physical quantities. To achieve the goals of accuracy, scalability and interpretability, the project poses an algebraic reformulation of the classical problem of learning the kernel. Specifically, for any given kernel algebra, the positive kernels in that algebra and their associated feature maps may be represented by positive matrices – leading to a convex optimization problem whose solution yields an explicit feature map which may be interpreted in terms of measurable physical quantities. Based on this framework, activation functions are used to define kernel algebras which are universal, yet which are dense in the set of all kernels and whose feature maps mimic those of the neural tangent kernel which defines neural networks – leading to improved accuracy of the algorithms. Next, a saddle-point representation and primal-dual approach is used to convert the kernel learning problem to quadratic programming – resulting in more scalable kernel learning algorithms. Finally, a singular value decomposition of the resulting feature map is obtained by solving an associated partial differential equation. This decomposition is used to identify key features in the data and, furthermore, yields reduced algorithms which scale linearly with the number of samples – implying scalability to datasets with tens of thousands of samples.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.