Machine Learning Beyond Prediction - Extracting Insights and Guiding Actions

由于深度神经网络在监督学习方面的突破，机器学习（ML）的应用激增，蔓延到无数的行业和社会决策中。在这种兴奋中，一些基本障碍往往被忽视。虽然机器学习系统通常被训练来估计条件概率，但在自动化系统中，它们的主要目的是指导行动。预测和决策之间的差异只是监督学习形式主义与现实世界目标之间的许多不匹配之一。例如，通常训练模型的目的不是简单地做出预测，而是提取定性的见解，例如因果推理、数据聚类或离群值检测。然而，当应用机器学习工具提取这些见解时，通常会隐式地使用强假设（例如由某些参数化概率分布生成的数据）。同样，机器学习的应用远远超出了这些假设的严格限制。另一方面，对于硬度结果的推导，大多数对所需资源（无论是计算时间还是训练样本大小）的理论分析都是基于最坏的情况，因此过于悲观。在这种观点下，大型神经网络似乎注定要失败。这个项目解决了机器学习形式主义和现实世界目标之间的三个差异。1)基于机器学习的工具的可解释性：传统的成功衡量标准，如统计准确性和计算效率，不足以满足人类的后续应用，因为社会期望问责制和可解释性。我将分析可解释性的正式概念，并研究这些概念如何影响预测准确性，并使模型易于监控。我将提出一些理论原则，在这些原则下，今天的深度学习工具可以被利用来提供超越其预测准确性的见解。2)聚类算法的引导选择：尽管无监督学习具有重要的实际意义，但目前此类任务的实际实现非常初级。对于给定的聚类任务，没有系统的聚类工具选择指南。我将通过开发指导任务恰当选择聚类范式的方法来解决这一关键缺陷。3) ML任务的最坏情况分析的替代方案：机器学习中出现的许多优化问题都是NP困难的。例如，即使是很小的神经网络的训练。同样，对于许多应用程序来说，这类问题通常都是在实际数据上处理的。实验证据表明，这种成功依赖于实际产生的数据的某种“驯服性”。我们建议通过提取输入的结构属性来解决这种理论与实践的差异，这些属性可以被认为是自然产生的输入数据，同时产生有效的算法来解决这些输入上的难题。

Owing to breakthroughs in supervised learning using deep neural networks, applications of machine learning (ML) have proliferated, spreading to countless industries and societal decisions. Amid this excitement, some fundamental obstacles are often ignored. While ML systems are typically trained to estimate conditional probabilities, in automated systems their primary purpose is to guide actions.  The discrepancy between predictions and decisions is just one among many mismatches between the supervised learning formalism and real-world goals. For example, often the purpose of training a model is not simply to make a prediction but rather to extract qualitative insights, such as causal inference, data clustering or outlier detection.However, when machine learning tools are applied to extract any of those insights, strong assumptions (such as the data being generated by some parameterized probability distribution) are used, often implicitly. Just the same, machine learning is applied far beyond the strict confines of those assumptions. On the other end, for deriving hardness results, most of the theoretical analysis of required resources (be it computational time or training sample sizes) refer to worst-case scenarios, therefore being overly pessimistic. Under that view, large neural networks seem doomed to fail. This project addresses three areas of discrepancy between ML formalism and such real world goals. 1)Interpretability of ML-based tools: Traditional measures of success, such as statistical accuracy and computational efficiency, do not suffice for human consequential applications, where society expects accountability and interpretability. I will analyze formal notions of interpretability and investigate how such notions effect prediction accuracy and render models amenable to monitoring. I will develop theoretical principles under which today's deep learning tools can be leveraged to confer insights beyond their predictive accuracy. 2) Guided selection of clustering algorithms: In spite of the major practical importance of unsupervised learning, current practical implementations of such tasks are very rudimentary. There exists no methodical guidance for clustering tool selection for a given clustering task. I shall address this crucial lacuna by developing methods to guide task appropriate choices of clustering paradigms. 3) Alternatives to worst-case analysis of ML tasks: Many optimization problems that arise in machine learning are NP hard. For example, the training of even small neural networks. Just the same, such problems are being handled routinely on real data for many applications. Experimental evidence suggests that this success relies on some "tameness" of practically arising data. We propose to address this theory-practice discrepancy by distilling structural properties of inputs that can be assumed to hold for naturally arising input data, while giving rise to efficient algorithms for solving hard problems on such inputs.