Machine Learning Beyond Prediction - Extracting Insights and Guiding Actions

超越预测的机器学习 - 提取见解和指导行动

• 批准号: RGPIN-2020-04333
• 负责人: BenDavid, Shai
• 金额: $ 2.99万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=750181
• 关键词: Machine; Learning; Beyond; Prediction; Extracting

Owing to breakthroughs in supervised learning using deep neural networks, applications of machine learning (ML) have proliferated, spreading to countless industries and societal decisions. Amid this excitement, some fundamental obstacles are often ignored. While ML systems are typically trained to estimate conditional probabilities, in automated systems their primary purpose is to guide actions.  The discrepancy between predictions and decisions is just one among many mismatches between the supervised learning formalism and real-world goals. For example, often the purpose of training a model is not simply to make a prediction but rather to extract qualitative insights, such as causal inference, data clustering or outlier detection.However, when machine learning tools are applied to extract any of those insights, strong assumptions (such as the data being generated by some parameterized probability distribution) are used, often implicitly. Just the same, machine learning is applied far beyond the strict confines of those assumptions. On the other end, for deriving hardness results, most of the theoretical analysis of required resources (be it computational time or training sample sizes) refer to worst-case scenarios, therefore being overly pessimistic. Under that view, large neural networks seem doomed to fail. This project addresses three areas of discrepancy between ML formalism and such real world goals. 1)Interpretability of ML-based tools: Traditional measures of success, such as statistical accuracy and computational efficiency, do not suffice for human consequential applications, where society expects accountability and interpretability. I will analyze formal notions of interpretability and investigate how such notions effect prediction accuracy and render models amenable to monitoring. I will develop theoretical principles under which today's deep learning tools can be leveraged to confer insights beyond their predictive accuracy. 2) Guided selection of clustering algorithms: In spite of the major practical importance of unsupervised learning, current practical implementations of such tasks are very rudimentary. There exists no methodical guidance for clustering tool selection for a given clustering task. I shall address this crucial lacuna by developing methods to guide task appropriate choices of clustering paradigms. 3) Alternatives to worst-case analysis of ML tasks: Many optimization problems that arise in machine learning are NP hard. For example, the training of even small neural networks. Just the same, such problems are being handled routinely on real data for many applications. Experimental evidence suggests that this success relies on some "tameness" of practically arising data. We propose to address this theory-practice discrepancy by distilling structural properties of inputs that can be assumed to hold for naturally arising input data, while giving rise to efficient algorithms for solving hard problems on such inputs.

由于深度神经网络在监督学习方面的突破，机器学习(ML)的应用已经激增，扩展到无数的行业和社会决策。在这种兴奋中，一些根本性的障碍往往被忽视。虽然ML系统通常被训练来估计条件概率，但在自动化系统中，它们的主要目的是指导行动。预测和决策之间的差异只是监督学习形式主义和现实世界目标之间的许多不匹配之一。例如，训练模型的目的通常不是简单地进行预测，而是提取定性的洞察力，例如因果推理、数据聚类或离群值检测。然而，当应用机器学习工具来提取这些洞察力中的任何一种时，通常隐含地使用强假设(例如由某些参数化的概率分布生成的数据)。同样，机器学习的应用远远超出了这些假设的严格限制。另一方面，为了得出困难结果，对所需资源(无论是计算时间还是训练样本大小)的大多数理论分析都提到了最坏的情况，因此过于悲观。在这种观点下，大型神经网络似乎注定要失败。这个项目解决了ML形式主义和这样的现实世界目标之间的三个方面的差异。1)基于ML的工具的可解释性：传统的成功衡量标准，如统计准确性和计算效率，不足以满足人类相应的应用，因为社会期望问责和可解释性。我将分析可解释性的正式概念，并调查这些概念如何影响预测精度和使模型易于监控。我将制定理论原则，在这些原则下，今天的深度学习工具可以被利用来提供超出其预测准确性的见解。2)有指导地选择聚类算法：尽管无监督学习具有重要的实际意义，但目前这类任务的实际实现非常初级。对于给定的集群任务，对于集群工具的选择没有系统的指导。我将通过开发指导任务适当选择集群范例的方法来解决这一关键缺陷。3)ML任务最坏情况分析的替代方法：机器学习中出现的许多优化问题都是NP困难的。例如，即使是小的神经网络的训练。同样，在许多应用程序中，这些问题都是在实际数据上进行常规处理的。实验证据表明，这一成功有赖于实际数据的某种“驯服”。我们建议通过提取输入的结构属性来解决这种理论-实践差异，这些结构属性可以被假设为适用于自然产生的输入数据，同时产生解决此类输入的困难问题的有效算法。