Machine Learning Beyond Prediction - Extracting Insights and Guiding Actions

超越预测的机器学习 - 提取见解和指导行动

• 批准号: RGPIN-2020-04333
• 负责人: BenDavid, Shai
• 金额: $ 2.99万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=739897
• 关键词: Machine; Learning; Beyond; Prediction; Extracting

Owing to breakthroughs in supervised learning using deep neural networks, applications of machine learning (ML) have proliferated, spreading to countless industries and societal decisions. Amid this excitement, some fundamental obstacles are often ignored. While ML systems are typically trained to estimate conditional probabilities, in automated systems their primary purpose is to guide actions.  The discrepancy between predictions and decisions is just one among many mismatches between the supervised learning formalism and real-world goals. For example, often the purpose of training a model is not simply to make a prediction but rather to extract qualitative insights, such as causal inference, data clustering or outlier detection.However, when machine learning tools are applied to extract any of those insights, strong assumptions (such as the data being generated by some parameterized probability distribution) are used, often implicitly. Just the same, machine learning is applied far beyond the strict confines of those assumptions. On the other end, for deriving hardness results, most of the theoretical analysis of required resources (be it computational time or training sample sizes) refer to worst-case scenarios, therefore being overly pessimistic. Under that view, large neural networks seem doomed to fail. This project addresses three areas of discrepancy between ML formalism and such real world goals. 1)Interpretability of ML-based tools: Traditional measures of success, such as statistical accuracy and computational efficiency, do not suffice for human consequential applications, where society expects accountability and interpretability. I will analyze formal notions of interpretability and investigate how such notions effect prediction accuracy and render models amenable to monitoring. I will develop theoretical principles under which today's deep learning tools can be leveraged to confer insights beyond their predictive accuracy. 2) Guided selection of clustering algorithms: In spite of the major practical importance of unsupervised learning, current practical implementations of such tasks are very rudimentary. There exists no methodical guidance for clustering tool selection for a given clustering task. I shall address this crucial lacuna by developing methods to guide task appropriate choices of clustering paradigms. 3) Alternatives to worst-case analysis of ML tasks: Many optimization problems that arise in machine learning are NP hard. For example, the training of even small neural networks. Just the same, such problems are being handled routinely on real data for many applications. Experimental evidence suggests that this success relies on some "tameness" of practically arising data. We propose to address this theory-practice discrepancy by distilling structural properties of inputs that can be assumed to hold for naturally arising input data, while giving rise to efficient algorithms for solving hard problems on such inputs.

由于使用深度神经网络的监督学习取得了突破，机器学习（ML）的应用已经激增，并扩展到无数行业和社会决策。 在这种兴奋中，一些基本的障碍往往被忽视。虽然ML系统通常被训练来估计条件概率，但在自动化系统中，它们的主要目的是指导行动。预测和决策之间的差异只是监督学习形式主义和现实世界目标之间的许多不匹配之一。例如，训练模型的目的往往不是简单地做出预测，而是提取定性的见解，如因果推断、数据聚类或离群值检测。然而，当机器学习工具被应用于提取任何这些见解时，通常会隐含地使用强假设（如数据由某些参数化概率分布生成）。同样，机器学习的应用远远超出了这些假设的严格限制。 另一方面，为了获得硬度结果，大多数对所需资源（无论是计算时间还是训练样本大小）的理论分析都涉及最坏情况，因此过于悲观。在这种观点下，大型神经网络似乎注定要失败。这个项目解决了ML形式主义和这种真实的世界目标之间的三个差异领域。 1)基于ML的工具的可解释性：传统的成功衡量标准，如统计准确性和计算效率，不足以满足人类的重要应用，社会期望问责制和可解释性。我将分析可解释性的正式概念，并研究这些概念如何影响预测的准确性，并使模型易于监控。我将开发理论原则，根据这些原则，可以利用当今的深度学习工具来提供超越其预测准确性的见解。2)引导选择聚类算法：尽管无监督学习具有重要的实际意义，但目前此类任务的实际实现非常初级。对于给定的聚类任务，聚类工具的选择没有系统的指导。我将通过开发方法来指导任务适当的集群模式选择来解决这个关键的空白。3)机器学习任务最坏情况分析的替代方案：机器学习中出现的许多优化问题都是NP难的。例如，即使是小型神经网络的训练。同样，对于许多应用程序来说，这样的问题在真实的数据上被常规地处理。实验证据表明，这种成功依赖于一些“驯服”的实际产生的数据。我们建议通过提取输入的结构特性来解决这种理论与实践的差异，这些结构特性可以假设为自然产生的输入数据，同时产生有效的算法来解决这些输入上的难题。