The Epistemology of Statistical Learning Theory

机器学习对科学和整个社会的影响越来越大。这就要求我们不断努力，不仅要更好地理解机器学习方法的广泛使用所带来的（社会、伦理和其他方面的）后果，还要更好地理解这些方法本身。如何解释他们明显的成功？它们在什么意义上和多大程度上得出可靠的结论？它们不可避免的缺陷和局限性是什么？这些问题与不确定推理或归纳推理的性质和正当性有关：基本的认识论问题。然而，迄今为止，哲学家们很少涉及这些与我们目前最突出的归纳推理方法有关的问题，这些方法是由现代机器学习算法给出的。与此同时，机器学习理论的研究人员已经开发出强大的数学框架来检查学习算法的功能。特别是统计学习理论（SLT），构成了现代机器学习的理论框架。我的项目目的是利用SLT的框架来研究归纳推理的哲学问题，并将这些研究嵌入到机器学习方法的更广泛的认识论评估中。这一主要目的在三个阶段的一系列目标中得到具体体现。第一阶段涉及归纳推理的基本限制，特别是在SLT的背景下，非免费午餐定理作为这些限制的表达。我将通过论证“不免费午餐”定理的最富有成效的解释，并通过说明哲学中的结果如何被理解为“不免费午餐”定理的实例，来得出一个统一的观点。第二阶段更关注学习方法的成功及其可能的解释：特别是在归纳推理中坚持简单性偏好，即奥卡姆剃刀原则。我将详细说明一个最小但真实的意义，在这个意义上，SLT为奥卡姆剃刀的正当性提供了论证。最后，在当前关于如何解释深度神经网络特定学习范式的实际成功的科学辩论中，前两个阶段结合在一起。在第三阶段，我将把这场辩论作为归纳推理哲学中的一个重要案例来分析。此外，这场辩论与关于机器学习理论的作用及其对传统认识论的影响的更广泛的讨论有关。我将反对这样一种观点，即机器学习的实际成功证明了一种低估理论分析作用的认识论。这也为我的项目的工作假设提供了辩护，即机器学习理论对其认识论评价是无价的。

Machine learning has an ever increasing impact on science and society as a whole. This calls for a persistent effort towards a better understanding, not only of the (societal, ethical, and otherwise) consequences of the ubiquitous use of machine learning methods, but also of these methods themselves. What explains their apparent success? In what sense and to what extent do they lead to reliable conclusions? What are their unavoidable pitfalls and limitations? These questions have to do with the nature and justification of uncertain or inductive inference: fundamental epistemological questions. Nevertheless, philosophers have to date engaged little with these questions in relation to our currently most prominent methods of inductive inference, those given by modern machine learning algorithms. At the same time, researchers in the theory of machine learning have developed powerful mathematical frameworks to examine the functioning of learning algorithms. Statistical learning theory (SLT), in particular, constitutes the theoretical framework underlying much of modern machine learning. The aim of my project is to employ the framework of SLT to investigate philosophical problems of inductive inference, and to embed these investigations in a wider epistemological appraisal of machine learning methods. This main aim finds concrete shape in a set of objectives within three stages. The first stage concerns the fundamental limitations of inductive inference, and specifically, in the context of SLT, the no-free-lunch theorems as expressions of these. I will work out a unified perspective by arguing for a most fruitful interpretation of the no-free-lunch theorems, and by giving an account of how results within philosophy can be understood as instantiations of the no-free-lunch theorems. The second stage rather concerns the success of learning methods, and possible explanations thereof: specifically the adherence to a simplicity preference in inductive inference, the principle of Occam's razor. I will spell out a minimal yet genuine sense in which SLT provides an argument for the justification of Occam’s razor.The first two stages come together, finally, in the current scientific debate on how to explain the practical success of the particular learning paradigm of deep neural nets. In the third stage I will analyze this debate as an important case study in the philosophy of inductive inference. This debate, moreover, ties in with a more general dicussion on the role of machine learning theory and its implications for traditional epistemology. I will argue against the persistent view that the practical success of machine learning warrants an epistemology that downplays the role of theoretical analysis. This also serves as a defense of the working assumption of my project that the theory of machine learning is invaluable for its epistemological appraisal.