Understanding Where Your Classifier Does (Not) Work -- The SCaPE Model Class for EMM
Understanding Where Your Classifier Does (Not) Work -- The SCaPE Model Class for EMM
复制标题
了解分类器在哪里工作（不工作）——EMM 的 SCaPE 模型类
DOI：
--
复制
发表时间：
2014
期刊：
影响因子：
--
通讯作者：
J. Thaele
中科院分区：
文献类型：
--
作者：
W. Duivesteijn;J. Thaele
FACT，第一个G-APD切伦科夫望远镜，探测高能宇宙粒子引起的空气簇射。人们希望将簇射分类为由伽马射线或背景粒子引起的。一般来说，在现实生活中的训练任务中获得任何反馈都是很重要的，但是我们可以通过研究我们的分类器在蒙特卡洛模拟数据上的性能来尝试了解它是如何工作的。为此，在本文中，我们开发了SCaPE（软分类器性能评估）模型类异常模型挖掘，这是一个本地模式挖掘框架，致力于突出多个目标之间的不寻常的相互作用。在我们的蒙特卡罗模拟数据中，我们将计算出的分类器概率和包含基本事实的二进制列作为目标：哪种粒子引起了相应的簇射。使用新开发的基于排名损失的质量度量，SCaPE模型类突出显示了搜索空间的子空间，其中分类器表现得特别好或特别差。这些子空间根据数据属性的条件到达，因此它们以领域专家理解的语言出现，这应该有助于他理解他/她的分类器在哪里不起作用。发现子群突出子空间的分类困难是由天体物理学解释证实，以及子空间，值得进一步调查。
FACT, the First G-APD Cherenkov Telescope, detects air showers induced by high-energetic cosmic particles. It is desirable to classify a shower as being induced by a gamma ray or a background particle. Generally, it is nontrivial to get any feedback on the real-life training task, but we can attempt to understand how our classifier works by investigating its performance on Monte Carlo simulated data. To this end, in this paper we develop the SCaPE (Soft Classifier Performance Evaluation) model class for Exceptional Model Mining, which is a Local Pattern Mining framework devoted to highlighting unusual interplay between multiple targets. In our Monte Carlo simulated data, we take as targets the computed classifier probabilities and the binary column containing the ground truth: which kind of particle induced the corresponding shower. Using a newly developed quality measure based on ranking loss, the SCaPE model class highlights subspaces of the search space where the classifier performs particularly well or poorly. These subspaces arrive in terms of conditions on attributes of the data, hence they come in a language a domain expert understands, which should aid him in understanding where his/her classifier does (not) work. Found subgroups highlight subspaces whose difficulty for classification is corroborated by astrophysical interpretation, as well as subspaces that warrant further investigation.