Don't forget, there is more than forgetting: new metrics for Continual Learning
Don't forget, there is more than forgetting: new metrics for Continual Learning
复制标题
不要忘记，不仅仅是忘记：持续学习的新指标
DOI：
--
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
D. Maltoni
中科院分区：
文献类型：
--
作者：
Natalia Díaz Rodríguez;Vincenzo Lomonaco;David Filliat;D. Maltoni
持续学习由算法组成，这些算法不断地自适应地思考时间，从一系列数据流/任务中学习，使更复杂的知识和技能能够逐步发展。在评估持续学习算法方面缺乏共识，以及几乎完全专注于忘记，促使我们提出了一套更全面的实现独立度量标准，考虑了我们认为在部署真实的持续学习的人工智能系统时值得考虑的几个因素：精度或性能随时间的推移，向后和向前的知识转移，内存开销以及计算效率。受标准多属性价值理论(MAVT)的启发，我们进一步提出将这些指标融合为一个分数进行排名，并在iCIFAR-100继续学习基准上用五种持续学习策略对我们的建议进行了评估。
Continual learning consists of algorithms that learn from a stream of data/tasks continuously and adaptively thought time, enabling the incremental development of ever more complex knowledge and skills. The lack of consensus in evaluating continual learning algorithms and the almost exclusive focus on forgetting motivate us to propose a more comprehensive set of implementation independent metrics accounting for several factors we believe have practical implications worth considering in the deployment of real AI systems that learn continually: accuracy or performance over time, backward and forward knowledge transfer, memory overhead as well as computational efficiency. Drawing inspiration from the standard Multi-Attribute Value Theory (MAVT) we further propose to fuse these metrics into a single score for ranking purposes and we evaluate our proposal with five continual learning strategies on the iCIFAR-100 continual learning benchmark.