Empirical Investigation of Neural Symbolic Reasoning Strategies
Empirical Investigation of Neural Symbolic Reasoning Strategies
复制标题
DOI：
10.48550/arxiv.2302.08148
复制
发表时间：
2023-02
期刊：
影响因子：
--
通讯作者：
Yoichi Aoki;Keito Kudo;Tatsuki Kuribayashi;Ana Brassard;Masashi Yoshikawa;Keisuke Sakaguchi;Kentaro Inui
中科院分区：
文献类型：
--
作者：
Yoichi Aoki;Keito Kudo;Tatsuki Kuribayashi;Ana Brassard;Masashi Yoshikawa;Keisuke Sakaguchi;Kentaro Inui
在生成中间推理步骤时，神经推理精度会提高。然而，这种改进的来源尚不清楚。在这里，我们调查和分解了生成符号推理的中间步骤的好处，具体地，我们分解了推理策略w.r.t.步骤粒度和链接策略。在一个纯符号数值推理数据集(例如，A=1，B=3，C=A+3，C？)上，我们发现推理策略的选择对性能有显著的影响，并且随着外推长度的增加，这一差距变得越来越大。令人惊讶的是，我们还发现，即使是在长度外推的情况下，某些结构也能带来近乎完美的性能。
Neural reasoning accuracy improves when generating intermediate reasoning steps. However, the source of this improvement is yet unclear.Here, we investigate and factorize the benefit of generating intermediate steps for symbolic reasoning.Specifically, we decompose the reasoning strategy w.r.t. step granularity and chaining strategy. With a purely symbolic numerical reasoning dataset (e.g., A=1, B=3, C=A+3, C?), we found that the choice of reasoning strategies significantly affects the performance, with the gap becoming even larger as the extrapolation length becomes longer.Surprisingly, we also found that certain configurations lead to nearly perfect performance, even in the case of length extrapolation.Our results indicate the importance of further exploring effective strategies for neural reasoning models.