Differentiable Synthesis of Program Architectures
Differentiable Synthesis of Program Architectures
复制标题
DOI：
--
复制
发表时间：
2021
期刊：
影响因子：
--
通讯作者：
Guofeng Cui;He Zhu
中科院分区：
文献类型：
--
作者：
Guofeng Cui;He Zhu
可微分程序最近因其可解释性、组合性以及利用可微分训练的效率而引起了人们的广泛兴趣。然而，合成可微程序需要对程序架构的组合、快速爆炸空间进行优化。尽管开发了有效的剪枝启发式方法，但以前的工作本质上是枚举程序架构的离散搜索空间，这是低效的。我们建议将程序架构搜索编码为学习由上下文无关语法引起的所有可能的程序派生的概率分布。这使得搜索算法能够有效地删除不太可能的程序派生，以综合最佳的程序架构。为此，开发了一种有效的基于梯度下降的方法，以在语法规则的离散空间的连续松弛中进行程序架构搜索。四个序列分类任务的实验结果表明，我们的程序合成器擅长发现程序架构，从而产生具有更高 F 1 分数的可微程序，同时比最先进的程序合成方法更高效。
Differentiable programs have recently attracted much interest due to their inter-pretability, compositionality, and their efficiency to leverage differentiable training. However, synthesizing differentiable programs requires optimizing over a combinatorial, rapidly exploded space of program architectures. Despite the development of effective pruning heuristics, previous works essentially enumerate the discrete search space of program architectures, which is inefficient. We propose to encode program architecture search as learning the probability distribution over all possible program derivations induced by a context-free grammar. This allows the search algorithm to efficiently prune away unlikely program derivations to synthesize optimal program architectures. To this end, an efficient gradient-descent based method is developed to conduct program architecture search in a continuous relaxation of the discrete space of grammar rules. Experiment results on four sequence classification tasks demonstrate that our program synthesizer excels in discovering program architectures that lead to differentiable programs with higher F 1 scores, while being more efficient than state-of-the-art program synthesis methods.