BRNet: Branched Residual Network for Fast and Accurate Predictive Modeling of Materials Properties
BRNet: Branched Residual Network for Fast and Accurate Predictive Modeling of Materials Properties
复制标题
DOI：
10.1137/1.9781611977172.39
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Vishu Gupta;W. Liao;Alok Ratan Choudhary;Ankit Agrawal
中科院分区：
文献类型：
--
作者：
Vishu Gupta;W. Liao;Alok Ratan Choudhary;Ankit Agrawal
机器学习（ML）和深度学习（DL）在材料科学领域越来越受欢迎，因为它们能够有效地提取和理解材料成分，结构和性能之间的数据驱动关系。通常，材料性能预测是具有基于矢量的输入材料表示的回归问题。虽然全连接层已被广泛用于深度神经网络来预测材料属性，但简单地添加越来越多的层来创建深度模型往往会由于消失梯度问题而降低其性能，从而限制使用。在本文中，我们研究并提出了构建深度回归神经网络的架构原则，该网络包括具有数字向量的全连接层，这些数字向量绕过了手动特征工程。我们引入了一种新型的具有分支残差学习的深度回归神经网络BRNet，它由层的分支组成，以最大化从输入或前一层学习的特征的变化，并在每层之后放置跳过连接，以最大限度地减少由于梯度消失而导致的信息损失。我们使用代表相应材料成分的元素分数的数值向量对无机材料属性进行BRNet模型训练，并将其性能与其他传统ML和DL技术（包括ElemNet和IRNet）进行比较。使用多个数据集（如OQMD，MP，JARVIS）进行训练和测试，我们表明，对于所有数据大小，BRNet模型都比最先进的ML方法和DL模型更准确，仅使用原始元素分数作为输入。我们还表明，BRNet的分支残差学习需要的参数更少，并且在训练阶段比其他神经网络具有更好的收敛性，从而实现更快的模型训练。
Machine Learning (ML) and Deep Learning (DL) have become increasingly popular in the field of materials science for building property prediction models owing to their ability to efficiently extract and understand data-driven relation-ships between materials composition, structure, and properties. In general, materials property prediction are regression problems with a vector-based input material representation. While fully connected layers have been widely used in deep neural networks to predict materials properties, simply adding more and more layers to create a deep model often degrades their performance due to the vanishing gradient problem, thereby limiting usage. In this paper, we study and propose architectural principles for building deep regression neural networks comprising fully connected layers with numerical vectors that bypass manual feature engineering. We introduce a novel deep regression neural network with branched residual learning, BRNet, consisting of branching of layers to maximize variation of features learned from the input or previous layer and places skip connections after each layer to minimize the information loss due to vanishing gradient. We perform BRNet model training for inorganic material properties using numerical vectors representing the elemental fractions of the compositions of the respective materials and compare its performance against other traditional ML and DL techniques, including ElemNet and IRNet. Using multiple datasets (such as OQMD, MP, JARVIS) for training and testing, we show that BRNet models are signif-icantly more accurate than the state-of-the-art ML methods and DL models for all data sizes by using only raw elemental fractions as input. We also show that BRNet’s branched residual learning requires fewer parameters and leads to better convergence during the training phase than other neural networks, thus resulting in faster model training.