Pre-Activation based Representation Learning to Enhance Predictive Analytics on Small Materials Data
Pre-Activation based Representation Learning to Enhance Predictive Analytics on Small Materials Data
复制标题
DOI：
10.1109/ijcnn54540.2023.10191086
复制
发表时间：
2023-06
期刊：
影响因子：
--
通讯作者：
Vishu Gupta;W. Liao;Alok Ratan Choudhary;Ankit Agrawal
中科院分区：
文献类型：
--
作者：
Vishu Gupta;W. Liao;Alok Ratan Choudhary;Ankit Agrawal
基于人工智能的预测建模因其从材料数据中提取和利用数据驱动信息的良好能力而在材料科学领域越来越受欢迎，用于训练性能预测模型。然而，目前的方法通常使用有限的手工设计的固定长度的表示，仅从可用的基于合成的信息获得，使得模型输入在处理小的和专门的训练数据集时成为绊脚石。在本文中，我们研究并提出了一种既适用又适用于跨领域推广使用的表示学习(RL)方法。我们介绍了一种RL技术，该技术利用从使用深度神经网络预训练的模型中提取的基于预激活的表示来最大化精度。我们通过利用在大数据集上训练的源模型来在小数据集上建立目标模型，使用基于组成的数值向量来表示材料的元素分数(EF)来对无机材料的性质进行模型训练，然后将其性能与传统的机器学习(ML)、深度神经网络和基于RL的图神经网络(GNN)模型进行比较，其中EF为输入，更多信息的物理属性(PA)为输入，以及传统的TL/RL技术。使用大的$(\sim 345k)$数据集进行源模型训练，使用小的计算$(\sim 28K)$和实验$(\sim 2K)$数据集进行目标模型训练和测试，我们表明，与SC模型和传统的TL/RL技术相比，所提出的RL方法在所有数据大小和属性下都有助于显著提高模型的精度，只使用EF作为输入。我们还通过计算p值进行了统计显著性分析，发现所提出的RL模型比SC、基于RL的GNN和传统的TL/RL模型在精度上确实有了显著的提高。
Artificial intelligence based predictive modeling has become increasingly sought-after in the field of materials science for training property prediction models due to their promising ability to extract and utilize data-driven information from materials data. However, current methods typically use limited hand-engineered fixed-length representations obtained from available composition-based information only, making model inputs a stumbling block when handling small and specialized training datasets. In this paper, we study and propose a method to perform representation learning (RL) that is both applicable and adaptive for generalized use across various domains. We introduce a RL technique that utilizes pre-activation based representations extracted from a model pre-trained using a deep neural network to maximize the accuracy. We perform model training for inorganic material properties using composition-based numerical vectors representing the elemental fractions (EF) of the materials by leveraging source models trained on large datasets to build target models on small datasets and then compare its performance against traditional machine learning (ML), deep neural network and RL-based graph neural network (GNN) models trained from scratch (SC) with EF as input, more informative physical attributes (PA) as input, as well as conventional TL/RL techniques. Using large $(\sim 345K)$ datasets for source model training and small computational $(\sim 28K)$ and experimental $(\sim 2K)$ datasets for target model training and testing, we show that the proposed RL methods help significantly improve the accuracy of the model as compared to the SC models and conventional TL/RL techniques for all data sizes and properties by using only EF as input. We also perform a statistical significance analysis by calculating the p-value to find that the observed improvement in the accuracy of proposed RL model over SC, RL-based GNN, and conventional TL/RL models is indeed significant.