Composable Workflow for Accelerating Neural Architecture Search Using In Situ Analytics for Protein Classification
Composable Workflow for Accelerating Neural Architecture Search Using In Situ Analytics for Protein Classification
复制标题
DOI：
10.1145/3605573.3605636
复制
发表时间：
2023-08
期刊：
影响因子：
--
通讯作者：
G. Channing;Ria Patel;Paula Olaya;A. Rorabaugh;Osamu Miyashita;Silvina Caíno-Lores;Catherine Schuman;F. Tama;Michela Taufer
中科院分区：
文献类型：
--
作者：
G. Channing;Ria Patel;Paula Olaya;A. Rorabaugh;Osamu Miyashita;Silvina Caíno-Lores;Catherine Schuman;F. Tama;Michela Taufer
神经架构搜索（NAS），用于自动设计科学数据集的神经网络（NN）架构，需要大量的计算资源和时间——通常需要数天或数周的GPU小时数和训练时间。我们设计了神经网络分析（A4NN）工作流，这是一个可组合的工作流，大大减少了设计准确高效的神经网络架构所需的时间和资源。我们引入了参数适应度预测策略，并将训练分布在多个加速器上，以减少聚合神经网络的训练时间。A4NN严格记录神经结构历史、模型状态和元数据，以重现对近最优神经网络的搜索。我们展示了A4NN在x射线自由电子激光（XFEL）实验模拟生成的数据集上减少训练时间和资源消耗的能力。当部署A4NN时，我们将训练时间减少了37%，所需的epoch减少了38%。
Neural architecture search (NAS), which automates the design of neural network (NN) architectures for scientific datasets, requires significant computational resources and time — often on the order of days or weeks of GPU hours and training time. We design the Analytics for Neural Network (A4NN) workflow, a composable workflow that significantly reduces the time and resources required to design accurate and efficient NN architectures. We introduce a parametric fitness prediction strategy and distribute training across multiple accelerators to decrease the aggregated NN training time. A4NN rigorously record neural architecture histories, model states, and metadata to reproduce the search for near-optimal NNs. We demonstrate A4NN’s ability to reduce training time and resource consumption on a dataset generated by an X-ray Free Electron Laser (XFEL) experiment simulation. When deploying A4NN, we decrease training time by up to 37% and epochs required by up to 38%.