IrEne: Interpretable Energy Prediction for Transformers
IrEne: Interpretable Energy Prediction for Transformers
复制标题
DOI：
10.18653/v1/2021.acl-long.167
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Qingqing Cao;Yash Kumar Lal;H. Trivedi;A. Balasubramanian;Niranjan Balasubramanian
中科院分区：
文献类型：
--
作者：
Qingqing Cao;Yash Kumar Lal;H. Trivedi;A. Balasubramanian;Niranjan Balasubramanian
NLP模型的现有基于软件的能源测量不准确，因为它们不考虑能耗和模型执行之间的复杂相互作用。我们提出了艾琳（Irene），这是一种可解释且可扩展的能源预测系统，可以准确预测广泛的基于变压器的NLP模型的推理能量消耗。艾琳（Irene）构建了一个模型树图，该图将NLP模型分解为模块，该模块进一步分解为低级机器学习（ML）原始图。艾琳（Irene）预测，ML原始基因的推理能源消耗是可推广的功能和细粒度运行时资源使用的函数。然后，艾琳递归地汇总了这些低级预测，以预测每个模块的能量，最后是整个模型的能量。多个变压器模型的实验表明，与地面真相相比，艾琳预测变压器模型的推理能量消耗率低于7％。相比之下，现有能源模型的误差超过50％。我们还展示了如何使用Irene进行能源瓶颈分析，并轻松评估不同建筑选择的能量影响。我们在https://github.com/stonybrooknlp/irene上发布代码和数据。
Existing software-based energy measurements of NLP models are not accurate because they do not consider the complex interactions between energy consumption and model execution. We present IrEne, an interpretable and extensible energy prediction system that accurately predicts the inference energy consumption of a wide range of Transformer-based NLP models. IrEne constructs a model tree graph that breaks down the NLP model into modules that are further broken down into low-level machine learning (ML) primitives. IrEne predicts the inference energy consumption of the ML primitives as a function of generalizable features and fine-grained runtime resource usage. IrEne then aggregates these low-level predictions recursively to predict the energy of each module and finally of the entire model. Experiments across multiple Transformer models show IrEne predicts inference energy consumption of transformer models with an error of under 7% compared to the ground truth. In contrast, existing energy models see an error of over 50%. We also show how IrEne can be used to conduct energy bottleneck analysis and to easily evaluate the energy impact of different architectural choices. We release the code and data at https://github.com/StonyBrookNLP/irene.