Reliability Models and Analysis for Triple-model with Triple-input Machine Learning Systems
Reliability Models and Analysis for Triple-model with Triple-input Machine Learning Systems
复制标题
DOI：
10.1109/dsc54232.2022.9888825
复制
发表时间：
2022-06
期刊：
影响因子：
--
通讯作者：
Qiang Wen;F. Machida
中科院分区：
文献类型：
--
作者：
Qiang Wen;F. Machida
机器学习（ML）模型已广泛应用于现实世界的系统。然而，ML模型的输出通常是不确定的，并且对真实的输入数据敏感，这对于设计高可靠性的基于ML的软件系统是一个很大的挑战。我们的研究旨在通过受N版本编程启发的软件架构方法来提高ML系统的可靠性。我们研究中考虑的N版本ML架构将多个输入数据集与多个版本的ML模型相结合，以通过共识确定最终的系统输出。在本文中，我们专注于三个版本的ML架构，并提出了可靠性模型，通过使用ML模型和输入数据集的多样性度量来分析系统的可靠性。所提出的模型允许我们比较的可靠性与三输入（TMTI）架构的三个版本和两个版本的架构与其他变体的三模型。通过数值分析，我们发现：（1）TMTI体系结构的可靠性高于其他三版本体系结构，但（2）TMTI体系结构的可靠性普遍低于双模型双输入系统（DMDI）。此外，我们还发现，较大的方差的模型差异性的负面影响TMTI的可靠性，而较大的方差的输入多样性有相反的影响。
Machine learning (ML) models have been widely applied to real-world systems. However, outputs of ML models are generally uncertain and sensitive to real input data, which is a big challenge in designing highly reliable ML-based software systems. Our study aims to improve the ML system reliability through a software architecture approach inspired by N-version programming. N-version ML architectures considered in our study combine multiple input data sets with multiple versions of ML models to determine the final system output by consensus. In this paper, we focus on three-version ML architectures and propose the reliability models for analyzing the system reliability by using diversity metrics for ML models and input data sets. The proposed model allows us to compare the reliability of a triple-model with triple-input (TMTI) architecture with other variants of three-version and two-version architectures. Through the numerical analysis of the proposed models, we find that i) the reliability of TMTI architecture is higher than other three-version architectures, but interestingly ii) it is generally lower than the reliability of double model with double input system (DMDI). Furthermore, we also find that a larger variance of model diversities negatively impacts the TMTI reliability, while a larger variance of input diversity has opposed impacts.