There Is Hope After All: Quantifying Opinion and Trustworthiness in Neural Networks.
There Is Hope After All: Quantifying Opinion and Trustworthiness in Neural Networks.
复制标题
毕竟有希望：量化神经网络中的意见和可信度。
DOI：
10.3389/frai.2020.00054
复制
发表时间：
2020
影响因子：
4
通讯作者：
Bogdan P
中科院分区：
文献类型：
--
作者：
Cheng M;Nazarian S;Bogdan P
人工智能（AI）在现代世界中扮演着至关重要的角色，尤其是当它被用作自主决策时。如今一个普遍的担忧是“人工智能有多值得信赖”。人类操作员遵循严格的教育课程和绩效评估，可以用来量化我们对他们的信任程度。为了量化人工智能决策者的信任，我们必须超越任务准确性，特别是在面对有限的、不完整的、误导性的、有争议的或嘈杂的数据集时。为了解决这些挑战，我们描述了DeepTrust，这是一个主观逻辑（SL）启发的框架，它构建了人工智能算法的概率逻辑描述，并考虑了数据集和内部算法工作的可信度。DeepTrust识别适当的多层神经网络（NN）拓扑，这些拓扑具有高的预测信任概率，即使在使用不可信数据进行训练时也是如此。在评估神经网络的意见和可信度时，我们发现数据的不确定意见并不总是恶意的，而不相信意见对信任的伤害最大。此外，信任概率并不一定与准确性相关。DeepTrust还提供了神经网络预测的预测信任概率，当神经网络在有问题的数据集下产生过度自信的输出时，这很有用。这些发现为设计和改进神经网络拓扑开辟了新的分析途径，通过优化意见和可信度，以及准确性，在多目标优化公式中，受空间和时间限制。
Artificial Intelligence (AI) plays a fundamental role in the modern world, especially when used as an autonomous decision maker. One common concern nowadays is “how trustworthy the AIs are.” Human operators follow a strict educational curriculum and performance assessment that could be exploited to quantify how much we entrust them. To quantify the trust of AI decision makers, we must go beyond task accuracy especially when facing limited, incomplete, misleading, controversial or noisy datasets. Toward addressing these challenges, we describe DeepTrust, a Subjective Logic (SL) inspired framework that constructs a probabilistic logic description of an AI algorithm and takes into account the trustworthiness of both dataset and inner algorithmic workings. DeepTrust identifies proper multi-layered neural network (NN) topologies that have high projected trust probabilities, even when trained with untrusted data. We show that uncertain opinion of data is not always malicious while evaluating NN's opinion and trustworthiness, whereas the disbelief opinion hurts trust the most. Also trust probability does not necessarily correlate with accuracy. DeepTrust also provides a projected trust probability of NN's prediction, which is useful when the NN generates an over-confident output under problematic datasets. These findings open new analytical avenues for designing and improving the NN topology by optimizing opinion and trustworthiness, along with accuracy, in a multi-objective optimization formulation, subject to space and time constraints.