SkillDetective: Automated Policy-Violation Detection of Voice Assistant Applications in the Wild
SkillDetective: Automated Policy-Violation Detection of Voice Assistant Applications in the Wild
复制标题
DOI：
--
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
Jeffrey Young;Song Liao;Long Cheng;Hongxin Hu;Huixing Deng
中科院分区：
文献类型：
--
作者：
Jeffrey Young;Song Liao;Long Cheng;Hongxin Hu;Huixing Deng
如今的语音个人助理 (VPA) 服务已得到大幅扩展，允许第三方开发人员构建语音应用程序并将其发布到市场（例如 Amazon Alexa 和 Google Assistant 平台）。为了阻止不道德的开发者，VPA 平台提供商指定了一组第三方开发者必须遵守的政策要求，例如，面向儿童的语音应用程序不允许收集个人数据。在这项工作中，我们的目标是通过对语音应用程序的全面动态分析来识别当前 VPA 平台中违反政策的语音应用程序。为此，我们设计和开发了 SKILL DETECTIVE，这是一种交互式测试工具，能够探索语音应用程序的行为并以自动方式识别策略违规行为。与之前的工作不同，SKILL DE ETECTIVE 从文本、图像和音频文件等多种来源，在更广泛的背景下评估语音应用程序是否符合 52 种不同的政策要求。通过 S KILL D ETECTIVE，我们测试了 54,055 个 Amazon Alexa 技能和 5,583 个 Google Assistant 操作，并从语音应用程序交互中收集了 518,385 个文本输出、大约 2,070 个独特的音频文件和 31,100 个独特的图像。我们发现 6,079 项技能和 175 项行为违反了至少一项政策要求。我们已向两家 VPA 供应商报告了我们的发现，并收到了他们的确认。
Today’s voice personal assistant (VPA) services have been largely expanded by allowing third-party developers to build voice-apps and publish them to marketplaces ( e.g. , the Amazon Alexa and Google Assistant platforms). In an effort to thwart unscrupulous developers, VPA platform providers have specifed a set of policy requirements to be adhered to by third-party developers, e.g. , personal data collection is not allowed for kid-directed voice-apps. In this work, we aim to identify policy-violating voice-apps in current VPA platforms through a comprehensive dynamic analysis of voice-apps. To this end, we design and develop S KILL D ETECTIVE , an interactive testing tool capable of exploring voice-apps’ behaviors and identifying policy violations in an automated manner. Distinc-tive from prior works, S KILL D ETECTIVE evaluates voice-apps’ conformity to 52 different policy requirements in a broader context from multiple sources including textual, image and audio fles. With S KILL D ETECTIVE , we tested 54,055 Amazon Alexa skills and 5,583 Google Assistant actions, and collected 518,385 textual outputs, approximately 2,070 unique audio fles and 31,100 unique images from voice-app interactions. We identifed 6,079 skills and 175 actions violating at least one policy requirement. We have reported our fndings to both VPA vendors, and received their acknowledgments.