RECIPE: Applying Open Domain Question Answering to Privacy Policies
RECIPE: Applying Open Domain Question Answering to Privacy Policies
复制标题
RECIPE：将开放域问答应用于隐私政策
DOI：
10.18653/v1/w18-2608
复制
发表时间：
2018
期刊：
影响因子：
--
通讯作者：
L. Subramanian
中科院分区：
文献类型：
--
作者：
Yan Shvartzshanider;Ananth Balashankar;Thomas Wies;L. Subramanian
我们描述了使用一个开放域问答模型（Chen等人，2017）来评估一个辅助分析公司隐私政策的域外问答任务的经验。具体而言，相关情境完整性参数提取器（RECIPE）试图回答情境完整性（CI）理论针对隐私声明中所描述的信息流提出的问题。这些问题具有简单的句法结构，答案本质上是事实性的或描述性的。该模型的F1分数达到了72.33，但我们注意到，将该模型的结果与一种基于神经依存句法分析器的方法相结合，与人工标注相比，F1分数显著提高到92.35。这表明，未来的工作如果能更明确地纳入来自像自然语言处理任务中的解析信号，就能在域外任务上有更好的泛化能力。
We describe our experiences in using an open domain question answering model (Chen et al., 2017) to evaluate an out-of-domain QA task of assisting in analyzing privacy policies of companies. Specifically, Relevant CI Parameters Extractor (RECIPE) seeks to answer questions posed by the theory of contextual integrity (CI) regarding the information flows described in the privacy statements. These questions have a simple syntactic structure and the answers are factoids or descriptive in nature. The model achieved an F1 score of 72.33, but we noticed that combining the results of this model with a neural dependency parser based approach yields a significantly higher F1 score of 92.35 compared to manual annotations. This indicates that future work which in-corporates signals from parsing like NLP tasks more explicitly can generalize better on out-of-domain tasks.