TWC: Small: CrowdVerify: Using the Crowd to Summarize Web Site Privacy Policies and Terms of Use Policies

每天的网络用户在处理他们上网时面临的越来越多的隐私问题时几乎没有指导。许多网站--有些是合法的，有些不那么合法--都有许多人认为意想不到或不受欢迎的行为。这些网站包括流行和知名的网站，以及旨在用“免费”试用来欺骗客户的网站。 这类网站通常在隐私政策和使用条款页面中详细说明他们的行为，但这些政策很少阅读，难以理解，有时故意用法律的术语，小文本和苍白的字体混淆。这项研究的目标是开发新的技术来确定和总结政策中最令人惊讶和最重要的部分。这项研究的结果将在网站上公开，并通过Web浏览器扩展。这项研究的主要研究活动将是设计，实施和评估CrowdVerify，一个将众包与机器学习技术相结合的系统，以标记网站最重要和最意想不到的行为。其核心思想是将给定的策略分割成更小的文本片段，让群组工作人员比较不同的片段，然后将结果聚合在一起。还将评估一些竞争对手的评分系统，以评估细分市场的重要性，包括ELO、Glicko和TrueSkill。利用这些结果，将建立计算模型，可以预测人们在网络政策中发现的最令人惊讶和最重要的东西。

Everyday web users have little guidance in handling the growing number of privacy issues they face when they go online. Many web sites - some legitimate, some less so - have behaviors many would consider unexpected or undesirable. These include popular and well-known web sites, as well as web sites that aim to dupe customers with "free" trials. These kinds of sites often detail their behaviors in privacy policies and terms of use pages, but these policies are rarely read, hard to understand, and sometimes intentionally obfuscated with legal jargon, small text, and pale fonts. The goal of this research is to develop new techniques to pinpoint and summarize the most surprising and most important parts of policies. The results of this research will be made publicly available on a web site and through web browser extensions.The major research activity for this research will be to design, implement, and evaluate CrowdVerify, a system that combines crowdsourcing with machine learning techniques to flag the most important and unexpected behaviors of web sites. The core idea is to slice up a given policy into smaller text segments, have crowd workers compare different segments, and then aggregate the results together. A number of competitor scoring systems will also be evaluated for rating the importance of segments, including ELO, Glicko, and TrueSkill. Using these results, computational models will be built that can predict what people find most surprising as well as most important in web policies.