BIGDATA: F: Privacy in Unsupervised Learning

现代数据集大部分是未标记的。对有用表示的无监督学习以更好地理解数据结构是数据科学和机器学习中的一个关键挑战;它在计算和社会科学中找到了应用，包括信息检索，Web挖掘和推荐系统。随着我们进一步进入大数据时代，并且要处理的数据量的增长速度超过了我们计算资源的增长速度，在这样的大数据集上执行无监督学习和数据分析的更好、更快的方法变得越来越必要。此外，随着物联网的出现，私人数据通过诸如智能手机、相机、麦克风、射频识别（RFID）读取器和社交网络等设备被相当普遍和无缝地收集，引起了对个人隐私的严重关注。因此，在这个项目中，我们对大数据应用中的隐私感知无监督学习进行了正式的研究。从无监督学习的随机优化角度来看，我们捕获了比以前在隐私文献中研究的更一般的学习问题。其中一类学习问题是非凸问题，例如矩阵学习、张量分解、深度学习等等。虽然这些问题中的大多数是NP难的，但在实践中我们发现我们可以有效地找到这些问题的解决方案。我们猜想，最近已被证明有效地找到局部极小值的一大类非凸问题的噪声随机梯度下降更新也保证隐私隐式。最后，我们考虑扩展的隐私模型，从一个单一的策展人到那些分布式学习，持续发布模型，流模型，和一个新的滑动窗口model.This奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响力审查标准进行评估的支持。

Modern data sets are largely unlabeled. Unsupervised learning of useful representations to better understand the structure in data is a critical challenge in data science and machine learning; it finds application in computational and social science, including information retrieval, web mining, and recommendation systems. As we progress further into the age of Big data, and the amount of data to be processed grows faster than the growth in our computational resources, better and faster ways for performing unsupervised learning and data analysis on such big data sets become ever more necessary. Furthermore, with the advent of the internet of things, private data is collected rather ubiquitously and seamlessly through devices such as smartphones, cameras, microphones, radio-frequency identification (RFID) readers, and social networks, raising serious concerns about an individual's privacy. Therefore, in this project, we initiate a formal investigation into privacy-aware unsupervised learning for Big data applications.Taking a stochastic optimization view of unsupervised learning, we capture more general learning problems than previously studied in the privacy literature. One such class of learning problems is non-convex problems, such as matrix learning, tensor factorization, deep learning, and many more. While most of these problems are NP-hard, in practice we find that we can efficiently find solutions to these problems. We conjecture that noisy stochastic gradient descent updates that have recently been shown to efficiently find local minima for a large class of non-convex problems also guarantees privacy implicitly. Finally, we consider extensions of the privacy model from that of a single curator to those to distributed learning, continual release model, streaming model, and a novel sliding window model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.