BIGDATA: F: Privacy in Unsupervised Learning

现代数据集基本上没有标记。无监督学习有用的表征以更好地理解数据中的结构是数据科学和机器学习中的一个关键挑战；它在计算科学和社会科学中得到了应用，包括信息检索、网络挖掘和推荐系统。随着我们进一步进入大数据时代，需要处理的数据量的增长速度超过了我们计算资源的增长速度，对这种大数据集进行更好、更快的无监督学习和数据分析的方法变得越来越必要。此外，随着物联网的出现，私人数据通过智能手机、相机、麦克风、射频识别（RFID）阅读器和社交网络等设备无处不在、无缝地收集，这引发了对个人隐私的严重担忧。因此，在这个项目中，我们对大数据应用中的隐私感知无监督学习进行了正式的研究。采用无监督学习的随机优化观点，我们捕获了比以前在隐私文献中研究过的更多的一般学习问题。其中一类学习问题是非凸问题，如矩阵学习、张量分解、深度学习等等。虽然这些问题中的大多数都是np困难的，但在实践中我们发现我们可以有效地找到这些问题的解决方案。我们推测，噪声随机梯度下降更新最近被证明可以有效地找到大量非凸问题的局部最小值，也可以隐式地保证隐私。最后，我们考虑了隐私模型的扩展，从单个管理员的隐私模型扩展到分布式学习、持续发布模型、流模型和一种新的滑动窗口模型。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Modern data sets are largely unlabeled. Unsupervised learning of useful representations to better understand the structure in data is a critical challenge in data science and machine learning; it finds application in computational and social science, including information retrieval, web mining, and recommendation systems. As we progress further into the age of Big data, and the amount of data to be processed grows faster than the growth in our computational resources, better and faster ways for performing unsupervised learning and data analysis on such big data sets become ever more necessary. Furthermore, with the advent of the internet of things, private data is collected rather ubiquitously and seamlessly through devices such as smartphones, cameras, microphones, radio-frequency identification (RFID) readers, and social networks, raising serious concerns about an individual's privacy. Therefore, in this project, we initiate a formal investigation into privacy-aware unsupervised learning for Big data applications.Taking a stochastic optimization view of unsupervised learning, we capture more general learning problems than previously studied in the privacy literature. One such class of learning problems is non-convex problems, such as matrix learning, tensor factorization, deep learning, and many more. While most of these problems are NP-hard, in practice we find that we can efficiently find solutions to these problems. We conjecture that noisy stochastic gradient descent updates that have recently been shown to efficiently find local minima for a large class of non-convex problems also guarantees privacy implicitly. Finally, we consider extensions of the privacy model from that of a single curator to those to distributed learning, continual release model, streaming model, and a novel sliding window model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.