BIGDATA: F: Privacy in Unsupervised Learning

现代数据集在很大程度上是没有标签的。对有用的表示进行无监督学习以更好地理解数据中的结构是数据科学和机器学习中的一个关键挑战；它在计算和社会科学中得到了应用，包括信息检索、Web挖掘和推荐系统。随着我们进一步进入大数据时代，要处理的数据量的增长速度快于我们计算资源的增长速度，对此类大数据集执行无监督学习和数据分析的更好、更快的方法变得更加必要。此外，随着物联网的到来，人们通过智能手机、摄像头、麦克风、射频识别(RFID)读取器和社交网络等设备无处不在地无缝收集私人数据，这引发了人们对个人隐私的严重担忧。因此，在本项目中，我们启动了面向大数据应用的隐私感知非监督学习的正式调查，采用非监督学习的随机优化观点，捕获了比以往隐私文献中所研究的更一般的学习问题。一类这样的学习问题是非凸问题，如矩阵学习、张量分解、深度学习等。虽然这些问题大多是NP难的，但在实践中我们发现我们可以有效地找到这些问题的解决方案。我们推测，最近被证明能有效地找到一大类非凸问题的局部极小的噪声随机梯度下降更新也隐含地保证了隐私。最后，我们考虑将隐私模式从单一策展人的模式扩展到分布式学习模式、持续发布模式、流模式和新颖的滑动窗口模式。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Modern data sets are largely unlabeled. Unsupervised learning of useful representations to better understand the structure in data is a critical challenge in data science and machine learning; it finds application in computational and social science, including information retrieval, web mining, and recommendation systems. As we progress further into the age of Big data, and the amount of data to be processed grows faster than the growth in our computational resources, better and faster ways for performing unsupervised learning and data analysis on such big data sets become ever more necessary. Furthermore, with the advent of the internet of things, private data is collected rather ubiquitously and seamlessly through devices such as smartphones, cameras, microphones, radio-frequency identification (RFID) readers, and social networks, raising serious concerns about an individual's privacy. Therefore, in this project, we initiate a formal investigation into privacy-aware unsupervised learning for Big data applications.Taking a stochastic optimization view of unsupervised learning, we capture more general learning problems than previously studied in the privacy literature. One such class of learning problems is non-convex problems, such as matrix learning, tensor factorization, deep learning, and many more. While most of these problems are NP-hard, in practice we find that we can efficiently find solutions to these problems. We conjecture that noisy stochastic gradient descent updates that have recently been shown to efficiently find local minima for a large class of non-convex problems also guarantees privacy implicitly. Finally, we consider extensions of the privacy model from that of a single curator to those to distributed learning, continual release model, streaming model, and a novel sliding window model.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.