BIGDATA: Small: DA: Statistical Machine Learning Methods for Scalable Data Analysis

大数据在现代工业和科学应用中无处不在，数据的规模和维度变得如此之大，以至于需要新的统计工具来进行有效的数据分析。该合作项目涉及罗格斯大学和微软研究院的研究人员，重点关注大数据分析先进计算方法的理论和算法开发。虽然要调查的问题是由各种Internet应用程序引起的，但期望得到的解决方案广泛适用于其他领域。该项目考虑了大数据分析中三个相互关联的主题：(a)对大数据集进行有效采样，以过滤不可靠的数据源并改进统计分析；(b)通过散列和稀疏随机投影技术可以最好地保存信息的降维技术；(c)可以直接处理大数据的机器学习大规模优化技术。这项工作的预期结果包括新的理论结果、新的数据分析算法及其开源软件实现。该研究的更广泛影响包括广泛传播的可扩展数据分析算法的开源实现，研究生和本科生的基于研究的培训和教育，以及导致机器学习基础研究与工业应用之间相互作用的学术-工业合作。

Big Data has become ubiquitous in modern industrial and scientific applications where the size and dimensionality of data are becoming so large as to require new statistical tools for efficient data analysis. This collaborative project involving researchers at Rutgers University and Microsoft Research focuses on the theoretical and algorithmic development of advanced computational methods for big data analytics. While the problems to be investigated are motivated by various Internet applications, the resulting solutions are expected to be broadly applicable to other domains. The project considers three interrelated main themes in big data analytics: (a) effective sampling of big datasets to filter out unreliable data source and improve statistical analysis; (b) dimensionality reduction techniques that can best preserve information via hashing and sparse random projection techniques; and (c) large scale optimization techniques for machine learning that can directly handle large datasize. Anticipated results of this work include new theoretical results, new data analytics algorithms, and their open source software implementations.Broader impacts of the research include broadly disseminated open source implementations of scalable data analytics algorithms, research-based training and education of graduate and undergraduate students, and academic-industrial collaborations resulting in an interplay between fundamental research in machine learning and industrial applications.