CAREER: Distilling information structure from big and dirty data: Efficient learning of clusters and graphs in modern datasets

职业：从大数据和脏数据中提取信息结构：现代数据集中集群和图的高效学习

• 批准号: 1252412
• 负责人: Aarti Singh
• 金额: $ 50万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-03-01 至 2018-02-28
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1252412&HistoricalAwards=false
• 关键词: CAREER; Distilling; information; structure; big

This CAREER project aims to advance the state-of-the-art in theory and methods for extracting clusters and graphs from big and dirty datasets arising in modern application domains. Clusters and graphs provide a meaningful representation of the structure of information contained in data, e.g. in neuroscience and health care domains, clustering patients with similar phenotypes and genotypes helps identify target groups for drug design, clustering fiber tracks generated by high-resolution Digital Surface Imaging (DSI) scans of brains help identify significant neural pathways, and graph structures can reflect connectivity between brain regions. The results of this work will significantly enhance the ability to exploit such modern datasets through new methods for learning clusters and graphs from data that is large-scale, high-dimensional, under-sampled, corrupted, and often only available in a compressed or streaming representation. Specifically, this project will develop computationally efficient and principled methods for learning clusters and graphs that can (i) perform unsupervised feature selection to discard irrelevant features in high dimensions, (ii) leverage feedback based on intelligent adaptive queries that focus resources on most informative variables and features, (iii) use compressive measurement design that adapts to the information structure for measurement and computation efficiency, and (iv) be able to handle noisy streaming data. The algorithms will be accompanied with performance guarantees in the form of a precise characterization of the mis-clustering rate and graph recovery error. Additionally, the project will investigate the tradeoffs between number of measurements, computational complexity and robustness in these problems. The methods and theory developed will be evaluated through simulations as well as their applicability to real datasets in neuroscience and healthcare domain, in collaboration with practitioners from these fields. The results of this research could potentially transform many application domains that involve grouping similar variables and learning complex interactions between them, based on big and dirty datasets. In particular, the neuroscience and healthcare applications are likely have very direct and significant implications for society. Accurately mapping neural pathways will help diagnose and treat brain pathologies at an early stage, and help understand brain functioning. Clustering patients and discovering disease spreading pathways based on few measurements of relevant genetic features or indicators could help prevent and cure diseases, and also minimize healthcare costs. The research activities will be tightly integrated with education efforts that aim to develop a diverse workforce that is better equipped with cross-disciplinary tools to address the challenges of modern datasets. The education plan includes development of two inter-disciplinary courses, and enhancement of the joint Statistics & Machine Learning PhD program at Carnegie Mellon University (CMU). Outreach activities include promoting undergraduate research, broadening participation of women and underrepresented groups in STEM fields through OurCS (Opportunities for Undergraduate Research in Computer Science), Andrew?s Leap (a summer enrichment program for area high school and middle school students) and CS4HS program aimed at High School and K-8 teachers at Carnegie Mellon University. The results of this project (including publications, data sets, and software) will be disseminated online at http://www.cs.cmu.edu/~aarti/research_projects/.

这个CAREER项目旨在推进最先进的理论和方法，从现代应用领域中产生的大而脏的数据集中提取集群和图形。聚类和图形提供了数据中包含的信息结构的有意义的表示，例如在神经科学和医疗保健领域，聚类具有相似表型和基因型的患者有助于识别药物设计的目标群体，聚类由大脑的高分辨率数字表面成像（DSI）扫描生成的纤维轨迹有助于识别重要的神经通路，图形结构可以反映大脑区域之间的连通性。这项工作的结果将大大提高利用这些现代数据集的能力，通过新的方法从大规模，高维，欠采样，损坏的数据中学习聚类和图形，并且通常只能以压缩或流表示形式提供。具体来说，该项目将开发计算效率高和原则性强的方法来学习集群和图形，这些方法可以（i）执行无监督的特征选择，以丢弃高维中不相关的特征，（ii）利用基于智能自适应查询的反馈，将资源集中在最具信息性的变量和特征上，（iii）使用适应于信息结构的压缩测量设计以获得测量和计算效率，以及（iv）能够处理有噪声的流数据。该算法将伴随着性能保证的形式，一个精确的表征的误聚类率和图形恢复错误。此外，该项目将调查在这些问题中的测量数量，计算复杂性和鲁棒性之间的权衡。开发的方法和理论将通过模拟以及它们对神经科学和医疗保健领域真实的数据集的适用性进行评估，并与这些领域的从业者合作。这项研究的结果可能会改变许多应用领域，这些应用领域涉及基于大型和脏数据集对相似变量进行分组并学习它们之间的复杂交互。特别是，神经科学和医疗保健应用可能对社会产生非常直接和重大的影响。准确绘制神经通路将有助于在早期诊断和治疗大脑病变，并有助于了解大脑功能。基于对相关遗传特征或指标的少量测量，对患者进行聚类并发现疾病传播途径，可以帮助预防和治疗疾病，并最大限度地降低医疗成本。研究活动将与教育工作紧密结合，旨在培养一支多元化的劳动力队伍，更好地配备跨学科工具，以应对现代数据集的挑战。该教育计划包括开发两门跨学科课程，并加强卡内基梅隆大学（CMU）的联合统计机器学习博士课程。外联活动包括促进本科生研究，通过OurCS（计算机科学本科生研究机会）扩大妇女和代表性不足的群体在STEM领域的参与，安德鲁？s Leap（针对地区高中和中学生的夏季充实计划）和针对卡内基梅隆大学高中和K-8教师的CS 4 HS计划。该项目的成果（包括出版物、数据集和软件）将在http://www.cs.cmu.edu/~aarti/research_projects/网站上公布。