EAGER-DynamicData: Judicious Censoring, Random Sketching, and Efficient Validate for Learning Patterns from Dynamically-Changing and Large-Scale Data Sets

抽象的。随着无处不在的传感器不断收集和记录大量信息，毫无疑问，这是一个数据泛滥的时代。从这些动态和大量的数据中学习，预计将带来重大的科学和工程进步，并随之而来的生活质量的改善。目前用于探索性研究的早期概念拨款旨在开发潜在的变革性模式识别技术，该技术将在动态变形(例如，由于患者运动)心脏磁共振图像以及从大规模医疗保健数据集中提取信息方面进行专门测试。该项目应对的巨大挑战包括：庞大的在线数据集和不断增长的数据集，这使得无法运行分析，尤其是以批量形式运行；此外，大规模数据集不可避免地存在噪音、动态、不完整、容易出现离群值和(非)故意遗漏，以及容易受到网络攻击的事实。该项目的大规模分析还将渗透到环境数据挖掘、神经科学和未来电网的跨学科利益中。在更广泛的范围内，所开发的技术将为基础科学和工程研究提供宝贵的工具，并促进社会对新兴大数据技术的接受，同时培训下一代数据科学专业人员。这项探索性研究的早期概念拨款旨在通过提出大规模学习工具及其性能分析来应对大数据挑战，这些工具利用两个未经测试、但具有潜在变革性的想法来提取海量和动态数据集的计算负担得起但信息丰富的子集，即i)自适应审查，以及ii)随机数据绘制和验证。此项目中的数据可以是静态的，也可以是非静态的；它们可以成批或按顺序提供(也称为在线)模式；它们可以以矢量、矩阵或一般多路阵列(称为张量)的形式收集；存在噪声、可能的离群值和(非)故意遗漏；以及数据处理在自适应或非自适应模式下可以是线性或非线性的。拟议的高风险-高回报研究集中了基本的大数据工具，包括压缩采样、矩阵和张量补全、异常和离群值识别、在线和并行优化技术。根据主要的推理任务，将进行三个相互交织的研究推动力：T1)大规模回归的自适应审查；T2)动态大规模张量的子空间跟踪和分配；以及T3)大规模聚类和分类的草图和验证。由此产生的创新工具将在医疗数据和多维磁共振成像中进行测试，最终目标是实时获取、处理和显示高分辨率生物医学电影。

Abstract. With pervasive sensors continuously collecting and recording massive amounts of information, there is no doubt this is an era of data deluge. Learning from these dynamic and large volumes of data is expected to bring significant science and engineering advances along with consequent improvements in quality of life. The present early-concept grant for exploratory research aims to develop potentially transformative pattern recognition techniques that will be specifically tested on dynamically deforming (due to e.g., patient motion) cardiac magnetic resonance images, as well as on information extraction from large-scale healthcare datasets. Big challenges that this project addresses, include the sheer volume of online and growing datasets, which makes it impossible to run analytics especially in batch form; and also the facts that large-scale datasets are inevitably noisy, dynamic, incomplete, prone to outliers and (un)intentional misses, as well as vulnerable to cyber-attacks. The project's large-scale analytics will also permeate interdisciplinary benefits to environmental data mining, neuroscience, and the future power grid. At a broader scale, the developed technologies will provide valuable tools for foundational science and engineering research, and promote societal embracing of the emergent big data technologies, along with training the next-generation of data science professionals.This early-concept grant for exploratory research aspires to tackle big data challenges by putting forth large-scale learning tools and their performance analyses that leverage two untested, but potentially transformative, ideas for extracting computationally affordable yet informative subsets of massive and dynamic datasets, namely i) adaptive censoring, and ii) random data sketching-and-validation. Data in this project can be stationary or nonstationary; they become available in batch or sequential (a.k.a. online) modes; they can be collected in vectors, matrices or general multi-way arrays (called tensors); noise, possibly outliers and (un)intentional misses are present; and data processing can be linear or nonlinear in adaptive or non-adaptive modes. The proposed high risk-high payoff research lies at the intersection of essential big data tools including compressive sampling, matrix and tensor completion, anomaly and outlier identification, online and parallel optimization techniques. In accordance with the major inference tasks, three intertwined research thrusts will be pursued: T1) Adaptive censoring for large-scale regressions; T2) Subspace tracking and imputation for dynamic large-scale tensors; and T3) Sketch-and-validate for large-scale clustering and classification. The resultant innovative tools will be tested in healthcare data, and multi-dimensional magnetic resonance imaging, having as ultimate goal high-resolution biomedical movies to be acquired, processed, and displayed in real time.