Large-Scale Machine Learning: Sparse Representations for Signal/Image Processing and System Modeling

能源网格网络、Web服务、生物信息学和传感器网络等基于计算机的技术的快速发展，对能够处理海量数据集、多假设和高维数据的新的模式识别/机器学习算法产生了巨大的需求。这定义了被称为大规模机器学习的领域。在这个方案中，我希望将我目前对非参数/半参数学习方法的研究扩展到大规模机器学习问题。这一挑战不仅包括开发新的数学技术，而且还包括在信号/图像处理和系统建模任务的框架内验证这些技术。还计划在电力工程、显微成像以及生物和网络信号的多变点检测等领域的具体应用中测试所提出的方法。我们的研究方案依赖于将现代非参数/半参数学习方法与给定过程的稀疏表示概念相结合的思想。稀疏性被粗略地定义为基于使用少量主导项的表示对所检查的问题的解。关键是发现稀疏(低维)表示及其在给定环境中的形式。到目前为止，稀疏学习算法主要集中在参数(有限维)线性模型上，其中观察到的数据点被表示为表示矩阵(词典)的少量向量的线性组合。在这个方案中，我们的目标是开发一类利用给定对象固有稀疏性的非参数和半参数学习算法。因此，我们希望超越迄今使用的线性和有限维稀疏模型，并将这些发现应用于多维系统、计算对称性和多个变点检测问题。我们相信，所提出的稀疏对象表示方法可以大大扩大机器学习的应用范围，并在所提出的案例研究的背景下改进现有算法。

Rapid advances in computer based technologies such as energy grid networks, web services, bioinformatics, and sensor networks, create a great demand for new pattern recognition/machine learning algorithms able to cope with massive data sets, multiple hypothesis, and data of high dimensionality. This defines the field referred to as large-scale machine learning. In this proposal, I wish to extend my current research on nonparametric/semiparametric learning methods to large-scale machine learning problems. This challenge includes not only the development of new mathematical techniques, but also to verify them in the framework of signal/image processing and system modeling tasks. Testing of the proposed methods in concrete applications within the areas of power engineering, microscopic imaging and multiple change-point detection for biological and network signals is also planned. Our research proposal relies on the idea of blending the modern nonparametric/semiparametric learning methodology with the concept of sparse representations of a given process. The sparsity is roughly defined as a solution to the examined problem based on the representation using a small number of dominating terms. The key is to discover a sparse (low-dimensional) representation and its form in a given setting. Thus far, sparse learning algorithms have mostly focused on parametric (finite dimensional) linear models, where an observed data point is expressed as a linear combination of a small number of vectors of the representation matrix (dictionary). In this proposal we aim at developing a class of nonparametric and semiparametric learning algorithms that utilize the inherent sparsity of a given object. Hence, we wish to go beyond the thus far used linear and finite dimensional sparsity models and apply these findings to problems stemming from multidimensional systems, computational symmetry and multiple change-point detection problems. We believe that the proposed sparse object representation methodology can substantially enlarge a scope of machine learning applications and improve the existing algorithms within the context of proposed case studies.