Decomposition Framework for Non-convex Nonsmooth Optimization with Applications in Data Analytics

传感器、通信和存储技术的快速发展导致数据的可用性达到前所未有的规模。 根据来源的不同，这些数据可能代表测量、图像、文本、时间序列和各种其他格式。 在将越来越多的数据转化为有用和可操作的信息方面仍然存在重大挑战。本项目的目标是通过现代大规模优化的透镜解决这一信息困境。 该项目支持研究有效处理大规模、非结构化、复杂数据的方法，以便在生物信息学、智能能源系统、制造业和医疗保健等应用中使用。 该项目还将让研究生参与研究活动，并将通过PI大学的现有计划支持对本科STEM学生的推广。该项目将专注于构建一个通用的优化和计算框架，以实现一些有前途但具有挑战性的大规模数据密集型应用。 这项研究包括两个主要方面。 第一个将建立和分析一个新的基于优化的原始对偶分解框架，将一个大的，紧密耦合的，非凸的问题转化为一系列独立的子问题可解决的并行机。 第二个应用分解框架的一些重要的新兴的数据密集型应用程序，包括高维聚类，主题建模，和强大的高维回归。 基本的问题，如最优性，收敛速度，并在高维的可扩展性将进行调查。 该项目将使用来自两个重要能源应用的数据来测试所开发的方法：智能电表和实时住宅光伏逆变器。

Rapid advances in sensor, communication and storage technologies have led to the availability of data on an unprecedented scale. Depending on the source, these data may represent measurements, images, texts, time-series and a variety of other formats. Significant challenges remain in translating the increasing amount of data to useful and actionable information. The objective of this project is to address this information dilemma through the lens of modern large-scale optimization. This project supports research on methods to effectively process large-scale, unstructured, complex data so as to be usable in applications such as bioinformatics, smart energy systems, manufacturing, and healthcare. The project will also engage graduate students in the research activities and will support outreach to undergraduate STEM students through an existing program at the PI's university. This project will focus on the construction of a general optimization and computational framework that enables a number of promising but challenging large-scale data-intensive applications. The research comprises two major thrusts. The first will build and analyze a novel optimization-based primal-dual decomposition framework that transforms a large, tightly coupled, non-convex problem into a sequence of independent subproblems solvable by parallel machines. The second applies the decomposition framework to a number of important emerging data-intensive applications, including high-dimensional clustering, topic modeling, and robust high-dimensional regression. Fundamental questions, such as optimality, convergence rates, and scalability in high dimension will be investigated. The project will test the developed methods using data from two important energy applications: smart energy meters and real-time residential photovoltaic inverters.