CAREER: Ordered Alignment Methods for Complex, High-Dimensional Data

数字设备正在以前所未有的速度收集时间数据，导致时间序列数据集的爆炸式增长，从生物医学数据到传感器网络再到多媒体数据。构建自动化工具来搜索、分类、检测和发现时间序列数据中的模式，需要有序的比对方法来确定两个序列之间的相似性。然而，这些方法的实用性与构成现代分析领域的复杂、高维数据之间存在差距。该项目旨在解决造成这一差距的三个关键因素，使用音乐和多媒体数据作为具有挑战性的测试平台。具体来说，这个项目的目标是设计有序的对齐方法，这些方法具有足够的可扩展性，可以用于交互式多媒体应用程序，足够灵活，可以处理复杂的结构化多媒体数据，并集成到现代机器学习模型中。该项目将在一所文理学院进行，涉及指导10-15名本科生，开发基于课程的研究经验，并在HMC与德国领先的音频和多媒体研究小组之间建立研究伙伴关系。该项目将解决关于有序对齐方法的三个基本问题，如动态时间翘曲（DTW）。第一个问题是，“我们如何使有序对齐具有足够的可伸缩性，以便在交互式多媒体应用程序中使用？”这将通过开发充分利用现代硬件的DTW的可并行近似值，以及对非常长的序列的DTW的基于哈希的近似值来探索。第二个问题是，“我们如何使有序对齐足够灵活，以处理复杂的结构化多媒体数据？”这将通过利用多个有序对齐阶段和基于状态的时间翘曲的组合来解决，其中时间翘曲特征依赖于潜在状态。第三个问题是，“我们如何将有序对齐整合到神经网络训练中？”这将通过解决两个当前的障碍来实现：通过离散特征的语言模型预训练来避免冷启动对齐问题，以及通过使用小批量和时代级数据处理来处理长序列。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Digital devices are collecting temporal data at an unprecedented rate, leading to an explosion of time series datasets ranging from biomedical data to sensor networks to multimedia data. Building automated tools to search, classify, detect, and discover patterns in time series data requires ordered alignment methods to determine the similarity between two sequences. However, there is a gap between the usefulness of these methods and the complex, high-dimensional data that make up much of the modern analytic landscape. This project aims to address three key factors contributing to this gap, using music and multimedia data as a challenging testbed. Specifically, the goal of this project is to design ordered alignment methods that are scalable enough to be used in interactive multimedia applications, flexible enough to handle complex, structured multimedia data, and integrated into modern machine learning models. The project will be carried out at a liberal arts college and involve mentoring 10-15 undergraduate research students, developing course-based research experiences, and establishing a research partnership between HMC and a leading audio and multimedia research group in Germany.The project will address three fundamental questions about ordered alignment methods like dynamic time warping (DTW). The first question is, “How can we make ordered alignment scalable enough to be used in interactive multimedia applications?”. This will be explored by developing parallelizable approximations of DTW that fully utilize modern hardware, as well as hashing-based approximations of DTW for very long sequences. The second question is, “How can we make ordered alignment flexible enough to handle complex, structured multimedia data?”. This will be addressed by utilizing compositions of multiple ordered alignment stages and state-based time warping, in which time warping characteristics depend on a latent state. The third question is, “How can we integrate ordered alignment into neural network training?”. This will be pursued by addressing two current obstacles: avoiding the cold start alignment problem through language model pretraining of discretized features, and handling long sequences by utilizing both mini-batch and epoch-level data processing.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.