Learning, Visualization, and the Analysis of Large-scale Multiple-media Data

该项目由学习和智能系统倡议资助，部分资金由MPS/OMA办公室提供。 科学和工程数据现在以新的形式大量出现。虽然从纯数值数据中分析和学习的问题已经得到了大量的研究，但我们目前缺乏分析多媒体数据集的原则性方法，这些数据集构成了许多现代实证研究的基础。这些现代数据集包含数字特征、符号逻辑描述、图像、文本、声音和其他媒体的混合。该项目提供了一个跨学科的研究工作，以创建统计基础和实用的机器学习算法，以利用越来越多的多媒体数据集。研究计划是通过与几个具有重大科学和社会重要性的大型多媒体数据库合作，开发解决这一问题的新方法。 该研究将为多媒体数据的分析提供理论基础和实用算法。例如，许多医疗机构现在收集详细的患者记录，可以分析这些记录以预测未来患者的治疗结果。这些医疗记录通常是由数字特征（例如，温度），符号特征（例如，性别），图像（例如，X射线），其它仪器数据（例如，EKG）、文本（例如，医生笔记）和其他数据。目前的数据分析算法简单地忽略了这些可用的功能，因为我们缺乏很好的理解方法来分析这样的多媒体数据。目前的研究试图开发新的方法，将能够利用收集在这些数据集的全部信息。其目标是扩展数据解释的基础，这些基础构成了许多实验科学和工程学科的基础。

This project is being funded through the Learning and Intelligent Systems (LIS) Initiative, with funds partially provided by the MPS/OMA office. Scientific and engineering data now come in large amounts and in new forms. Although the problem of analyzing and learning from purely numerical data has been heavily studied, we currently lack principled methods for analyzing the multiple-media data sets that form the basis of many modern empirical studies. These modern data sets contain a mixture of numerical features, symbolic logic descriptions, images, text, sound, and other media. This project offers an interdisciplinary research effort to create the statistical foundations and practical machine learning algorithms needed to take advantage of the growing number of such multiple-media data sets. The research plan is to develop new approaches to this problem by working with several large-scale multiple-media databases of significant scientific and societal importance. This research will provide the theoretical foundations and practical algorithms for analyzing multiple-media data in a broad range of application domains. For example, many medical institutions now collect detailed patient records that can be analyzed to predict treatment outcomes for future patients. These medical records are typically multiple-media records consisting of numerical features (e.g., temperature), symbolic features (e.g., gender), images (e.g., x-rays), other instrument data (e.g., EKG), text (e.g., physicians' notes), and other data. Current data analysis algorithms simply ignore most of these available features, because we lack well-understood methods for analyzing such multiple-media data. The current research seeks to develop new approaches that will be able to utilize the full information collected in such data sets. The goal is to extend the foundations of data interpretation that form the basis for many experimental sciences and engineering disciplines.