Visually enhanced exploration of high-dimensional data

我们建议创建新的软件技术来探索大型高维数据集，其中数据点的数量从数万到数百万不等。我们的工具将帮助人们洞察复杂的问题，这些问题取决于从几十到数百个变量之间的关系。以前处理这些问题的方法假设，只有在整个有限数据集可以加载到软件系统中之后，才会进行探索。我们还将解决不能简单地加载完整数据集的问题领域；必须首先进行采样，然后才能使用交互式可视化或机器学习来解释它。出现这种情况的原因有几个。有些数据集本质上是连续的，必须对其进行采样以对其进行离散化。其他数据集是有限的，但太大了，无法完整处理。在一些应用领域，采集样本的成本在计算上是如此昂贵，以至于只需采集最少量的样本。我们的方法基于从信号处理和统计分析、可视化、机器学习和人机交互的紧密耦合的方法。这四种方法之间的协作相互作用，其中每一种方法都从其他方法的最新结果中获得信息，并反过来为其他三种方法的下一阶段调查提供信息，这将是我们雄心勃勃的目标成功的关键。我们将i)为高维空间开发新的采样策略；ii)为快速交互探索开发新的降维技术；iii)为数据集分类开发新的机器学习方法；以及iv)为这三种方法开发新的交互技术和用户界面。我们还将进行非正式和正式评估，以判断我们的目标领域问题是否正在得到解决；以及vi)展示一个集成系统，作为我们实验的平台和向我们的合作公司转让技术的手段。

We propose to create new software technology for the exploration of large high-dimensional datasets, where the number of data points ranges from tens of thousands to millions. Our tools will help people develop insight into complex problems that depend on the relationships between many variables, ranging from tens to hundreds. Previous approaches to handling these problems assumed that exploration would only occur after an entire, finite dataset could be loaded into a software system. We will also address problem domains where the full dataset cannot simply be loaded; sampling must occur first, before interactive visualization or machine learning can be used to interpret it. This situation can occur for several reasons. Some datasets are inherently continuous and must be sampled to discretize them. Other datasets are finite but simply too large to handle in their entirety. In some application domains, the cost of taking a sample is so computationally expensive that only a minimal number of them should be taken.Our approach is based on a tight coupling of approaches from signal processing and statistical analysis, visualization, machine learning, and human-computer interaction. The collaborative interplay between these four approaches, where each is informed by the results to date on the others, and in turn informs the next stage of investigating the other three, will be critical to the success of our ambitious goals.We will i) develop new sampling strategies for high-dimensional spaces; ii) develop new dimensionality reduction techniques for fast interactive exploration; iii) develop new machine learning approaches for categorizing datasets; and iv) develop novel interaction techniques and user interfaces for these three approaches. We will also v) conduct both informal and formal evaluations to judge whether our targeted domain problems are being solved; and vi) demonstrate an integrated system as a platform for our experiments and a means for technology transfer to our partner companies.