Guided Analytics for the Visual Exploration of Higher Dimensional Data

在现代数据中，规模很重要。对于我们可能测量的几乎每一个变量，都有大量的，也许是数以百万计的观测结果，而且，在每次观测中测量100个，甚至1000个变量越来越成为常规。例如，想象一下标准普尔500指数股票在五年期间的每日收盘价。数据由大约1,250 （= 5 x 250）天（或观察）和大约500只股票（或变量）组成。例如，如果记录每小时的价格，并记录每只股票每小时的开盘、收盘和平均价格，那么这些计数可能会大得多。在检查这些数据时，我们希望找到规律，发现之前没有预料到的东西，遇到一个“啊哈！”的科学发现时刻。为此，人类的视觉系统已经进化到能够发现不寻常的东西，注意到模式，并看到关系。计算机交互式数据可视化确实允许人们“看到”数据中正在发生的事情。但数据的规模是压倒性的。为了简单地了解每对股票之间的关系，我们必须查看124,750个地块！这是不可能的。建议的研究是将数学结构、计算资源和统计模型用于解决问题，让计算机指导分析师只查看可能“有趣”的少数几张图（甚至100张也可以节省）。它不仅提供了找到这些数据的方法，还提供了有效查看这些数据并将其合并到报告中的技术。这需要确定哪些情节是“有趣的”，以及我们可以根据数据计算出哪些情节是有趣的。在某些分析中有趣的内容在其他分析中并不有趣，因此必须考虑许多不同的“有趣”度量。这项研究提出了几个这样的措施是广泛适用的，以及一些是特殊的选择应用。一旦我们对124,750个可能的地块计算出不同的兴趣度量，我们必须从中进行选择。这项研究为这种选择提供了工具。一旦我们选择了10个或100个有趣的图的子集，我们需要了解它们是如何关联的，一对变量与另一对变量之间的关系，以及我们可以从这些关系中推断出什么。在这里，研究引入了数学图论来提供一个将图相互联系起来的结构。对这种结构的分析可能会揭示变量之间的关系。将开发这些结构的概率和统计模型，以帮助确保我们的推断是可靠的。在整个过程中，软件将被设计并（通过开源许可）向公众传播。通过将软件交给数据分析师，我们希望使探索性可视化、数据分析和科学发现变得更容易。

With modern data, size matters. Large numbers, perhaps millions, of observations are available for almost every variable we might measure, and, increasingly, it has become routine to measure 100s, even 1000s of variables on each observation. For example, imagine looking at daily closing prices of S&P 500 stocks over a five-year period. The data consists of about 1,250 (= 5 x 250) days (or observations) and about 500 stocks (or variables). These counts could be dramatically larger, if, for example, hourly prices were recorded and for each hour the opening, closing, and average prices for that hour were recorded for every stock. In examining such data, we hope to find patterns, to uncover something not previously anticipated, to encounter an "aha!" moment of scientific discovery. To this end, the human visual system has evolved to literally spot the unusual, to notice patterns, and to see relations. Computer interactive data visualization literally allows one "to see" what is going on in the data. But the size of the data is overwhelming. To simply look at the relation between every pair of stocks, we would have to look at 124,750 plots! It is not possible. The proposed research is to bring mathematical structure, computational resources, and statistical modelling to bear on the problem, to have the computer guide the analyst to view only those few plots (even 100 would be a saving) that might be "interesting" to look at. It will provide not just the means to find these, but the technology to efficiently view them, and incorporate them into a report. This requires determining which plots are "interesting", and what we might calculate on the data to tell that a plot was interesting. What is interesting in some analyses is not interesting in others, so many different measures of "interestingness" must be considered. This research proposes to develop several such measures that are of wide applicability as well as some that are peculiar to selected applications. Once we have calculated different measures of interestingness on our 124,750 possible plots, we must select amongst them. The research is providing tools for such selection. Once we have selected our subsets of 10s or 100s of interesting plots, we need to understand how they are related, one pair of variables to another, and what we might infer from those relations if anything. Here, the research brings mathematical graph theory to provide a structure relating the plots to one another. Analysis of that structure might then shed light on the relations between variables. Probability and statistical models of these structures will be developed to help ensure our inferences are reliable. Throughout, software will be designed and disseminated (through open-source licensing) to the general public. By putting the software in the hands of the data analysts we hope to make exploratory visualization, data analysis, and scientific discovery a little easier.