Pilot Projects to Explore Large Data Sets

大型数据集在现代工业、技术和科学中是一个既定的事实，需要统计学的关注来处理诸如以可靠和计算上可行的方式检测新模式和关系等核心问题。 虽然这种需要是显而易见的，但将统计科学应用于大型数据集的途径并没有明确规划，部分原因是数据量庞大，无法直接使用标准技术。 该提案包括一对相互关联的试点项目，旨在刺激统计科学进入这一迅速变化的领域。 每一个都有一个主要的工业伙伴，投入了大量的资源。 这些项目和合作伙伴是药物发现（Glaxo Wellcome，Research Traingle Park，NC）和电信欺诈（AT T实验室，Murray Hill，NJ）。 在这两种情况下，都有对整个行业具有高风险影响的具体科学知识;每一种都是推测性的，因为从数据到信息再到知识的路径事先并不知道。 在药物发现中，关键的问题是找到新的有效的，无毒的化合物。 将开发新的统计方法来搜索机器人合成和筛选的最新进展所产生的大量数据集，以确定化合物的关键特征，从而在分子的高度复杂（和非常高维）描述的情况下得到更好的化合物。 在每天数以亿计的长途电话中，有一小部分是非法或未经授权的，但每年造成数亿美元的费用。 问题是要尽可能快地发现和确定不寻常的、可能具有欺诈性的交易模式，特别注意控制错误率（误报、未能发现），由于不断作出的决定数量巨大，这一问题更加严重。 这两个问题领域具有共同的特征：对数据中重要或不寻常模式的顺序统计搜索;在绝对数量上高度复杂的数据，每个数据点的高维描述，或其来源的多样性。 这些问题的解决将由跨学科研究人员团队在分散的地点进行，并由NISS密切管理。 该GOALI项目由MPS多学科活动办公室（OMA）和数学科学部（DMS）联合支持。

Large data sets are a given in modern industry, technology and science, and demand statistical attention to treat such central issues as detecting new patterns and relationships in a reliable and computationally feasible way. While the need is apparent, paths to bringing statistical science to bear on large data sets are not clearly mapped, in part because the sheer volume of the data prevents direct use of standard techniques. This proposal comprises a pair of interconnected pilot projects targeted to stimulate statistical science entree to this rapidly changing terrain. Each has a major industria;l partner, which has committed substantial resources. The projects and partners are Drug Discovery (Glaxo Wellcome, Research Traingle Park, NC) and Telecommunications Fraud (AT&T Laboratories, Murray Hill, NJ). In both instances there are specific scientific isues with high-stakes implications for the industry at large; each is speculative, in the sense that the path from data to information to knowledge is not known in advance. In drug discovery, the critical problem is to find new potent, non-toxic compounds. New statistical methods will be developed to search substantial data sets generated by recent advances in robotic synthesis and screening in order to identify key features of compounds, leading to better ones, in the presence of highly complex (and very high-dimensional) descriptions of the molecules. Within the hundreds of millions of long distance calls per day a small fraction are illegal or unauthorized, but result in costs of hundreds of millions of dollars annually. The problem is to detect and characterize, as rapidly as possible, patterns of transactions that are unusual, and potentially fraudulent, with special attention to controlling error rates (false alarms, failures to detect), an issue exacerbated by the enormous multiplicity of decisions made on a continual basis. The two problem areas have common features: sequential statistical search for important or unusual patters in the data; data that are highly complex in sheer quantity, in high-dimensional description of each data point, or in the diversity of their sources. Approaching these issues will be done by teams of cross-disciplinary researchers at distributed sites and closely managed by NISS. This GOALI project is jointly supported by the MPS Office of Multidisciplinary Activities (OMA) and the Division of Mathematical Sciences (DMS).