Pilot Projects to Explore Large Data Sets

在现代工业、技术和科学中，大数据集是给定的，需要统计上的关注，以可靠和计算上可行的方式处理诸如检测新模式和关系等核心问题。尽管需求是显而易见的，但将统计科学应用于大型数据集的途径并未被清楚地描绘出来，部分原因是数据量太大，无法直接使用标准技术。这项建议包括两个相互关联的试点项目，旨在刺激统计科学进入这一快速变化的领域。每一家都有一个主要的行业；L合伙人，其中投入了大量的资源。这些项目和合作伙伴是药物发现(葛兰素史克，北卡罗来纳州研究训练公园)和电信欺诈(AT&&T实验室，新泽西州默里希尔)。在这两种情况下，都存在对整个行业具有高风险影响的特定科学问题；每一种情况都是投机性的，从某种意义上说，从数据到信息再到知识的路径并不是事先知道的。在药物发现中，关键问题是寻找新的有效、无毒的化合物。将开发新的统计方法来搜索由机器人合成和筛选方面的最新进展产生的大量数据集，以便在存在对分子的高度复杂(和非常高维)描述的情况下识别化合物的关键特征，从而产生更好的特征。在每天数以亿计的长途电话中，一小部分是非法或未经授权的，但每年导致数亿美元的成本。问题是要尽快发现和描述不寻常和潜在欺诈的交易模式，并特别注意控制错误率(错误警报、检测失败)，这一问题因不断作出的各种决定而加剧。这两个问题领域有共同的特点：顺序统计搜索数据中的重要或不寻常的模式；数据在数量上高度复杂，每个数据点的高维描述，或其来源的多样性。处理这些问题将由分布在不同地点的跨学科研究人员团队完成，并由NISS密切管理。该GOALI项目由MPS多学科活动办公室(OMA)和数学科学处(DMS)共同支持。

Large data sets are a given in modern industry, technology and science, and demand statistical attention to treat such central issues as detecting new patterns and relationships in a reliable and computationally feasible way. While the need is apparent, paths to bringing statistical science to bear on large data sets are not clearly mapped, in part because the sheer volume of the data prevents direct use of standard techniques. This proposal comprises a pair of interconnected pilot projects targeted to stimulate statistical science entree to this rapidly changing terrain. Each has a major industria;l partner, which has committed substantial resources. The projects and partners are Drug Discovery (Glaxo Wellcome, Research Traingle Park, NC) and Telecommunications Fraud (AT&T Laboratories, Murray Hill, NJ). In both instances there are specific scientific isues with high-stakes implications for the industry at large; each is speculative, in the sense that the path from data to information to knowledge is not known in advance. In drug discovery, the critical problem is to find new potent, non-toxic compounds. New statistical methods will be developed to search substantial data sets generated by recent advances in robotic synthesis and screening in order to identify key features of compounds, leading to better ones, in the presence of highly complex (and very high-dimensional) descriptions of the molecules. Within the hundreds of millions of long distance calls per day a small fraction are illegal or unauthorized, but result in costs of hundreds of millions of dollars annually. The problem is to detect and characterize, as rapidly as possible, patterns of transactions that are unusual, and potentially fraudulent, with special attention to controlling error rates (false alarms, failures to detect), an issue exacerbated by the enormous multiplicity of decisions made on a continual basis. The two problem areas have common features: sequential statistical search for important or unusual patters in the data; data that are highly complex in sheer quantity, in high-dimensional description of each data point, or in the diversity of their sources. Approaching these issues will be done by teams of cross-disciplinary researchers at distributed sites and closely managed by NISS. This GOALI project is jointly supported by the MPS Office of Multidisciplinary Activities (OMA) and the Division of Mathematical Sciences (DMS).