Pilot Projects to Explore Large Data Sets

点击翻译按钮获取中文摘要

Large data sets are a given in modern industry, technology and science, and demand statistical attention to treat such central issues as detecting new patterns and relationships in a reliable and computationally feasible way. While the need is apparent, paths to bringing statistical science to bear on large data sets are not clearly mapped, in part because the sheer volume of the data prevents direct use of standard techniques. This proposal comprises a pair of interconnected pilot projects targeted to stimulate statistical science entree to this rapidly changing terrain. Each has a major industria;l partner, which has committed substantial resources. The projects and partners are Drug Discovery (Glaxo Wellcome, Research Traingle Park, NC) and Telecommunications Fraud (AT&T Laboratories, Murray Hill, NJ). In both instances there are specific scientific isues with high-stakes implications for the industry at large; each is speculative, in the sense that the path from data to information to knowledge is not known in advance. In drug discovery, the critical problem is to find new potent, non-toxic compounds. New statistical methods will be developed to search substantial data sets generated by recent advances in robotic synthesis and screening in order to identify key features of compounds, leading to better ones, in the presence of highly complex (and very high-dimensional) descriptions of the molecules. Within the hundreds of millions of long distance calls per day a small fraction are illegal or unauthorized, but result in costs of hundreds of millions of dollars annually. The problem is to detect and characterize, as rapidly as possible, patterns of transactions that are unusual, and potentially fraudulent, with special attention to controlling error rates (false alarms, failures to detect), an issue exacerbated by the enormous multiplicity of decisions made on a continual basis. The two problem areas have common features: sequential statistical search for important or unusual patters in the data; data that are highly complex in sheer quantity, in high-dimensional description of each data point, or in the diversity of their sources. Approaching these issues will be done by teams of cross-disciplinary researchers at distributed sites and closely managed by NISS. This GOALI project is jointly supported by the MPS Office of Multidisciplinary Activities (OMA) and the Division of Mathematical Sciences (DMS).