权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

StatScale: Statistical Scalability for Streaming Data

StatScale：流数据的统计可扩展性

基本信息

批准号：
EP/N031938/1
负责人：
Idris Eckley
金额：
$ 350.52万
依托单位：
Lancaster University
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2016
资助国家：
英国
起止时间：
2016 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FN031938%2F1
关键词：
StatScale Statistical Scalability Streaming Data

项目摘要

We live in the age of data. Technology is transforming our ability to collect and store data on unprecedented scales. From the use of Oyster card data to improve London's transport network, to the Square Kilometre Array astrophysics project that has the potential to transform our understanding of the universe, Big Data can inform and enrich many aspects of our lives. Due to the widespread use of sensor-based systems in everyday life, with even smartphones having sensors that can monitor location and activity level, much of the explosion of data is in the form of data streams: data from one or more related sources that arrive over time. It has even been estimates that there will be over 30 billion devices collecting data streams by 2020. The important role of Statistics within "Big Data" and data streams has been clear for some time. However the current tendency has been to focus purely on algorithmic scalability, such as how to develop versions of existing statistical algorithms that scale better with the amount of data. Such an approach, however, ignores the fact that fundamentally new issues often arise when dealing with data sets of this magnitude, and highly innovative solutions are required. Model error is one such issue. Many statistical approaches are based on the use of mathematical models for data. These models are only approximations of the real data-generating mechanisms. In traditional applications, this model error is usually small compared with the inherent sampling variability of the data, and can be overlooked. However, there is an increasing realisation that model error can dominate in Big Data applications. Understanding the impact of model error, and developing robust methods that have excellent statistical properties even in the presence of model error, are major challenges. A second issue is that many current statistical approaches are not computationally feasible for Big Data. In practice we will often need to use less efficient statistical methods that are computationally faster, or require less computer memory. This introduces a statistical-computational trade-off that is unique to Big Data, leading to many open theoretical questions, and important practical problems.The strategic vision for this programme grant is to investigate and develop an integrated approach to tackling these and other fundamental statistical challenges. In order to do this we will focus in particular on analysing data streams. An important issue with this type of data is detecting changes in the structure of the data over time. This will be an early area of focus for the programme, as it has been identified as one of seven key problem areas for Big Data. Moreover it is an area in which our research will lead to practically important breakthroughs. Our philosophy is to tackle methodological, theoretical and computational aspects of these statistical problems together, an approach that is only possible through the programme grant scheme. Such a broad perspective is essential to achieve the substantive fundamental advances in statistics envisaged, and to ensure our new methods are sufficiently robust and efficient to be widely adopted by academics, industry and society more generally.

我们生活在数据时代。技术正在改变我们以前所未有的规模收集和存储数据的能力。从使用牡蛎卡数据来改善伦敦的交通网络，到有可能改变我们对宇宙的理解的平方公里阵列天体物理学项目，大数据可以为我们的生活提供信息并丰富我们生活的许多方面。由于基于传感器的系统在日常生活中的广泛使用，甚至智能手机也具有可以监控位置和活动水平的传感器，因此大部分数据爆炸都是以数据流的形式出现的：来自一个或多个相关来源的数据随着时间的推移而到达。据估计，到2020年将有超过300亿台设备收集数据流。一段时间以来，统计在“大数据”和数据流中的重要作用已经很明显。然而，目前的趋势是纯粹关注算法的可扩展性，例如如何开发现有统计算法的版本，以便更好地扩展数据量。然而，这种方法忽略了这样一个事实，即在处理这种规模的数据集时，经常会出现根本性的新问题，需要高度创新的解决方案。模型误差就是这样一个问题。许多统计方法都是基于对数据使用数学模型。这些模型只是真实的数据生成机制的近似值。在传统的应用中，这种模型误差与数据固有的抽样变异性相比通常很小，可以忽略不计。然而，越来越多的人意识到，模型误差可能在大数据应用中占主导地位。了解模型误差的影响，并开发出强大的方法，即使在模型误差的存在下也具有出色的统计特性，这是一个重大的挑战。第二个问题是，目前的许多统计方法在计算上对大数据不可行。在实践中，我们经常需要使用效率较低的统计方法，这些方法在计算上更快，或者需要更少的计算机内存。这引入了大数据所特有的计算-计算权衡，导致许多开放的理论问题和重要的实际问题。该计划赠款的战略愿景是调查和开发一种综合方法来应对这些和其他基本的统计挑战。为了做到这一点，我们将特别关注分析数据流。这类数据的一个重要问题是检测数据结构随时间的变化。这将是该计划的早期重点领域，因为它已被确定为大数据的七个关键问题领域之一。此外，这是一个我们的研究将导致实际重要突破的领域。我们的理念是共同解决这些统计问题的方法、理论和计算方面，这种方法只有通过方案赠款计划才能实现。这种广泛的视角对于实现所设想的统计方面的实质性根本性进展至关重要，并确保我们的新方法足够强大和有效，以便被学术界，工业界和社会广泛采用。