Methods for big data, sparsity, and environmental thresholds

大数据、稀疏性和环境阈值的方法

• 批准号: RGPIN-2021-03970
• 负责人: Tomal, Jabed
• 金额: $ 1.31万
• 依托单位: Thompson Rivers University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=742798
• 关键词: Methods; big; data; sparsity; environmental

With the rapid progress of technology, research in science and engineering is undergoing a revolution as big data are harnessed for various purposes. Thus, the long-term goal of this research program is to advance the science in statistics, machine learning and statistical ecology by exploiting the useful information in big data. The short-term goals are developing: ensemble of models based on diverse subsets of variables to improve prediction performances for the response variable of interest with applications to big data in drug discovery and genetics, Bayesian statistical tests and inference for sparsity in big data with applications to genetics, and ecological models to detect environmental thresholds via the changes in slope and variance against human induced disturbances to nature. Unlike discarding information, the applicant proposes ensemble methods which utilize more useful variables instead of throwing some out. The hypothesis is that filtering variables is like losing information which reduces the prediction power of a model. To address this issue, the applicant proposes methods which group useful variables into diverse subsets and aggregates them in an ensemble by fitting a model to each subset and aggregating models across the subsets. The novelty of the method is that the subsets of variables are obtained adaptively, rather than the current trend of using either known or random subsets. Many statistical methods force sparsity, the condition of scant information in big data, in model building which lacks enough reasons and justification. Instead, the applicant proposes Bayesian statistical tests for sparsity to be used before model building. The novelty of the method is that the amount of sparsity to induce in model building is supported by the evidence measured in data. Relationship between an ecological response and environmental disturbance often represent threshold effects via the changes in slope and variance. Presently, such relationship is represented by a model via the changes in either slope or variance. There is a need for developing a model which can estimate threshold effects via the changes in slope and variance. Also, many ecologists often know where the changes in slope and variance might occur in the stress-response relationship. The applicant proposes a Bayesian model which can estimate the changes in slope and variance, simultaneously, by adding the prior knowledge from ecologists. The proposed program will advance the research in statistics by better utilizing the information in big data. The applications in drug discovery and genetics will improve human and animal health with a potential to develop new drugs and identification of the causes of disease. In environmental ecology, the research will identify threshold effects of human induced disturbances to nature leading to sustainable management of aquatic habitat. The trained HQPs will be an asset in academia and industries in Canada and beyond.

随着技术的快速进步，科学和工程研究正在经历一场革命，大数据被用于各种目的。因此，该研究项目的长期目标是通过利用大数据中的有用信息来推进统计学、机器学习和统计生态学的科学发展。短期目标正在制定：基于不同变量子集的模型集合，以提高对感兴趣的响应变量的预测性能，应用于药物发现和遗传学中的大数据，贝叶斯统计测试和大数据稀疏性的推断，应用于遗传学，以及生态模型，通过针对人类对自然造成的干扰的斜率和方差的变化来检测环境阈值。与丢弃信息不同，申请人提出了利用更有用的变量而不是丢弃一些变量的集成方法。假设是过滤变量就像丢失信息一样，会降低模型的预测能力。为了解决这个问题，申请人提出了将有用变量分组为不同子集并通过将模型拟合到每个子集并聚合跨子集的模型来将它们聚合在集合中的方法。该方法的新颖之处在于自适应地获得变量子集，而不是当前使用已知或随机子集的趋势。许多统计方法在模型构建中强加稀疏性，即大数据中信息匮乏的情况，缺乏足够的理由和依据。相反，申请人提出在模型构建之前使用稀疏性的贝叶斯统计测试。该方法的新颖之处在于，模型构建中引入的稀疏量得到了数据测量证据的支持。生态响应和环境干扰之间的关系通常通过斜率和方差的变化来表示阈值效应。目前，这种关系由模型通过斜率或方差的变化来表示。需要开发一种可以通过斜率和方差的变化来估计阈值效应的模型。此外，许多生态学家通常知道压力-反应关系中斜率和方差的变化可能发生在哪里。申请人提出了一种贝叶斯模型，该模型可以通过添加生态学家的先验知识来同时估计斜率和方差的变化。拟议的计划将通过更好地利用大数据中的信息来推进统计研究。在药物发现和遗传学方面的应用将改善人类和动物的健康，并有可能开发新药和识别疾病原因。在环境生态学中，该研究将确定人类对自然造成的干扰的阈值效应，从而实现水生栖息地的可持续管理。训练有素的总部人员将成为加拿大及其他地区学术界和工业界的资产。