Mitigating Risks Associated with Big Data Solutions

降低与大数据解决方案相关的风险

• 批准号: RGPIN-2015-06075
• 负责人: Miranskyy, Andriy
• 金额: $ 1.31万
• 依托单位: Ryerson University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=708190
• 关键词: Mitigating; Risks; Associated; Big; Data

Today's ubiquitous communication and computing technologies generate magnitudes of data far beyond that available even a decade ago. The Big Data technology and services market is expected to grow at a compound annual rate of 27% from $9.8 billion in 2012 to $32.4 billion in 2017, leading to multi-fold increase in the amount of technical data generated. Modern organisations have unprecedented opportunities to gain new knowledge about their products, processes and services, which can be used to make projects more efficient, improve products and services, identify problems, take pre-emptive measures, and adapt business goals to new opportunities. Applications of new knowledge include predictive models which reduce, for example, hospital readmission rates, traffic congestion, and unnecessary power generation. Exploiting these opportunities requires addressing challenges posed by solutions for processing Big Data (BDS). These complex solutions have many dynamic components, such as distributed compute nodes, networks, databases, middleware, and business intelligence layers. Any component can fail its interactions with others, possibly leading to crashing failure of the solution or quality degradation (e.g., performance, reliability, security). Therefore, they are challenging to develop and maintain. Big Data environments lack methods, tools, processes and techniques to support the disciplined development and maintenance of BDS. Despite a significant body of knowledge on processing data generated by BDS, most techniques cannot process the volumes of operational data generated by BDS. My goal is to help improve testing and maintenance of the BDS by reaching the following objectives: 1) Build defect prediction models for the BDS to improve its General Testing and Maintenance process; 2) Develop trace analysis for the BDS to speed up root cause determination and the removal of redundant test cases. To reach the objectives, I will create novel scalable methods, techniques, and tools capable of processing the operational data. To the best of my knowledge, these objectives are novel and significant for practice. The anticipated results will help create a preliminary theory of automated problem determination techniques for BDS. Transfer of the results to Canadian industry will improve product quality and speed up defect detection and fixing, allowing developers to create more new functionality, and reduce maintenance effort and investment.

今天无处不在的通信和计算技术产生的数据量远远超过十年前。大数据技术和服务市场预计将以27%的复合年增长率增长，从2012年的98亿美元增长到2017年的324亿美元，导致产生的技术数据量成倍增长。现代组织有前所未有的机会获得有关其产品，流程和服务的新知识，这些知识可用于提高项目效率，改进产品和服务，发现问题，采取先发制人的措施，并使业务目标适应新的机会。新知识的应用包括预测模型，例如，减少医院再入院率，交通拥堵和不必要的发电。 利用这些机会需要应对处理大数据（BDS）的解决方案所带来的挑战。这些复杂的解决方案具有许多动态组件，例如分布式计算节点、网络、数据库、中间件和商业智能层。任何组件都可能无法与其他组件进行交互，可能导致解决方案崩溃或质量下降（例如，性能、可靠性、安全性）。因此，它们的开发和维护具有挑战性。 大数据环境缺乏方法，工具，流程和技术来支持BDS的规范开发和维护。尽管在处理工商发展服务产生的数据方面有大量的知识，但大多数技术都无法处理工商发展服务产生的大量业务数据。我的目标是通过达到以下目标来帮助改进BDS的测试和维护：1）为BDS构建缺陷预测模型，以改进其一般测试和维护过程; 2）为BDS开发跟踪分析，以加快根本原因确定和删除冗余测试用例。为了实现目标，我将创建能够处理操作数据的新颖的可扩展方法，技术和工具。 据我所知，这些目标是新颖的，对实践具有重要意义。预期的结果将有助于创建一个初步的理论自动化的问题确定技术的BDS。将成果转移到加拿大工业将提高产品质量，加快缺陷检测和修复，使开发人员能够创建更多的新功能，并减少维护工作和投资。