Ensuring the Trustworthiness of Data Science Systems in a Cost Effective Manner

以具有成本效益的方式确保数据科学系统的可信度

• 批准号: 533280-2018
• 负责人: Hassan, Ahmed
• 金额: $ 10.09万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2021
• 资助国家: 加拿大
• 起止时间: 2021-01-01 至 2022-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=719917
• 关键词: Ensuring; Trustworthiness; Data; Science; Systems

The pipelines (i.e., processes and techniques) that enable the extraction of knowledge and insights from structured or unstructured data are commonly referred to as Data Science (DS). Large scale software systems that integrate DS pipelines are commonly referred to as DS systems. Integrating DS pipelines into production systems brings challenges that go well beyond the classical Machine Learning programs (e.g., quality of the fit of an ML model). Two notable challenges for DS systems today are: 1) Scalability challenges: The large amount of data that DS systems must process and their rapid release cycles require a rethinking of how testing processes are being conducted at such scale. For instance how can we perform an extended month-long test when new releases are being deployed on a weekly basis! 2) Explainability challenges: Carefully and confidently understanding why a particular event occurred or determining whether a DS system is performing correctly remains a major undertaking. Moreover the rationale for the decisions of a DS system must remain stable across versions unless there are good reasons to warrant otherwise.The proposed collaborative research program will develop approaches to enable "Trustworthy Computing" within the context of DS systems in a cost effective and scalable manner. Both the industrial (BlackBerry) and academic (SAIL) partners are world-recognized leaders in "Trustworthy Computing" with over a decade working on optimizing and testing the performance and reliability of highly secure systems. The research builds on their extensive experience and will train nine HQPs in software performance engineering, security engineering, and empirical software engineering, as well as essential communication and presentation skills.

能够从结构化或非结构化数据中提取知识和见解的管道（即流程和技术）通常被称为数据科学（DS）。集成了DS管道的大型软件系统通常被称为DS系统。将DS管道集成到生产系统中带来的挑战远远超出了传统的机器学习程序（例如，ML模型的拟合质量）。当前DS系统面临的两个显著挑战是：1)可伸缩性挑战：DS系统必须处理的大量数据及其快速的发布周期要求重新考虑如何在如此大规模的情况下执行测试过程。例如，当新版本每周都在部署时，我们如何执行延长的长达一个月的测试！2)可解释性挑战：仔细而自信地理解特定事件发生的原因或确定DS系统是否正确运行仍然是一项主要任务。此外，DS系统决策的基本原理必须在不同版本之间保持稳定，除非有充分的理由不这样做。拟议的合作研究计划将开发方法，以经济有效和可扩展的方式在DS系统的背景下实现“可信计算”。工业（黑莓）和学术（SAIL）合作伙伴都是世界公认的“可信计算”领域的领导者，在优化和测试高度安全系统的性能和可靠性方面有超过十年的工作经验。该研究以他们丰富的经验为基础，将培训9名hqp，包括软件性能工程、安全工程和经验软件工程，以及必要的沟通和演示技巧。