Ensuring the Trustworthiness of Data Science Systems in a Cost Effective Manner

以具有成本效益的方式确保数据科学系统的可信度

• 批准号: 533280-2018
• 负责人: Hassan, Ahmed
• 金额: $ 10.09万
• 依托单位: Queen's University
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=698456
• 关键词: Ensuring; Trustworthiness; Data; Science; Systems

The pipelines (i.e., processes and techniques) that enable the extraction of knowledge and insights from structured or unstructured data are commonly referred to as Data Science (DS). Large scale software systems that integrate DS pipelines are commonly referred to as DS systems. Integrating DS pipelines into production systems brings challenges that go well beyond the classical Machine Learning programs (e.g., quality of the fit of an ML model). Two notable challenges for DS systems today are: 1) Scalability challenges: The large amount of data that DS systems must process and their rapid release cycles require a rethinking of how testing processes are being conducted at such scale. For instance how can we perform an extended month-long test when new releases are being deployed on a weekly basis! 2) Explainability challenges: Carefully and confidently understanding why a particular event occurred or determining whether a DS system is performing correctly remains a major undertaking. Moreover the rationale for the decisions of a DS system must remain stable across versions unless there are good reasons to warrant otherwise. The proposed collaborative research program will develop approaches to enable "Trustworthy Computing" within the context of DS systems in a cost effective and scalable manner. Both the industrial (BlackBerry) and academic (SAIL) partners are world-recognized leaders in "Trustworthy Computing" with over a decade working on optimizing and testing the performance and reliability of highly secure systems. The research builds on their extensive experience and will train nine HQPs in software performance engineering, security engineering, and empirical software engineering, as well as essential communication and presentation skills.

可以从结构化或非结构化数据中提取知识和见解的管道（即过程和技术）通常称为数据科学（DS）。集成DS管道的大型软件系统通常称为DS系统。将DS管道集成到生产系统中带来的挑战远远超出了经典的机器学习计划（例如，ML模型的拟合质量）。当今DS系统面临的两个值得注意的挑战是：1）可伸缩性挑战：DS系统必须处理的大量数据及其快速释放周期需要重新思考如何以这种规模进行测试过程。例如，当每周部署新版本时，我们如何进行长达一个月的测试！ 2）解释性挑战：仔细，自信地理解为什么发生特定事件或确定DS系统是否正确执行的情况仍然是一项重大任务。此外，除非有充分的理由另有原因有充分的理由，否则DS系统决策的理由必须保持稳定。 拟议的协作研究计划将以成本效益和可扩展的方式开发在DS系统中启用“值得信赖的计算”的方法。工业（黑莓）和学术（帆）合作伙伴都是世界认可的“值得信赖计算”的领导者，十多年来致力于优化和测试高度安全系统的性能和可靠性。该研究以其丰富的经验为基础，将在软件性能工程，安全工程和经验软件工程以及基本的沟通和演示技巧方面培训9个HQP。