Ensuring the Trustworthiness of Data Science Systems in a Cost Effective Manner

管道（即，能够从结构化或非结构化数据中提取知识和洞察力的过程和技术通常被称为数据科学（DS）。集成DS流水线的大规模软件系统通常被称为DS系统。将DS管道集成到生产系统中带来的挑战远远超出了经典的机器学习程序（例如，ML模型的拟合质量）。目前DS系统面临的两个显著挑战是：1）可扩展性挑战：DS系统必须处理的大量数据及其快速发布周期需要重新思考如何在这种规模下进行测试过程。例如，当每周都在部署新版本的时候，我们怎么能执行一个月的测试呢？2)可解释性挑战：仔细而自信地理解特定事件发生的原因或确定DS系统是否正确运行仍然是一项重大任务。此外，一个DS系统的决策的理由必须保持稳定的版本，除非有很好的理由，以保证otherwise.拟议的合作研究计划将开发的方法，使“可信计算”的DS系统的范围内，在一个成本效益和可扩展的方式。工业（BlackBerry）和学术（SAIL）合作伙伴都是世界公认的“可信计算”领导者，十多年来一直致力于优化和测试高度安全系统的性能和可靠性。这项研究建立在他们丰富的经验，并将培训九个HQP在软件性能工程，安全工程，经验软件工程，以及基本的沟通和演示技巧。

The pipelines (i.e., processes and techniques) that enable the extraction of knowledge and insights from structured or unstructured data are commonly referred to as Data Science (DS). Large scale software systems that integrate DS pipelines are commonly referred to as DS systems. Integrating DS pipelines into production systems brings challenges that go well beyond the classical Machine Learning programs (e.g., quality of the fit of an ML model). Two notable challenges for DS systems today are: 1) Scalability challenges: The large amount of data that DS systems must process and their rapid release cycles require a rethinking of how testing processes are being conducted at such scale. For instance how can we perform an extended month-long test when new releases are being deployed on a weekly basis! 2) Explainability challenges: Carefully and confidently understanding why a particular event occurred or determining whether a DS system is performing correctly remains a major undertaking. Moreover the rationale for the decisions of a DS system must remain stable across versions unless there are good reasons to warrant otherwise.The proposed collaborative research program will develop approaches to enable "Trustworthy Computing" within the context of DS systems in a cost effective and scalable manner. Both the industrial (BlackBerry) and academic (SAIL) partners are world-recognized leaders in "Trustworthy Computing" with over a decade working on optimizing and testing the performance and reliability of highly secure systems. The research builds on their extensive experience and will train nine HQPs in software performance engineering, security engineering, and empirical software engineering, as well as essential communication and presentation skills.