Ensuring the Trustworthiness of Data Science Systems in a Cost Effective Manner

点击翻译按钮获取中文摘要

The pipelines (i.e., processes and techniques) that enable the extraction of knowledge and insights from structured or unstructured data are commonly referred to as Data Science (DS). Large scale software systems that integrate DS pipelines are commonly referred to as DS systems. Integrating DS pipelines into production systems brings challenges that go well beyond the classical Machine Learning programs (e.g., quality of the fit of an ML model). Two notable challenges for DS systems today are: 1) Scalability challenges: The large amount of data that DS systems must process and their rapid release cycles require a rethinking of how testing processes are being conducted at such scale. For instance how can we perform an extended month-long test when new releases are being deployed on a weekly basis! 2) Explainability challenges: Carefully and confidently understanding why a particular event occurred or determining whether a DS system is performing correctly remains a major undertaking. Moreover the rationale for the decisions of a DS system must remain stable across versions unless there are good reasons to warrant otherwise.The proposed collaborative research program will develop approaches to enable "Trustworthy Computing" within the context of DS systems in a cost effective and scalable manner. Both the industrial (BlackBerry) and academic (SAIL) partners are world-recognized leaders in "Trustworthy Computing" with over a decade working on optimizing and testing the performance and reliability of highly secure systems. The research builds on their extensive experience and will train nine HQPs in software performance engineering, security engineering, and empirical software engineering, as well as essential communication and presentation skills.