SEAML: A Machine Learning Model Lineage Platform (Phase Ia)

现代软件应用程序严重依赖大数据和机器学习，作为其基本用例的核心资产或支持基础设施。不同国家颁布的隐私和安全法律限制了如何以负责任的方式使用数据。为了解决社会问题，负责任地使用数据成为机器学习密集型系统的关键要求。当数据在希望分担数据使用和机器学习模型生命周期责任的利益相关者之间共享时，管理数据变得更加具有挑战性。该项目提出了一个安全和可审计的数据和ML模型谱系平台(SEAML)。SEAML有一个通用的目的，但它可以使利益相关者生态系统受益最大，该生态系统基于电子合同定义数据和模型谱系和使用。SEAML将区块链用于数据、模型和运营的元数据存储，并作为一种机制来定义数据和模型运营的不可否认合同。该平台具有数据工作流定义和执行引擎，用户可以在其中指定数据源、对数据的操作顺序、对数据和操作的访问策略。元数据被定义为包含指向数据、模型和操作的超链接的记录，它使用分布式标识符以及散列码等其他数据特征来唯一标识数据。通过私钥/公钥控制对真实数据和模型的访问。

Modern software applications rely heavily on big data and machine learning, either as a core asset of their basic use cases or as supporting infrastructure. Privacy and security laws mandated by different countries constrain how the data can be used in an accountable manner. To address social concerns, responsible use of data becomes a key requirement for Machine Learning intensive systems. Governing data becomes even more challenging when it is shared among stakeholders who want a shared responsibility on data usage and machine learning models life cycle. This project proposes a secure and auditable data and ML models lineage platform (SEAML). SEAML has a general purpose, but it can benefit the most an ecosystem of stakeholders that define data and model lineage and usage based on electronic contracts. SEAML uses blockchain for data, models and operations' metadata storage, and as a mechanism to define non-repudiable contracts on data and model operations. The platform has a data workflow definition and execution engine where the user specifies the data sources, the sequence of operations on data, the access policies on data and operations. Metadata, defined as records containing hyper links to data, models and operations, uses distributed identifiers, DIDin addition to other data characteristics such as hash codes to uniquely identify data. Access to real data and to models is controlled through private/public keys.