TWC: Medium: Collaborative: Efficient Repair of Learning Systems via Machine Unlearning

如今，个人和组织利用机器学习系统来调节室温、提供建议、检测恶意软件、预测地震、预测天气、机动车辆，并将大数据转化为洞察力。不幸的是，这些系统容易受到各种恶意攻击，并可能造成灾难性的后果。例如，攻击者可能通过向机器学习模型的训练集中注入精心制作的样本（即“污染”模型）来欺骗入侵检测系统，使其忽略未来攻击的警告信号。该项目正在创建一种机器学习方法，以及必要的算法、技术和系统，以便在学习系统受到损害后有效地修复它。机器学习提供了针对学习系统的各种攻击的最后手段，并且是对其他现有防御的补充。机器学习的关键观点是，大多数学习系统都可以转换成一种可以增量更新的形式，而无需从头开始进行昂贵的再培训。例如，几种常见的学习技术（如朴素贝叶斯分类器）可以转换为非自适应统计查询学习形式，这种学习形式只依赖于常数个数的求和，每个求和都是训练数据样本的一些有效可计算变换的和。为了以这种形式修复受损的学习系统，操作员添加或删除受影响的训练样本，并通过更新常量求和来重新计算训练模型。这种方法产生了巨大的加速——重新训练的渐近加速等于训练集的大小。通过unlearning，操作员可以通过从训练集中移除注入的样本来有效地纠正被污染的学习系统，通过向训练集中添加逃避的样本来加强逃避的学习系统，并通过忘记攻击者窃取的样本来防止系统推理攻击，使未来的攻击无法对样本进行任何推断。

Today individuals and organizations leverage machine learning systems to adjust room temperature, provide recommendations, detect malware, predict earthquakes, forecast weather, maneuver vehicles, and turn Big Data into insights. Unfortunately, these systems are prone to a variety of malicious attacks with potentially disastrous consequences. For example, an attacker might trick an Intrusion Detection System into ignoring the warning signs of a future attack by injecting carefully crafted samples into the training set for the machine learning model (i.e., "polluting" the model). This project is creating an approach to machine unlearning and the necessary algorithms, techniques, and systems to efficiently and effectively repair a learning system after it has been compromised. Machine unlearning provides a last resort against various attacks on learning systems, and is complementary to other existing defenses. The key insight in machine unlearning is that most learning systems can be converted into a form that can be updated incrementally without costly retraining from scratch. For instance, several common learning techniques (e.g., naive Bayesian classifier) can be converted to the non-adaptive statistical query learning form, which depends only on a constant number of summations, each of which is a sum of some efficiently computable transformation of the training data samples. To repair a compromised learning system in this form, operators add or remove the affected training sample and re-compute the trained model by updating a constant number of summations. This approach yields huge speedup -- the asymptotic speedup over retraining is equal to the size of the training set. With unlearning, operators can efficiently correct a polluted learning system by removing the injected sample from the training set, strengthen an evaded learning system by adding evasive samples to the training set, and prevent system inference attacks by forgetting samples stolen by the attacker so that no future attacks can infer anything about the samples.