EAGER: Building a Provable Differentially Private Real-time Data-blind ML Algorithm: A case study on Enhancing STEM Student Engagement in Online Learning

2019冠状病毒病大流行可能已经结束，但课程交付形式的转变——远程授课或混合授课——仍在使用，大学也意识到它们吸引更多不同学生群体的潜力，而不是纯粹的校园课程。Zoom等平台上这种灵活的教育形式将继续存在。为了提供更好的学习体验，教育工作者需要衡量学生对课程的参与程度。但是，在授课的同时，评估在线参与度是一项挑战。机器学习技术可以在课堂上帮助教育工作者，这样就可以评估课堂参与动态，并实时采取适当的干预措施。然而，数据驱动的机器学习（ML）技术将其用户置于隐私丢失的风险中，即使在单个学生的个人工作站中托管分布式机器学习程序，也可以学习用户的模式并将模式报告给全局学习者，后者将结果结果合并到全局ML模型中。尽管没有私有数据离开本地工作站，但是分布在网络上的单个模式可能会泄漏私有数据。该项目将构建具有隐私意识的创新型学生参与检测技术。这个项目的主要新颖之处在于它能够在不直接访问的情况下从各种类型的学生参与数据中实时学习。在平台化的在线教育中，该项目将为用户增加隐私保障，而未被充分代表的STEM学生可以安全地与教育工作者和同龄人互动，促进社区探究学习模式。该项目旨在设计一种分布式机器学习范式，该范式引入了三种分层分类的学习器节点，这将通过一种新的神经网络架构不可知的梯度共享算法来促进，该算法将使任何从节点之间共享的部分梯度重建原始数据的协调尝试被证明是难以处理的。框架的分层组织使其能够有效地在来自部分可观察模型体系结构的部分梯度中提供一定程度的混淆。研究方法将受到梯度共享算法中差分隐私概念的激励。与最先进的基于梯度的防御算法相比，该项目将引入关于如何选择梯度组件来分布和优化可学习参数的新概念，而不会在构建全局模型时产生任何额外的计算开销。该项目将由两个研究重点推动：(1)设计一个可证明的隐私感知分布式机器学习框架，(2)利用该新框架评估科罗拉多大学丹佛分校学生在平台化在线STEM教育中的参与度。该研究将从黑盒的角度解决分布式机器学习中的一个开放问题，其中完整的梯度和模型架构都是未知的。因此，它有可能在其他需要隐私感知ML的领域被采用。项目成果将立即受益：1)本科STEM学生，同时提高学生的保留率和整体学习体验；2)在线STEM教师将能够在公平、注重隐私和包容的学习环境中实时评估学生的参与度。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The COVID-19 pandemic may be over, but transitions in course delivery format—going remote, or hybrid—are still being used and universities appreciate their potential to attract more diverse groups of students than purely on-campus classes. This flexible education format in platforms like Zoom is here to stay. To deliver better learning experiences, educators need to gauge students' engagement in courses. But, while lecturing, it is challenging to assess engagement online. Machine learning technology can help educators during lectures so that the classroom engagement dynamics can be estimated, and proper interventions can be taken in real time. However, data-driven machine learning (ML) technology puts its users at risk of privacy loss, even with distributed machine learning programs hosted in individual students’ personal workstations that learn patterns of their users and report the patterns back to a global learner that merges the resulting findings into a global ML model. Although no private data is leaving local workstations, the individual patterns distributed across the network can leak private data. This project will build innovative privacy-aware student-engagement detection technology. The main novelty of this project will be in its capacity to learn in real-time from various types of student engagement data without directly accessing it. In platformized online education, the project will add privacy guarantee to users, while underrepresented STEM students can safely interact with educators and peers to facilitate the community of inquiry model of learning.The project aims to design a distributed machine learning paradigm that introduces three hierarchical categories of learner nodes that will be facilitated by a novel neural network architecture agnostic gradient sharing algorithm that will make any coordinated attempt to reconstruct original data from the partial gradients shared between nodes provably intractable. The hierarchical organization of the framework makes it effective at providing a level of obfuscation in partial gradients coming from partially observable model architecture. The research methodology will be motivated by concepts of differential privacy in gradient sharing algorithms. The project will introduce new concepts regarding how to select the gradient components to distribute and to optimize learnable parameters without incurring any additional computational overhead in building a global model, compared to the state-of-the-art gradient-based defense algorithms. The project will be driven by two research thrusts: (1) design of a provable privacy-aware distributed machine learning framework, (2) leveraging the novel framework in estimating student engagement in platformized online STEM education at University of Colorado Denver. The research effort will solve an open problem in the distributed machine learning from a black-box perspective where both full gradients and model architecture are unknown. Therefore, it has potential to be adopted in other areas where privacy aware ML is a requirement. The project outcomes will provide immediate benefits to 1) undergraduate STEM students while improving student retention and overall learning experiences, 2) online STEM instructors who will be able to gauge student engagement in real-time with an equitable, privacy-aware and inclusive learning environment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.