EAGER: Building a Provable Differentially Private Real-time Data-blind ML Algorithm: A case study on Enhancing STEM Student Engagement in Online Learning

COVID-19大流行可能已经结束，但课程交付形式的转变-远程或混合-仍在使用，大学欣赏其吸引更多学生群体的潜力，而不是纯粹的校园课程。像Zoom这样的平台上的这种灵活的教育形式将继续存在。为了提供更好的学习体验，教育工作者需要衡量学生对课程的参与度。但是，在讲课时，在线评估参与度是一项挑战。机器学习技术可以在讲座期间帮助教育工作者，以便可以估计课堂参与动态，并可以真实的时间采取适当的干预措施。然而，数据驱动的机器学习（ML）技术使其用户面临隐私丢失的风险，即使是在个别学生的个人工作站中托管的分布式机器学习程序，这些程序学习用户的模式并将模式报告给全局学习者，将结果合并到全局ML模型中。虽然没有私有数据离开本地工作站，但分布在网络上的各个模式可能会泄漏私有数据。该项目将建立创新的隐私意识学生参与检测技术。该项目的主要新奇之处在于，它能够从各种类型的学生参与数据中实时学习，而无需直接访问这些数据。在平台化的在线教育中，该项目将为用户增加隐私保障，而代表性不足的STEM学生可以安全地与教育工作者和同龄人互动，以促进学习的探究模型社区。该项目旨在设计一个分布式机器学习范式，介绍了学习节点的三个层次类别，这将通过一种新的神经网络架构不可知的梯度共享算法来促进，该算法将使任何协调的尝试从节点之间共享的部分梯度重建原始数据可证明是棘手的。 该框架的分层组织使其能够有效地在来自部分可观察模型架构的部分梯度中提供一定程度的混淆。研究方法的动机是梯度共享算法中的差分隐私概念。该项目将引入有关如何选择梯度分量来分布和优化可学习参数的新概念，与最先进的基于梯度的防御算法相比，在构建全局模型时不会产生任何额外的计算开销。该项目将由两个研究方向驱动：（1）设计一个可证明的隐私感知分布式机器学习框架，（2）利用新的框架来评估科罗拉多丹佛大学平台化在线STEM教育的学生参与度。研究工作将从黑盒的角度解决分布式机器学习中的一个开放问题，其中完整的梯度和模型架构都是未知的。因此，它有可能被其他需要隐私感知ML的领域采用。该项目的成果将为1）本科STEM学生提供直接的好处，同时提高学生的保留率和整体学习体验，2）在线STEM教师将能够以公平的方式实时衡量学生的参与度，隐私-该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准。

The COVID-19 pandemic may be over, but transitions in course delivery format—going remote, or hybrid—are still being used and universities appreciate their potential to attract more diverse groups of students than purely on-campus classes. This flexible education format in platforms like Zoom is here to stay. To deliver better learning experiences, educators need to gauge students' engagement in courses. But, while lecturing, it is challenging to assess engagement online. Machine learning technology can help educators during lectures so that the classroom engagement dynamics can be estimated, and proper interventions can be taken in real time. However, data-driven machine learning (ML) technology puts its users at risk of privacy loss, even with distributed machine learning programs hosted in individual students’ personal workstations that learn patterns of their users and report the patterns back to a global learner that merges the resulting findings into a global ML model. Although no private data is leaving local workstations, the individual patterns distributed across the network can leak private data. This project will build innovative privacy-aware student-engagement detection technology. The main novelty of this project will be in its capacity to learn in real-time from various types of student engagement data without directly accessing it. In platformized online education, the project will add privacy guarantee to users, while underrepresented STEM students can safely interact with educators and peers to facilitate the community of inquiry model of learning.The project aims to design a distributed machine learning paradigm that introduces three hierarchical categories of learner nodes that will be facilitated by a novel neural network architecture agnostic gradient sharing algorithm that will make any coordinated attempt to reconstruct original data from the partial gradients shared between nodes provably intractable. The hierarchical organization of the framework makes it effective at providing a level of obfuscation in partial gradients coming from partially observable model architecture. The research methodology will be motivated by concepts of differential privacy in gradient sharing algorithms. The project will introduce new concepts regarding how to select the gradient components to distribute and to optimize learnable parameters without incurring any additional computational overhead in building a global model, compared to the state-of-the-art gradient-based defense algorithms. The project will be driven by two research thrusts: (1) design of a provable privacy-aware distributed machine learning framework, (2) leveraging the novel framework in estimating student engagement in platformized online STEM education at University of Colorado Denver. The research effort will solve an open problem in the distributed machine learning from a black-box perspective where both full gradients and model architecture are unknown. Therefore, it has potential to be adopted in other areas where privacy aware ML is a requirement. The project outcomes will provide immediate benefits to 1) undergraduate STEM students while improving student retention and overall learning experiences, 2) online STEM instructors who will be able to gauge student engagement in real-time with an equitable, privacy-aware and inclusive learning environment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.