EAGER: Building a Provable Differentially Private Real-time Data-blind ML Algorithm: A case study on Enhancing STEM Student Engagement in Online Learning

新冠肺炎的流行或许已经结束，但课程交付模式的转变--远程授课或混合授课--仍在使用，大学意识到它们有潜力吸引比纯粹的校园课堂更多样化的学生群体。在Zoom这样的平台上，这种灵活的教育模式将继续存在。为了提供更好的学习体验，教育工作者需要衡量学生对课程的参与度。但是，在讲课的同时，在网上评估参与度是一件具有挑战性的事情。机器学习技术可以在授课期间帮助教育工作者，从而可以估计课堂参与的动态，并可以实时采取适当的干预措施。然而，数据驱动的机器学习(ML)技术将其用户置于隐私损失的风险中，即使在单个学生的个人工作站上托管的分布式机器学习程序也是如此，这些程序学习他们用户的模式并将模式报告给全球学习者，后者将结果合并到全球ML模型中。虽然没有私有数据离开本地工作站，但分布在网络上的单个模式可能会泄露私有数据。该项目将构建具有隐私意识的创新学生参与度检测技术。这个项目的主要新奇之处在于，它能够从各种类型的学生参与数据中实时学习，而不是直接访问它。在平台化的在线教育中，该项目将为用户增加隐私保障，而未被代表的STEM学生可以安全地与教育者和同行进行交互，以促进学习的探究模式社区。该项目旨在设计一个分布式机器学习范式，引入三个层次类别的学习节点，并将通过一种新颖的神经网络体系结构不可知的梯度共享算法来促进，该算法将使从节点之间共享的部分梯度重建原始数据的任何协调尝试变得困难。该框架的层次化组织使其能够有效地在来自部分可观察的模型体系结构的部分梯度中提供模糊级别。研究方法将受到梯度共享算法中的差异隐私概念的启发。与最先进的基于梯度的防御算法相比，该项目将引入关于如何选择要分布的梯度分量并优化可学习参数的新概念，而不会在建立全局模型时招致任何额外的计算开销。该项目将由两个研究项目推动：(1)设计一个可证明的隐私感知分布式机器学习框架，(2)利用该新框架来评估学生对科罗拉多大学丹佛大学平台化在线STEM教育的参与度。这项研究工作将从黑盒的角度解决分布式机器学习中的一个开放问题，其中完全梯度和模型体系结构都是未知的。因此，它有可能被采用在其他领域，隐私意识ML是一个要求。项目成果将直接惠及1)本科生STEM学生，同时改善学生保留和整体学习体验，2)在线STEM讲师，他们将能够在公平、隐私意识和包容性的学习环境中实时评估学生的参与度。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The COVID-19 pandemic may be over, but transitions in course delivery format—going remote, or hybrid—are still being used and universities appreciate their potential to attract more diverse groups of students than purely on-campus classes. This flexible education format in platforms like Zoom is here to stay. To deliver better learning experiences, educators need to gauge students' engagement in courses. But, while lecturing, it is challenging to assess engagement online. Machine learning technology can help educators during lectures so that the classroom engagement dynamics can be estimated, and proper interventions can be taken in real time. However, data-driven machine learning (ML) technology puts its users at risk of privacy loss, even with distributed machine learning programs hosted in individual students’ personal workstations that learn patterns of their users and report the patterns back to a global learner that merges the resulting findings into a global ML model. Although no private data is leaving local workstations, the individual patterns distributed across the network can leak private data. This project will build innovative privacy-aware student-engagement detection technology. The main novelty of this project will be in its capacity to learn in real-time from various types of student engagement data without directly accessing it. In platformized online education, the project will add privacy guarantee to users, while underrepresented STEM students can safely interact with educators and peers to facilitate the community of inquiry model of learning.The project aims to design a distributed machine learning paradigm that introduces three hierarchical categories of learner nodes that will be facilitated by a novel neural network architecture agnostic gradient sharing algorithm that will make any coordinated attempt to reconstruct original data from the partial gradients shared between nodes provably intractable. The hierarchical organization of the framework makes it effective at providing a level of obfuscation in partial gradients coming from partially observable model architecture. The research methodology will be motivated by concepts of differential privacy in gradient sharing algorithms. The project will introduce new concepts regarding how to select the gradient components to distribute and to optimize learnable parameters without incurring any additional computational overhead in building a global model, compared to the state-of-the-art gradient-based defense algorithms. The project will be driven by two research thrusts: (1) design of a provable privacy-aware distributed machine learning framework, (2) leveraging the novel framework in estimating student engagement in platformized online STEM education at University of Colorado Denver. The research effort will solve an open problem in the distributed machine learning from a black-box perspective where both full gradients and model architecture are unknown. Therefore, it has potential to be adopted in other areas where privacy aware ML is a requirement. The project outcomes will provide immediate benefits to 1) undergraduate STEM students while improving student retention and overall learning experiences, 2) online STEM instructors who will be able to gauge student engagement in real-time with an equitable, privacy-aware and inclusive learning environment.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.