CAREER: A deeper look at state-dependent noise in systems

传统的控制模型通常假定任何环境噪声或干扰都与系统本身的状态无关。虽然这一假设大大简化了模型，但它并不总是正确的。在标准的线性控制视角下，一些具有状态相关观测噪声(通常表现为乘性噪声)的系统将被认为是不可控的。这一职业项目的重点是为这类系统发现新的非线性控制策略，并为其性能提供可证明的保证。这项研究有可能通过交互地从乘性破坏的测量中提取更多有用的信息来减少设计的保守主义。分析并为改进的模型提供保证的能力可以更好地表征安全关键系统的风险，在这些系统中，状态依赖的噪声项不能被忽视。这一职业项目的教育计划包括三项有组织的干预措施，以提高高等教育的包容性。这些措施包括：(1)通过PI新颖的低师工程课程招募不同的新学生，该课程通过实际操作的硬件实验室展示线性代数的现实应用。这一职业项目将为两年制社区大学传播这一新课程铺平道路。(2)为学生建立明确的支持结构，并通过面向包容性学生学习小组的自动化、可扩展和基于反馈的过程建立社区。(3)将可获得和结构化的大规模研究经验整合到常规课程中，这可以降低所有本科生的进入门槛，拓宽研究途径。系统中的状态依赖噪声可以通过模型线性化、参数不确定性或漂移和时间抖动来产生。零均值乘性观测噪声带来的一个关键技术挑战是它会破坏符号信息，因此线性策略可能是无限次优的。然而，利用控制的双重性质的非线性策略可以做得更好。这个职业项目通过三个步骤来研究这样的策略。(1)由于PI的前人工作表明，同时包含勘探和开发元素的控制策略可以很好地执行，本项目将首先计算特定周期非线性控制策略对状态的概率分布。这些分布的形状将指导使用计算方法(例如最大似然、政策梯度)寻找新的控制策略。(2)在可能难以确定最优战略的情况下，将使用基于后悔的提法来提供保证。信息论的比特级模型将被用来扩展到慢变化的乘性噪声和非零均值的情况。(3)最后，该项目将探索在指导政策搜索强化学习框架中使用乘性噪音增强本地模型的方法。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Traditional models in control typically assume that any environmental noise or disturbance is independent of the state of the system itself. While this assumption greatly simplifies the models, it is not always true. Under a standard linear control perspective, some systems with state-dependent observation noise, (which often manifests as multiplicative noise), would be considered uncontrollable. This CAREER project focuses on discovering new non-linear control strategies for such systems and providing provable guarantees for their performance. This research has the potential to reduce conservativism of designs by interactively extracting more useful information from multiplicatively-corrupted measurements. The ability to analyze and give guarantees for refined models can better characterize the risk for safety-critical systems where state-dependent noise terms cannot be ignored. The education plan for this CAREER project includes three structured interventions to improve inclusion in higher education. These include: (1) Recruit diverse new students through the PI’s novel lower-division engineering course that showcases real-world applications of linear-algebra through hands-on hardware labs. This CAREER project will build pathways to disseminate this new curriculum to two-year community colleges. (2) Develop explicit support structures for students and build community through an automated, scalable and feedback-based process for inclusive student study groups. (3) Integrate accessible and structured research-at-scale experiences into regular coursework, which can reduce the barrier-to-entry and broaden research access for all undergraduate students.State-dependent noise in systems can emerge through model linearization, parameter uncertainty or drift, and timing jitter. A key technical challenge posed by zero-mean multiplicative observation noise is that it can destroy sign information, and thus linear strategies can be unboundedly suboptimal. However, non-linear strategies that leverage the dual nature of control can do better. This CAREER project investigates such strategies through three steps. (1) Since the PI’s prior work suggests that control strategies with both exploration and exploitation elements can perform well, this project will first compute the probability distributions induced on the state by specific periodic non-linear control strategies. The shapes of these distributions will guide the search for new control strategies using computational approaches (e.g. maximum likelihood, policy gradient). (2) Regret-based formulations will be used to provide guarantees where optimality of strategies may be difficult to establish. Information-theoretic bit-level models will be used to extend to the case of slow-varying multiplicative noise and non-zero mean. (3) Finally, the project will explore approaches to augment local models with multiplicative noise in guided policy search for reinforcement learning frameworks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.