CAREER: A deeper look at state-dependent noise in systems

传统的控制模型通常假设任何环境噪声或干扰与系统本身的状态无关。虽然这个假设大大简化了模型，但它并不总是正确的。在标准的线性控制角度下，一些具有状态依赖的观测噪声（通常表现为乘法噪声）的系统将被认为是不可控的。这个CAREER项目的重点是为这些系统发现新的非线性控制策略，并为它们的性能提供可证明的保证。这项研究有可能通过交互式地从乘性损坏的测量中提取更多有用的信息来减少设计的保守性。对精细模型的分析和保证能力可以更好地表征安全关键系统的风险，其中状态相关噪声项不能被忽视。CAREER项目的教育计划包括三个结构化的干预措施，以提高高等教育的包容性。这些措施包括：(1)通过PI新颖的低级别工程课程招收不同的新生，该课程通过动手硬件实验室展示线性代数的实际应用。这个职业项目将建立途径，将这个新课程传播到两年制社区大学。(2)为学生建立明确的支持结构，并通过自动化、可扩展和基于反馈的流程为包容性学生学习小组建立社区。(3)将无障碍和结构化的大规模研究经验整合到常规课程中，这可以降低进入门槛，拓宽所有本科生的研究机会。系统中的状态相关噪声可以通过模型线性化、参数不确定性或漂移以及时序抖动产生。零均值乘性观测噪声带来的一个关键技术挑战是它会破坏符号信息，因此线性策略可能是无界次优的。然而，利用控制的双重性质的非线性策略可以做得更好。这个CAREER项目通过三个步骤来研究这些策略。(1)由于PI先前的工作表明，同时包含勘探和开采元素的控制策略都可以很好地执行，因此本项目将首先计算特定周期非线性控制策略对状态的概率分布。这些分布的形状将指导使用计算方法（例如最大似然、策略梯度）寻找新的控制策略。(2)在难以确定策略最优性的情况下，将使用基于遗憾的公式提供保证。信息论的位级模型将被用于扩展到缓慢变化的乘性噪声和非零均值的情况。(3)最后，该项目将探索在强化学习框架的指导策略搜索中增强带有乘法噪声的局部模型的方法。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Traditional models in control typically assume that any environmental noise or disturbance is independent of the state of the system itself. While this assumption greatly simplifies the models, it is not always true. Under a standard linear control perspective, some systems with state-dependent observation noise, (which often manifests as multiplicative noise), would be considered uncontrollable. This CAREER project focuses on discovering new non-linear control strategies for such systems and providing provable guarantees for their performance. This research has the potential to reduce conservativism of designs by interactively extracting more useful information from multiplicatively-corrupted measurements. The ability to analyze and give guarantees for refined models can better characterize the risk for safety-critical systems where state-dependent noise terms cannot be ignored. The education plan for this CAREER project includes three structured interventions to improve inclusion in higher education. These include: (1) Recruit diverse new students through the PI’s novel lower-division engineering course that showcases real-world applications of linear-algebra through hands-on hardware labs. This CAREER project will build pathways to disseminate this new curriculum to two-year community colleges. (2) Develop explicit support structures for students and build community through an automated, scalable and feedback-based process for inclusive student study groups. (3) Integrate accessible and structured research-at-scale experiences into regular coursework, which can reduce the barrier-to-entry and broaden research access for all undergraduate students.State-dependent noise in systems can emerge through model linearization, parameter uncertainty or drift, and timing jitter. A key technical challenge posed by zero-mean multiplicative observation noise is that it can destroy sign information, and thus linear strategies can be unboundedly suboptimal. However, non-linear strategies that leverage the dual nature of control can do better. This CAREER project investigates such strategies through three steps. (1) Since the PI’s prior work suggests that control strategies with both exploration and exploitation elements can perform well, this project will first compute the probability distributions induced on the state by specific periodic non-linear control strategies. The shapes of these distributions will guide the search for new control strategies using computational approaches (e.g. maximum likelihood, policy gradient). (2) Regret-based formulations will be used to provide guarantees where optimality of strategies may be difficult to establish. Information-theoretic bit-level models will be used to extend to the case of slow-varying multiplicative noise and non-zero mean. (3) Finally, the project will explore approaches to augment local models with multiplicative noise in guided policy search for reinforcement learning frameworks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.