Dynamic Discrete Choice Estimation with Partially Observable States and Hidden Dynamics

具有部分可观察状态和隐藏动态的动态离散选择估计

• 批准号: 2048395
• 负责人: Yanling Chang
• 金额: $ 34.98万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2048395&HistoricalAwards=false
• 关键词: Dynamic; Discrete; Choice; Estimation; Partially

This award will investigate rigorous methods for data-driven model identification and estimation for stochastic dynamic decision processes. Many dynamical systems are modeled via discrete states that evolve via stochastic transitions and control inputs. Rather than assume a model structure in advance, this project develops methods to "learn" the model structure, including preferences of the controller and model states, from available data. The project will develop these new methods in the context of technology-assisted human performance in vehicle operations. Humans drivers operating vehicles on a busy highway often engage in secondary tasks that demand cognitive effort (e.g. attending to a business-related phone call, listening to a podcast). The driver is aware of their engagement in the secondary task but may be only imperfectly aware of its effect on their own ability to drive safely. On-board autonomous technology with the ability to accurately estimate the driver’s safety level may improve performance through auditory warnings or partial automation (e.g., adaptive cruise control). However, existing estimation methodologies are built on the assumption that the driver has perfect state observability, whereas the driving task’s overall level of safety is only imperfectly observed. This project provides a novel estimation method to build a predictive model of the agent’s driving behavior by considering imperfectly observed driving states (e.g. safe and unsafe driving). The research benefits the society by enabling on-board automation system to monitor the driver's activities and potentially intervene to improve driving performance and safety.This award will develop new methodologies and algorithms for learning a model of dynamic decisions with hidden states. The research draws on and generalizes dynamic discrete choice models to consider memoryless partially observable states. This project will also examine estimation of (non-exponential) semi-Markov hidden models with state-dependent sojourn time distributions. This research will rigorously examine whether and/or under what conditions the models are identifiable and ascertain the implications for robustness of estimation results. The new methods will leverage experimental collection of observable data known to correlate with distraction (e.g., breathing rate, heart rate variability, eye-tracking) together with observed actions (e.g., maneuvers) to estimate a compact model of the agent's control policies and the dynamics of partially observable states (safe and unsafe driving). The model and estimation approach will be validated through a high-fidelity driving simulation study.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项将研究随机动态决策过程的数据驱动模型识别和估计的严格方法。 许多动态系统是通过离散状态建模的，这些离散状态通过随机转换和控制输入来演化。该项目不是预先假设模型结构，而是开发方法来“学习”模型结构，包括控制器的偏好和模型状态，从可用数据中。 该项目将在车辆操作中技术辅助人类表现的背景下开发这些新方法。 在忙碌的高速公路上驾驶车辆的人类驾驶员经常从事需要认知努力的次要任务（例如，接听与业务相关的电话，收听播客）。驾驶员意识到他们参与了次要任务，但可能只是不完全意识到它对他们自己安全驾驶能力的影响。能够准确估计驾驶员安全水平的车载自主技术可以通过听觉警告或部分自动化（例如，自适应巡航控制）。然而，现有的估计方法是建立在假设驾驶员具有完美的状态可观测性，而驾驶任务的整体安全水平只是不完美的观察。该项目提供了一种新的估计方法，通过考虑不完全观察到的驾驶状态（例如安全和不安全驾驶）来建立智能体驾驶行为的预测模型。该研究通过使车载自动化系统能够监控驾驶员的活动并可能进行干预以改善驾驶性能和安全性而造福社会。该奖项将开发新的方法和算法，用于学习具有隐藏状态的动态决策模型。该研究借鉴和推广动态离散选择模型，考虑无记忆部分可观测状态。这个项目也将研究估计（非指数）半马尔可夫隐模型与状态依赖逗留时间分布。 本研究将严格审查是否和/或在什么条件下的模型是可识别的，并确定估计结果的鲁棒性的影响。新方法将利用已知与分心相关的可观察数据的实验收集（例如，呼吸率、心率变异性、眼睛跟踪）以及观察到的动作（例如，机动），以估计代理的控制策略和部分可观察状态（安全和不安全驾驶）的动态的紧凑模型。该模型和估算方法将通过高保真驾驶模拟研究进行验证。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Structural Estimation of Partially Observable Markov Decision Processes

• DOI: 10.1109/tac.2022.3217908
• 发表时间: 2020-08
• 期刊: IEEE Transactions on Automatic Control
• 影响因子: 6.8
• 作者: Yanling Chang;Alfredo Garcia;Zhide Wang;Lu Sun

World Model Learning from Demonstrations with Active Inference: Application to Driving Behavior

• DOI: 10.1007/978-3-031-28719-0_9
• 发表时间: 2022
• 作者: Ran Wei;Alfredo Garcia;Anthony D. McDonald;G. Markkula;J. Engström;Isaac Supeene;Matthew O'Kelly