Collaborative Research: ITR: A Robotics-Based Computational Environment to Simulate the Human Hand

该项目通过1)基于模型的非线性状态和参数估计、2)智能自适应控制和3)图像处理的基础研究，开发了一个完整的基于视觉的无人机控制的贝叶斯框架。我们具体介绍了如何使用地面传感器(和机载航空电子设备)处理实时视频数据，以提取空间和态势信息(例如，车辆状态和型号参数)。只使用固定的摄像机，来自图像序列的信息与自适应控制器集成，该控制器直接向无人机传输致动器命令。我们的研究基础设施包括一架带有定制航空电子设备的X-Cell-60 R/C直升机，地面上的摄像机，以及执行所有必要处理的PC地面站。一个关键方面是超越基于传统视觉的运动估计和跟踪，利用新的递归贝叶斯估计方法，允许与控制系统完全耦合。启发式地，这涉及到状态(车辆位置、姿态和速度)和模型参数(质量、惯性矩、气动力等)的概率密度估计的传播。视觉组件对图像似然度进行建模，并描述在给定当前状态的情况下观察图像的概率。该估计在使用Sigma-Point Filter(SPF)框架的递归滤波过程中将视觉测量与动态车辆模型相结合。SPF方法是机器学习领域的一个新发展，被证明远远优于标准的基于扩展卡尔曼滤波的估计方法。这项研究的智力价值既有助于单独的组成部分领域，也有助于整体。以建议的方式集成不同的组件代表了一种跨学科的新方法，在集成传感、信息处理和控制方面提供了新的研究机会和应用。除了基础研究，对技术的更广泛影响包括可以在这种受控环境中研究的明显的商业和军事应用(例如，舰载直升机的视觉辅助垂直起降，或在城市环境中的灵活机动)。核心技术还可以扩展到其他信息技术领域，从图像跟踪和检测，到复杂生物系统的控制。

This project develops an integrated Bayesian framework for vision based control of Unmanned Aerial Vehicles (UAVs) through fundamental research in 1) model-based nonlinear state and parameter estimation, 2) intelligent adaptive control, and 3) image processing. We specifically address how real-time video data can be processed with ground-based sensors (and on-board avionics) to extract spatial and situational information (e.g., vehicle state and model parameters). Using only stationary video cameras, information from the sequence of images are integrated with an adaptive controller that transmits actuator commands directly to the UAV. Our research infrastructure consist of an X-Cell-60 R/C helicopter with custom avionics, video cameras on the ground, and a PC ground-station to perform all necessary processingA key aspect is to go beyond traditional vision based motion estimation and tracking, utilizing new approaches to recursive Bayesian estimation allowing full coupling with the control system. Heuristically, this involves the propagation of probabilistic density estimates for the state (vehicle position, attitude, and velocities) and model parameters (mass, moments of inertia, aerodynamic forces, etc.). The vision components models the ``image likelihood'' and describes the probability of observing the image given the current state. The estimation combines the vision measurements with the dynamic vehicle model in a recursive filtering procedure using a Sigma-Point Filter (SPF) framework. SPF methods are a recent development in machine learning, and are shown to be far superior to standard EKF based estimation approaches. The intellectual merit of the research contributes to both the individual component areas as well as the integrated whole. The integration of the different components in the proposed manner represents an interdisciplinary new approach, providing new research opportunities and applications in integrated sensing, information processing, and control. Beyond basic research, the broader impact to technology includes the obvious commercial and military applications that can be studied in this controlled environment (e.g. visually assisted vertical take-off and landing for ship board helicopters, or agile maneuvering through urban environments). The core technologies can also be extended to other information technology areas from image tracking and detection, to control of complex biological systems.