Collaborative Research: ITR: A Robotics-Based Computational Environment to Simulate the Human Hand

本课题通过对基于模型的非线性状态和参数估计、智能自适应控制和图像处理等方面的基础研究，开发了一种基于视觉的无人机控制集成贝叶斯框架。我们特别讨论了如何使用地面传感器（和机载航空电子设备）处理实时视频数据，以提取空间和态势信息（例如，车辆状态和模型参数）。仅使用固定摄像机，来自图像序列的信息与自适应控制器集成，该控制器直接向无人机传输执行器命令。我们的研究基础设施包括一架带有定制航空电子设备的X-Cell-60 R/C直升机、地面摄像机和一个PC地面站，以执行所有必要的处理。关键方面是超越传统的基于视觉的运动估计和跟踪，利用递归贝叶斯估计的新方法，允许与控制系统完全耦合。启发式地，这涉及到状态（车辆位置、姿态和速度）和模型参数（质量、转动惯量、空气动力等）的概率密度估计的传播。视觉组件对“图像可能性”建模，并描述给定当前状态下观察图像的概率。该算法利用西格玛点滤波（SPF）框架将视觉测量结果与动态车辆模型相结合，进行递归滤波。SPF方法是机器学习的最新发展，并且被证明远远优于标准的基于EKF的估计方法。该研究的智力价值既有助于个别组成领域，也有助于整体。以提出的方式整合不同组件代表了一种跨学科的新方法，为集成传感，信息处理和控制提供了新的研究机会和应用。除了基础研究之外，对技术的更广泛影响包括可以在这种受控环境中研究的明显的商业和军事应用（例如，舰载直升机的视觉辅助垂直起降，或通过城市环境的敏捷机动）。核心技术还可以扩展到其他信息技术领域，从图像跟踪和检测，到复杂生物系统的控制。

This project develops an integrated Bayesian framework for vision based control of Unmanned Aerial Vehicles (UAVs) through fundamental research in 1) model-based nonlinear state and parameter estimation, 2) intelligent adaptive control, and 3) image processing. We specifically address how real-time video data can be processed with ground-based sensors (and on-board avionics) to extract spatial and situational information (e.g., vehicle state and model parameters). Using only stationary video cameras, information from the sequence of images are integrated with an adaptive controller that transmits actuator commands directly to the UAV. Our research infrastructure consist of an X-Cell-60 R/C helicopter with custom avionics, video cameras on the ground, and a PC ground-station to perform all necessary processingA key aspect is to go beyond traditional vision based motion estimation and tracking, utilizing new approaches to recursive Bayesian estimation allowing full coupling with the control system. Heuristically, this involves the propagation of probabilistic density estimates for the state (vehicle position, attitude, and velocities) and model parameters (mass, moments of inertia, aerodynamic forces, etc.). The vision components models the ``image likelihood'' and describes the probability of observing the image given the current state. The estimation combines the vision measurements with the dynamic vehicle model in a recursive filtering procedure using a Sigma-Point Filter (SPF) framework. SPF methods are a recent development in machine learning, and are shown to be far superior to standard EKF based estimation approaches. The intellectual merit of the research contributes to both the individual component areas as well as the integrated whole. The integration of the different components in the proposed manner represents an interdisciplinary new approach, providing new research opportunities and applications in integrated sensing, information processing, and control. Beyond basic research, the broader impact to technology includes the obvious commercial and military applications that can be studied in this controlled environment (e.g. visually assisted vertical take-off and landing for ship board helicopters, or agile maneuvering through urban environments). The core technologies can also be extended to other information technology areas from image tracking and detection, to control of complex biological systems.