EAPSI: Implementation of Advanced Computer Vision Techniques to Improve Real Time Imaging of Single Molecule Conformation Changes

该项目的目标是实现先进的计算机视觉算法，用于自动分析单分子的超慢动作电影。这项研究将与东京大学的原野幸二博士和他的团队合作进行，他们首次直接观察到单个分子正在经历特殊而重要的形状变化。这一事件是用配备了高速摄像机的强大显微镜拍摄下来的。了解分子的形状或构象的变化对于分析复杂的化学体系和反应至关重要。这项技术每秒产生超过100 GB的视频数据。目前，用于将这些数据编译成有意义的信息的暴力方法需要巨大的计算时间和能力。计算机视觉技术将极大地提高这种分析的速度和准确性，并在这一关键领域带来更多的科学成果。该项目也代表了美国和日本之间重要的科学知识和技术交流，并将导致未来在类似项目上的合作。Harano的团队首次使用先进的成像技术，使用配备了1600fps电子计数直接检测相机的像差校正高分辨率电子显微镜来观察和测量分子键的旋转。在进行有意义的分析之前，必须对这项技术产生的海量数据集进行处理。目前，该技术依赖于用于帧的自动分类、分类和对齐的数学互相关算法；对于1600fps的逐帧手动分类来说，这项任务过于繁琐。然而，互相关在某种程度上是一种蛮力方法，通过实现特征匹配计算机视觉算法可以极大地提高这一过程的速度和精度。该项目将实施和评估先进的计算机视觉算法，用于自动处理原野博士的成像技术产生的数据。该奖项由东亚和太平洋夏季学院计划设立，支持一名美国研究生的夏季研究，由NSF和日本科学促进会共同资助。

The goal of this project is to implement advanced computer vision algorithms for the automatic analysis of ultra-slow motion movies of single molecules. This research will be conducted in collaboration with Dr. Koji Harano and his team at the University of Tokyo, who have for the first time directly observed a single molecule undergoing a particular and important change in shape. The event was captured on video using a powerful microscope equipped with a high speed camera. Understanding the shape, or conformation, changes of molecules is critical to the analysis of complex chemical systems and reactions. This technique produces over 100 gigabytes of data per second of video. Currently, the brute-force method that is used to compile this data into meaningful information requires tremendous computing time and power. Computer vision techniques will vastly increase the speed and accuracy of this analysis and lead more scientific output in this critical area. This project also represents an important exchange of scientific knowledge and techniques between the United States and Japan and will lead to future collaboration on similar projects.Dr. Harano's team used an advanced imaging technique to observe and measure a molecular ó-bond rotation for the first time using an aberration corrected high resolution transmission electron microscope equipped with a 1600 fps electron counting direct detection camera. The massive datasets generated by this technique must be processed before meaningful analysis can take place. Presently the technique relies on mathematical cross-correlation algorithms for automatic sorting, classification, and alignment of frames; a task far too cumbersome for manual frame by frame sorting at 1600 fps. However, cross-correlation is somewhat of a brute-force approach and the speed and accuracy of this process can be greatly increased by implementing feature matching computer vision algorithms. This project will implement and evaluate advanced computer vision algorithms for the automatic processing of data generated by Dr. Harano's imaging technique.This award, under the East Asia and Pacific Summer Institutes program, supports summer research by a U.S. graduate student and is jointly funded by NSF and the Japan Society for the Promotion of Science.