EAPSI: Implementation of Advanced Computer Vision Techniques to Improve Real Time Imaging of Single Molecule Conformation Changes

该项目的目标是实现先进的计算机视觉算法，用于自动分析单分子的超慢动作电影。这项研究将与东京大学的Koji Harano博士和他的团队合作进行，他们首次直接观察到单个分子在形状上发生了特殊而重要的变化。这一事件是用配备高速摄像机的强大显微镜拍下的。了解分子的形状或构象变化对于分析复杂的化学系统和反应是至关重要的。这种技术每秒产生超过100gb的视频数据。目前，将这些数据编译成有意义的信息的暴力破解方法需要大量的计算时间和能力。计算机视觉技术将极大地提高这种分析的速度和准确性，并在这一关键领域引领更多的科学产出。该项目还代表了美国和日本之间科学知识和技术的重要交流，并将导致未来在类似项目上的合作。Harano的团队使用了一种先进的成像技术，首次使用配备1600 fps电子计数直接检测相机的像差校正高分辨率透射电子显微镜来观察和测量分子ó-bond旋转。在进行有意义的分析之前，必须对这种技术产生的大量数据集进行处理。目前，该技术依赖于数学相互关联算法来自动排序、分类和对齐帧；这个任务对于1600帧/秒的手动逐帧排序来说太麻烦了。然而，相互关联在某种程度上是一种蛮力方法，通过实现特征匹配计算机视觉算法可以大大提高该过程的速度和准确性。该项目将实施和评估先进的计算机视觉算法，用于自动处理由Harano博士的成像技术生成的数据。该奖项由美国国家科学基金会和日本科学促进会共同资助，隶属于东亚和太平洋暑期研究所项目，支持一名美国研究生进行暑期研究。

The goal of this project is to implement advanced computer vision algorithms for the automatic analysis of ultra-slow motion movies of single molecules. This research will be conducted in collaboration with Dr. Koji Harano and his team at the University of Tokyo, who have for the first time directly observed a single molecule undergoing a particular and important change in shape. The event was captured on video using a powerful microscope equipped with a high speed camera. Understanding the shape, or conformation, changes of molecules is critical to the analysis of complex chemical systems and reactions. This technique produces over 100 gigabytes of data per second of video. Currently, the brute-force method that is used to compile this data into meaningful information requires tremendous computing time and power. Computer vision techniques will vastly increase the speed and accuracy of this analysis and lead more scientific output in this critical area. This project also represents an important exchange of scientific knowledge and techniques between the United States and Japan and will lead to future collaboration on similar projects.Dr. Harano's team used an advanced imaging technique to observe and measure a molecular ó-bond rotation for the first time using an aberration corrected high resolution transmission electron microscope equipped with a 1600 fps electron counting direct detection camera. The massive datasets generated by this technique must be processed before meaningful analysis can take place. Presently the technique relies on mathematical cross-correlation algorithms for automatic sorting, classification, and alignment of frames; a task far too cumbersome for manual frame by frame sorting at 1600 fps. However, cross-correlation is somewhat of a brute-force approach and the speed and accuracy of this process can be greatly increased by implementing feature matching computer vision algorithms. This project will implement and evaluate advanced computer vision algorithms for the automatic processing of data generated by Dr. Harano's imaging technique.This award, under the East Asia and Pacific Summer Institutes program, supports summer research by a U.S. graduate student and is jointly funded by NSF and the Japan Society for the Promotion of Science.