Streamlining Volumetric Imaging, Analysis and Publication Using Immersive Virtual Reality

使用沉浸式虚拟现实简化体积成像、分析和发布

• 批准号: 10011054
• 负责人: Gianfranco Doretto
• 金额: $ 115.96万
• 依托单位: ISTOVISR
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-05-01 至 2021-10-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10011054
• 关键词: 3-Dimensional; 3D virtual reality; 4D Imaging; Address; Awareness; Basic Science; Big Data; Clinical Research; Collection; Communication; Communities; Computer software; Coupling; Data; Data Analyses; Data Collection; Depth Perception; Educational Materials; Foundations; General Population; Goals; Hearing; Hour; Image; Image Analysis; Imaging Device; Imaging technology; Information Distribution; Ingestion; Instruction; Investments; Journals; Language; Lasers; Lighting; Machine Learning; Manuals; Manuscripts; Marketing; Methods; Microscope; Microscopy; Modeling; Modernization; Monitor; Neurosciences; Pathway interactions; Positioning Attribute; Process; Production; Publications; Publishing; Reporting; Resolution; Resort; Running; Scanning; Science; Scientist; Services; Specialist; Speed; Structure; Techniques; Three-Dimensional Image; Three-Dimensional Imaging; Time; Training; Visual; Visualization; Work; adaptation algorithm; base; data dissemination; data exploration; experience; experimental study; feature detection; field study; image processing; imaging modality; improved; large datasets; learning strategy; machine learning algorithm; movie; novel; optogenetics; pre-clinical research; scale up; software development; structural biology; tool; virtual reality; virtual reality environment

Over the past 15 years, new imaging technologies and methods for high throughput imaging have revolutionized structural biology by extending the resolution and scale of collected images in 3 dimensions. The resulting image volumes are more typically hundreds of GB to even tens of TB and in some cases approach PB sizes. These file sizes pose challenges for image acquisition, image analysis, and communication of a representative set of raw data and quantification. Image acquisition runs can be lengthy and expensive, and often errors are not identified until after the completion of scanning. Large files contain many structures, and require machine learning (ML) strategies in a context that permits error correction. Scientific communication requires tools for ready access to raw data, and more efficient methods to communicate the rapidly accumulating sets of scientific information. We propose to leverage virtual reality (VR) and verbal communication within the VR environment, to streamline each of these stages of scientific work, by capitalizing on the more natural abilities for stereoscopic vision and hearing to process scenes and language. Based upon the tool base and direct volume rendering of large files that we have established in our VR software, called syGlass, we will first integrate VR into the microscope controls for tuning the microscope and then efficiently inspecting images in 3D as they are acquired (Aim 1). Next, we will introduce novel domain adaptation techniques in the ML field to scale up 3D image quantification capabilities for current acquisition sizes, by coupling them with user-optimized experiences that do not require ML expertise, and yet provide automated and accurate results (Aim 2). Finally, we will provide tools to efficiently generate narrated scientific presentations in VR for use in the lab setting, as manuscript publications, and for production of educational materials (Aim 3). In each of these activities, we will introduce paradigm shifts in the management of experiments, analysis of the resulting data, and publication of manuscripts and materials to other scientists and the general public.

在过去的15年中，新的成像技术和方法实现了高通量成像