Multimodal Interaction with Biological Molecules in Virtual Environments

先进计算科学的一个核心问题是让科学家和工程师能够理解大型数据集和大型模拟的结果。最广为人知的传递这种理解的方法是可视化，它允许用户看到数据。然而，一些数据的复杂性和性质使得使用其他感官来表示是合适的。例如，形状可以通过触摸来确定。本课题针对大规模复杂科学计算问题，研究多感官虚拟环境。为了更好地集中研究努力并验证结果，它使用与复杂大分子数据集的多模式相互作用作为驱动应用。它的目标是设计技术和工具，通过视觉、听觉和触觉(力)反馈，在虚拟环境中与非常大的生物分子数据集进行有意义的相互作用。大多数生物分子数据集由几个基元(如原子类型、氨基酸侧链和核酸碱基)组成，这些基元结合在一起产生了丰富的生物复杂性。研究人员认为，这是富有代表性的其他科学可视化的子学科，但足够专业，以提供一个明确的研究目标。研究包括在顺序和并行架构上使用感知相关的多分辨率渲染的多模式反馈。所提出的改进使用了一组大多数生物分子数据集中共有的三个特征-有限的基序复杂性、有限的精度和可变水平的可感知细节-来优化多模式渲染过程。大多数科学可视化数据集至少共享两个特征，因此这里的进展应该会进一步推动虚拟现实辅助科学计算的其他领域的前沿研究。在这里，研究人员将使用常见主题的分层预渲染表示法，以允许在多模式上下文中以交互速度进行真实感。它们还将利用处理器对可变分辨率整数运算的支持，以及生物分子数据有限的动态范围和边界输入精度，通过以计算精度换取速度来透明地加速多模式渲染。最后，他们将利用这样一个事实，即人们的感官对最不明显的刺激有阈值，而且他们通常只在有限的区域感知高细节，以探索在一个集成和连贯的框架中处理跨多个感官的不同细节级别的渲染的技术。研究人员将通过与两名从事蛋白质折叠和理性药物设计的在校生物化学研究人员以及其他访问科学家和学者的合作，在真实数据集上验证所提出的方法的有效性。

A central problem of advanced computational science is making large datasets and the results of large simulations understandable by scientists andengineers. The best-known means of imparting this understanding isvisualization, which allows users to see the data. However, the complexityand nature of some data makes it appropriate to represent using the othersenses. For example, shape can be determined by touch. This project studiesmultisensory virtual environments for large--scale complex scientificcomputing problems. To better focus the research efforts and validate theresults, it uses multimodal interaction with complex big-molecular datasetsas the driving application. It aims to devise techniques and tools formeaningful interactions with very large biomolecular data sets in virtualenvironments with visual, aural, and haptic (force) feedback. Mostbiomolecular datasets consist of a few primitives (such as atom types, ammoacid sidechains, and nucleic acid bases) that are combined to yield richmosaics of biological complexity. The researchers believe this is richenough to be representative of other scientific visualizationsub-disciplines and yet specialized enough to offer a well-defined researchtarget.The research includes multimodal feedback using perception-dependentmultiresolution rendering on sequential and parallel architectures. Theproposed advances optimize the multimodal rendering processes using a setof three characteristics common across most biomolecular datasets - limitedmotif complexity, bounded accuracy, and variable level of perceptibledetail. Most scientific visualization datasets share at least two of thesecharacteristics and therefore advances here should further the cutting-edgeresearch in other areas of virtual-reality-assisted scientific computing.Here, the researchers will use hierarchical pre-rendered representations ofcommon motifs to allow realism at interactive speeds in multimodalcontexts. They will also take advantage of processor support for variableresolution integer arithmetic and the limited dynamic range and boundedinput accuracy of biomolecular data to transparently speed up multimodalrendering by trading computational accuracy for speed. Finally, they willtake advantage of the fact that people's senses have thresholds of theleast perceptible stimuli, and that they typically perceive high details inonly a limited region, to explore techniques to address variablelevel-of-detail renderings in an integrated and coherent framework acrossmultiple senses. The researchers will validate the efficacy of proposedapproaches on real-life datasets through a collaboration with two on-campusbiochemistry researchers working on protein folding and rational drugdesign and other visiting scientists and scholars.