Graphics Pipelines for Next Generation Mixed Reality Systems
下一代混合现实系统的图形管道
基本信息
- 批准号:EP/T01346X/1
- 负责人:
- 金额:$ 106.49万
- 依托单位:
- 依托单位国家:英国
- 项目类别:Research Grant
- 财政年份:2020
- 资助国家:英国
- 起止时间:2020 至 无数据
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
In the past twenty years display technology has moved on considerablywith improvements such as higher resolution, faster frame rates andhigh-dynamic range colour. In the same period graphics processingunits (GPUs or graphics cards) have become significantly faster withbroader functionality. However, we argue that current implementationsof the traditional graphics pipeline, which is based on therasterisation and z-buffering, are unsuited to emerging displays.In particular, the types of near-eye displays used for augmentedreality and virtual reality provide new challenges. Since theircontexts of use are very different, it is not even clear whatproperties are ideal. Should the displays be multi-focal orvari-focal? How fast do display update rates need to be to supportregistered augmented reality systems? How do we exploit the propertiesof the human visual system to render more efficiently?The traditional computer graphics pipeline puts emphasis on generatingfull images at the display size and rate, where that display might be1920x1080 at 60Hz or higher. For a near-eye display, it is clear thatsuch a pipeline is nowhere near suitable: already one consumer HMD isdemanding 2480x1080 at 90Hz, roughly double the bandwidth that a commondesktop display requires. At these rates, there will still be problemswith visual acuity and latency (e.g. there is an inherent 11ms displaylag). Future HMDs are touted with resolutions of 8K pixels across, butthis will require a very significant increase in graphics compute andthus power consumption. Not only is this expensive, it could limit theform factor of the device and thus user-acceptance of the technology.There is very exciting work going on in display technologies at themoment. For example, Varjo (a project partner) are building a HMD thathas a moving display insert that tracks the eye; Facebook have built ademonstration display that uses a focal surface to generate multiplefocal depths in the same image.In comparison, the graphics pipeline is taken mostly as a given.Recently, the proposers, along with a small group of colleagues in thefield, have started to the challenge the status quo. We don't proposeto ditch the highly optimised compute units in graphics cards, butrather to study frameworks within they can be exploited more readily.We believe that by reformulating the graphics pipeline and payingattention to the very specific needs of near-eye displays, we canradically reduce the power required from GPUs, and thus make near-eyedisplay more usable.We will focus on three connected challenges that we have labelled thelatency, redundancy and bandwidth challenges. First, we will targetextremely low latency displays. We will develop systems that achieve>1000 fps visual output, with latency under 1ms and study how theseimpact visual response. Second, we will explore stronger decoupling offrame-based rendering from display. We note that in near-eye displaysmost pixels are wasted, and thus we target novel spatial and temporalalgorithms that reduce redundancy. Third, to exploit redundancy moregenerally, we need to use it to reduce bandwidth between graphics cardand display. Taking inspiration from the concept of surface lightfields, our concept of ambient fields will render to buffers that areexpected to be valid for re-rendering to the user for 10-100ms. In summary, we believe that the current graphics pipeline and itsassociated implementation in GPUs is unsuited to drive near-eyedisplays. We want near-eye displays to be low-cost, power efficientand highly acceptable to users. To achieve this, we propose newalgorithms that can use GPU capabilities more effectively. We targetreducing redundant compute to enable lower latency and lower bandwidthrequirements through the graphics pipeline.
在过去的二十年中,显示技术已经取得了长足的进步,如更高的分辨率,更快的帧速率和高动态范围的色彩。在同一时期,图形处理单元(gpu或图形卡)变得更快,功能更广泛。然而,我们认为传统图形管道的当前实现,这是基于光化和z缓冲,是不适合新兴显示器。特别是,用于增强现实和虚拟现实的近眼显示器类型带来了新的挑战。由于它们的使用环境非常不同,甚至不清楚哪种属性是理想的。显示器应该是多焦的还是多焦的?显示更新速度需要多快才能支持已注册的增强现实系统?我们如何利用人类视觉系统的特性来更有效地渲染?传统的计算机图形管道强调以显示尺寸和速率生成完整的图像,其中显示可能是60Hz或更高的1920x1080。对于近眼显示器,很明显,这样的管道是不合适的:已经有一个消费级HMD要求2480x1080的90Hz带宽,大约是普通桌面显示器所需带宽的两倍。在这样的速率下,仍然会有视觉敏锐度和延迟的问题(例如,有一个固有的11毫秒的显示延迟)。未来的头戴式显示器被吹捧为8K像素的分辨率,但这将需要非常显著的图形计算和功耗的增加。这不仅是昂贵的,它可能会限制设备的外形因素,从而限制用户对该技术的接受程度。目前,在显示技术方面正在进行非常令人兴奋的工作。例如,Varjo(一个项目合作伙伴)正在制造一种HMD,它有一个移动的显示插入,可以跟踪眼睛;Facebook已经建立了一个演示显示器,使用一个焦点表面在同一张图像中产生多焦点深度。相比之下,图形管道大多是给定的。最近,这些提议者与该领域的一小群同事一起,开始挑战现状。我们不建议抛弃图形卡中高度优化的计算单元,而是研究可以更容易地利用它们的框架。我们相信,通过重新制定图形管道并关注近眼显示的特定需求,我们可以从根本上降低gpu所需的功率,从而使近眼显示更可用。我们将重点关注三个相互关联的挑战,即延迟、冗余和带宽挑战。首先,我们将以极低延迟显示为目标。我们将开发系统,实现1000帧/秒的视觉输出,延迟低于1ms,并研究这些如何影响视觉响应。其次,我们将探索基于帧的渲染与显示的更强解耦。我们注意到,在近眼显示中,大多数像素被浪费了,因此我们的目标是新颖的空间和时间算法,以减少冗余。第三,为了更广泛地利用冗余,我们需要使用它来减少显卡和显示器之间的带宽。从表面光场的概念中获得灵感,我们的环境场概念将渲染到缓冲区,预计将在10-100ms内有效地重新渲染给用户。总之,我们认为当前的图形管道及其在gpu中的相关实现不适合驱动近眼显示。我们希望近眼显示器低成本、节能,并为用户所接受。为了实现这一点,我们提出了可以更有效地利用GPU功能的新算法。我们的目标是减少冗余计算,通过图形管道实现更低的延迟和更低的带宽需求。
项目成果
期刊论文数量(8)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Ubiq-exp: A toolkit to build and run remote and distributed mixed reality experiments
Ubiq-exp:用于构建和运行远程分布式混合现实实验的工具包
- DOI:10.3389/frvir.2022.912078
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Steed A
- 通讯作者:Steed A
Exploring Server-Centric Scalability for Social VR
探索以服务器为中心的社交 VR 可扩展性
- DOI:10.1109/ds-rt58998.2023.00016
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Friston S
- 通讯作者:Friston S
Metameric Inpainting for Image Warping.
用于图像变形的同色异谱修复。
- DOI:10.1109/tvcg.2022.3216712
- 发表时间:2023
- 期刊:
- 影响因子:5.2
- 作者:Dos Anjos RK
- 通讯作者:Dos Anjos RK
Perceptually guided computer-generated holography
感知引导计算机生成全息术
- DOI:10.1117/12.2610251
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Aksit K
- 通讯作者:Aksit K
Beyond Flicker, Beyond Blur: View-coherent Metameric Light Fields for Foveated Display
超越闪烁,超越模糊:用于注视点显示的视图相干同色异谱光场
- DOI:10.1109/vrw55335.2022.00152
- 发表时间:2022
- 期刊:
- 影响因子:0
- 作者:Kohli P
- 通讯作者:Kohli P
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Anthony Steed其他文献
警察の施策の認知が信頼に与える影響―伝統的信頼モデル、正統性モデル、主要価値類似性モデルの統合
警察政策认知对信任的影响:传统信任模型、合法性模型、键值相似度模型的融合
- DOI:
- 发表时间:
2022 - 期刊:
- 影响因子:0
- 作者:
Anthony Steed;Tuukka M. Takala;Dan Archer;Robert W. Lindeman;Wallace Lages;島田貴仁・讃井知・鈴木あい・春田悠佳 - 通讯作者:
島田貴仁・讃井知・鈴木あい・春田悠佳
Willingness of volunteers from Canadian Blood Service's Stem Cell Registry to donate blood, marrow, and other tissues for regenerative therapy
加拿大血液服务干细胞登记处的志愿者捐献血液、骨髓和其他组织用于再生治疗的意愿
- DOI:
10.1111/trf.15695 - 发表时间:
2020 - 期刊:
- 影响因子:2.9
- 作者:
Gary Liao;Karen Gilmore;Anthony Steed;H. Elmoazzen;D. Allan - 通讯作者:
D. Allan
AccompliceVR: Lending Assistance to Immersed Users by Adding a Generic Collaborative Layer
AccompliceVR:通过添加通用协作层为沉浸式用户提供帮助
- DOI:
10.1109/vrw62533.2024.00180 - 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Anthony Steed - 通讯作者:
Anthony Steed
Reviving the Euston Arch: A Mixed Reality Approach to Cultural Heritage Tours
复兴尤斯顿拱门:文化遗产之旅的混合现实方法
- DOI:
10.1109/ismar-adjunct60411.2023.00181 - 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
Ziwen Lu;Jingyi Zhang;Kalila Shapiro;Nels Numan;S. Julier;Anthony Steed - 通讯作者:
Anthony Steed
Comparison of hand tracking-based and controller-based interaction in a consumer virtual reality game
- DOI:
10.1007/s10055-025-01190-5 - 发表时间:
2025-07-25 - 期刊:
- 影响因子:5.000
- 作者:
Anthony Steed;Jiaqi Lai - 通讯作者:
Jiaqi Lai
Anthony Steed的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Anthony Steed', 18)}}的其他基金
Context Aware network architectures for Sending Multiple Senses (CASMS)
用于发送多重感知的上下文感知网络架构 (CASMS)
- 批准号:
EP/P004016/1 - 财政年份:2017
- 资助金额:
$ 106.49万 - 项目类别:
Research Grant
Open3D: Collaborative Editing for 3D Virtual Worlds
Open3D:3D 虚拟世界的协作编辑
- 批准号:
EP/M013685/1 - 财政年份:2015
- 资助金额:
$ 106.49万 - 项目类别:
Research Grant
Industrial Doctorate Centre: Virtual Environments, Imaging and Visualisation
工业博士中心:虚拟环境、成像和可视化
- 批准号:
EP/G037159/1 - 财政年份:2009
- 资助金额:
$ 106.49万 - 项目类别:
Training Grant
Eye Catching: Supporting tele-communicational eye-gaze in Collaborative Virtual Environments
引人注目:支持协作虚拟环境中的远程通信视线
- 批准号:
EP/E010032/1 - 财政年份:2006
- 资助金额:
$ 106.49万 - 项目类别:
Research Grant
相似海外基金
LTREB: Integrating real-time open data pipelines and forecasting to quantify ecosystem predictability at day to decadal scales
LTREB:集成实时开放数据管道和预测,以量化每日到十年尺度的生态系统可预测性
- 批准号:
2327030 - 财政年份:2024
- 资助金额:
$ 106.49万 - 项目类别:
Continuing Grant
Machine learning augmented molecular simulation pipelines for modelling allosteric modulation of protein function
机器学习增强分子模拟管道,用于模拟蛋白质功能的变构调节
- 批准号:
2871271 - 财政年份:2023
- 资助金额:
$ 106.49万 - 项目类别:
Studentship
Development of bioinformatic pipelines for multi omics data to model the effect of small molecule driven lipid metabolism on autophagy regulation
开发多组学数据的生物信息学管道,以模拟小分子驱动的脂质代谢对自噬调节的影响
- 批准号:
BB/Y512540/1 - 财政年份:2023
- 资助金额:
$ 106.49万 - 项目类别:
Training Grant
Planning: HBCU-UP: OPEN: Planning the Use of Strategic Partnerships to Build Pipelines for CHIPS through Morehouse College
规划:HBCU-UP:开放:规划利用战略合作伙伴关系通过莫尔豪斯学院为 CHIPS 建设管道
- 批准号:
2332411 - 财政年份:2023
- 资助金额:
$ 106.49万 - 项目类别:
Standard Grant
Preparing Successful STEM Teachers to Serve Students in High-Need Schools through Pathways, Pipelines, and Partnerships
培养成功的 STEM 教师通过途径、渠道和合作伙伴关系为高需求学校的学生提供服务
- 批准号:
2243351 - 财政年份:2023
- 资助金额:
$ 106.49万 - 项目类别:
Continuing Grant
Developing unbiased AI/Deep learning pipelines to strengthen lung cancer health disparities research
开发公正的人工智能/深度学习管道以加强肺癌健康差异研究
- 批准号:
10841956 - 财政年份:2023
- 资助金额:
$ 106.49万 - 项目类别:
Conference: Pipelines to Pathways: Humanizing Diversity in STEM Conference 2022
会议:从管道到途径:2022 年 STEM 会议人性化多样性
- 批准号:
2243212 - 财政年份:2023
- 资助金额:
$ 106.49万 - 项目类别:
Standard Grant
Improving the Robustness, Efficiency, and Transparency of Continuous Integration and Delivery Pipelines
提高持续集成和交付管道的稳健性、效率和透明度
- 批准号:
RGPIN-2022-02962 - 财政年份:2022
- 资助金额:
$ 106.49万 - 项目类别:
Discovery Grants Program - Individual
Development of Algorithm for Estimating Seismic Damage of Buried Pipelines Using State-Vector Space on Road Surface Ground Failures
路面地面破坏状态向量空间估计埋地管道地震损伤算法的开发
- 批准号:
22K18303 - 财政年份:2022
- 资助金额:
$ 106.49万 - 项目类别:
Grant-in-Aid for Challenging Research (Pioneering)
Developing multi-level pipelines for senolytics discoveries
开发用于 senolytics 发现的多层管道
- 批准号:
2753055 - 财政年份:2022
- 资助金额:
$ 106.49万 - 项目类别:
Studentship