RINGS: Bumblebee: A Neural Network Transformer Architecture for Summarization and Prediction in Interactive XR Applications
RINGS:Bumblebee:用于交互式 XR 应用中汇总和预测的神经网络 Transformer 架构
基本信息
- 批准号:2148367
- 负责人:
- 金额:$ 100万
- 依托单位:
- 依托单位国家:美国
- 项目类别:Continuing Grant
- 财政年份:2022
- 资助国家:美国
- 起止时间:2022-05-01 至 2025-04-30
- 项目状态:未结题
- 来源:
- 关键词:
项目摘要
Extended Reality (XR) applications tightly couple virtual content with the physical world through technologies such as virtual and augmented reality. These applications are compute-intensive, latency-sensitive, and bandwidth-hungry, making them ideal drivers for next-generation communication architectures. Current best-effort networking techniques struggle to meet the high demand of XR workloads leading to dropped or delayed network packets, observable jitter, and poor Quality-of-Experience. Fortunately, there are statistical correlations between past and future packets that go under-exploited in modern XR systems. The goal of this project is to develop a fast, resilient, and adaptive general neural network transformer-based architecture for interactive XR applications using machine learning (ML) to model, historically summarize, adapt to, predict, and then utilize streaming time-series data. The proposed modeling strategy is transformer-based, but it can represent context relationships going arbitrarily far back in time, something ordinarily infeasible, and it can also rapidly adapt to changing statistics via a novel adaptive final neural network layer. This work will result in a system that can transparently interpose network control messages to reduce bandwidth through forecasting and extrapolation. It will also help to mask packet delays and drops over wireless channels improving network coordination critical to many XR applications like AR guided surgery, search and rescue, digital telepresence and automotive heads up displays.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.
扩展现实(XR)应用通过虚拟和增强现实等技术将虚拟内容与物理世界紧密结合在一起。这些应用是计算密集型、延迟敏感型和带宽密集型应用,这使它们成为下一代通信架构的理想驱动因素。当前的尽力而为网络技术难以满足XR工作负载的高要求,从而导致网络数据包丢失或延迟、明显的抖动和糟糕的体验质量。幸运的是,在现代XR系统中,过去和未来的数据包之间存在统计相关性,这些数据包未得到充分利用。该项目的目标是开发一种快速、弹性和自适应的通用神经网络变压器架构,用于交互式XR应用程序,使用机器学习(ML)来建模、历史总结、适应、预测并随后利用流时间序列数据。所提出的建模策略是基于变压器的,但它可以表示任意远古时代的上下文关系,这通常是不可行的,它还可以通过一种新颖的自适应最终神经网络层来快速适应不断变化的统计信息。这项工作将导致一个系统,可以透明地插入网络控制消息,以减少带宽通过预测和外推。它还将有助于掩盖无线通道上的分组延迟和丢弃,改善网络协调,这对许多XR应用程序至关重要,如AR制导手术、搜索和救援、数字网真和汽车平视显示器。该奖项反映了NSF的法定使命,并通过使用基金会的智力优势和更广泛的影响审查标准进行评估,被认为值得支持。
项目成果
期刊论文数量(1)
专著数量(0)
科研奖励数量(0)
会议论文数量(0)
专利数量(0)
Scaling VR Video Conferencing
扩展 VR 视频会议
- DOI:10.1109/vr55154.2023.00080
- 发表时间:2023
- 期刊:
- 影响因子:0
- 作者:Dasari, Mallesham;Lu, Edward;Farb, Michael W.;Pereira, Nuno;Liang, Ivan;Rowe, Anthony
- 通讯作者:Rowe, Anthony
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
数据更新时间:{{ journalArticles.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ monograph.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ sciAawards.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ conferencePapers.updateTime }}
{{ item.title }}
- 作者:
{{ item.author }}
数据更新时间:{{ patent.updateTime }}
Anthony Rowe其他文献
MeshReduce: Scalable and Bandwidth Efficient 3D Scene Capture
MeshReduce:可扩展且带宽高效的 3D 场景捕获
- DOI:
10.1109/vr58804.2024.00026 - 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Tao Jin;Mallesham Dasari;Connor Smith;Kittipat Apicharttrisorn;Srinivasan Seshan;Anthony Rowe - 通讯作者:
Anthony Rowe
StageAR: Markerless Mobile Phone Localization for AR in Live Events
StageAR:现场活动中 AR 的无标记手机定位
- DOI:
- 发表时间:
2024 - 期刊:
- 影响因子:0
- 作者:
Tao Jin;Shengxi Wu;Mallesham Dasari;Kittipat Apicharttrisorn;Anthony Rowe - 通讯作者:
Anthony Rowe
The Cyber-Physical Metaverse - Where Digital Twins and Humans Come Together
网络物理元宇宙 - 数字孪生和人类相聚的地方
- DOI:
- 发表时间:
2023 - 期刊:
- 影响因子:0
- 作者:
Dirk Elias;D. Ziegenbein;Philipp Mundhenk;A. Hamann;Anthony Rowe - 通讯作者:
Anthony Rowe
<em>Haemonchus contortus</em> infection in sheep: Parasite fecundity correlates with worm size and host lymphocyte counts
- DOI:
10.1016/j.vetpar.2008.01.040 - 发表时间:
2008-05-31 - 期刊:
- 影响因子:
- 作者:
Anthony Rowe;Kate McMaster;David Emery;Nicholas Sangster - 通讯作者:
Nicholas Sangster
EUS diagnosis and endoscopic management of iatrogenic biliary stent placement into the portal vein
- DOI:
10.1016/j.gie.2014.04.033 - 发表时间:
2014-08-01 - 期刊:
- 影响因子:
- 作者:
Muhammad F. Dawwas;Jason Y. Huang;Anthony Rowe;Douglas B. Yim;Jason B. Samarasena;Kenneth J. Chang;John G. Lee - 通讯作者:
John G. Lee
Anthony Rowe的其他文献
{{
item.title }}
{{ item.translation_title }}
- DOI:
{{ item.doi }} - 发表时间:
{{ item.publish_year }} - 期刊:
- 影响因子:{{ item.factor }}
- 作者:
{{ item.authors }} - 通讯作者:
{{ item.author }}
{{ truncateString('Anthony Rowe', 18)}}的其他基金
NSF CPS: Student Travel Grant Cyber-Physical Systems Week 2017
NSF CPS:2017 年网络物理系统周学生旅行补助金
- 批准号:
1740941 - 财政年份:2017
- 资助金额:
$ 100万 - 项目类别:
Standard Grant
PFI:BIC - A Cost-effective Accurate and Resilient Indoor Positioning System
PFI:BIC - 经济高效、精确且有弹性的室内定位系统
- 批准号:
1534114 - 财政年份:2015
- 资助金额:
$ 100万 - 项目类别:
Standard Grant
CPS: Frontiers: Collaborative Research: ROSELINE: Enabling Robust, Secure and Efficient Knowledge of Time Across the System Stack
CPS:前沿:协作研究:ROSELINE:在整个系统堆栈中实现稳健、安全和高效的时间知识
- 批准号:
1329644 - 财政年份:2014
- 资助金额:
$ 100万 - 项目类别:
Continuing Grant
相似海外基金
The evolution of mimicry in a changing world: how shifting bumblebee communities affect selection on a colour-polymorphic hoverfly
不断变化的世界中拟态的演变:熊蜂群落的变化如何影响颜色多态食蚜蝇的选择
- 批准号:
2604204 - 财政年份:2022
- 资助金额:
$ 100万 - 项目类别:
Studentship
Tracking of pollinator movements at a landscape scale: bumblebee foraging and dispersal
在景观尺度上追踪传粉媒介的运动:大黄蜂的觅食和传播
- 批准号:
571809-2022 - 财政年份:2022
- 资助金额:
$ 100万 - 项目类别:
University Undergraduate Student Research Awards
Bumblebee conservation in agricultural landscapes - the role of landscape structure and the Environmental Land Management Scheme
农业景观中的熊蜂保护 - 景观结构的作用和环境土地管理计划
- 批准号:
2729523 - 财政年份:2022
- 资助金额:
$ 100万 - 项目类别:
Studentship
Investigating the effects of amino acids/sugars/bitter toxicants on bumblebee foraging preferences and behaviour
研究氨基酸/糖/苦味毒物对熊蜂觅食偏好和行为的影响
- 批准号:
2600514 - 财政年份:2021
- 资助金额:
$ 100万 - 项目类别:
Studentship
Tracking of pollinator movements at a landscape scale: bumblebee foraging and dispersal
在景观尺度上追踪传粉媒介的运动:大黄蜂的觅食和传播
- 批准号:
564476-2021 - 财政年份:2021
- 资助金额:
$ 100万 - 项目类别:
University Undergraduate Student Research Awards
ArcticBuzz: bumblebee community responses to climate change in Lapland
ArcticBuzz:拉普兰大黄蜂群落应对气候变化
- 批准号:
2445781 - 财政年份:2020
- 资助金额:
$ 100万 - 项目类别:
Studentship
Host plant quality and microclimate effects on native bumblebee pollinators
寄主植物质量和小气候对本地熊蜂传粉者的影响
- 批准号:
541184-2019 - 财政年份:2019
- 资助金额:
$ 100万 - 项目类别:
University Undergraduate Student Research Awards
Bumblebee Behaviour during Ecological Twilight
生态黄昏期间大黄蜂的行为
- 批准号:
2254597 - 财政年份:2019
- 资助金额:
$ 100万 - 项目类别:
Studentship
Breeding and disease screening for the introduction of at risk bumblebee species
引进高危熊蜂物种的育种和疾病筛查
- 批准号:
536308-2018 - 财政年份:2018
- 资助金额:
$ 100万 - 项目类别:
Alexander Graham Bell Canada Graduate Scholarships - Master's
Analyzing historical plant-pollinator interactions by conducting pollen metabarcoding on museum specimens of German bumblebee species
通过对德国熊蜂物种的博物馆标本进行花粉元条形码分析历史植物与传粉者的相互作用
- 批准号:
352447832 - 财政年份:2017
- 资助金额:
$ 100万 - 项目类别:
Priority Programmes