权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Design the Future 2: CrowdDesignVR

设计未来 2：CrowdDesignVR

基本信息

批准号：
EP/R004471/1
负责人：
Per Ola Kristensson
金额：
$ 71.42万
依托单位：
University of Cambridge
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2018
资助国家：
英国
起止时间：
2018 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=EP%2FR004471%2F1
关键词：
Design Future CrowdDesignVR

项目摘要

Our initial proposal CrowdDesign set out to explore how we can aid rapid prototyping of mobile sensor-based user interfaces by exploiting the versatile sensor capabilities of mobile phones. The primary objective was to investigate if we can crowdsource such sensor-dependent tasks to mobile devices in order to assist designers in rapidly evaluating new interaction techniques in situ. We have identified a very strong research trajectory that motivates continuing the CrowdDesign project beyond this year: CrowdDesignVR. In this follow-up project we propose to substantially extend the scope of the CrowdDesign project and elevate it from the smartphone platform and into a virtual reality platform. To enable the exploration of a very promising research trajectory for crowdsourced human-computer interaction, we need to invest time and effort into realising a high-quality crowdsourcing platform for VR.CrowdDesignVR will be the first crowdsourcing system for virtual reality. It will distribute tasks across the Steam VR distribution network, which allows it to reach a large sample of VR users. Prior research cannot reach this scale, as research has been limited to opportunity-sampling local participants and then train them to use a specific VR system. In contrast, by enabling access to thousands or even tens of thousands of Steam users, CrownDesignVR facilitates user interaction data collection at a scale that is several orders of magnitudes larger. This provides a number of wider benefits: 1) we can during the course of the project create more accurate models of human actions; 2) we can collect sufficient training data to train machine learning models, such as deep neural network models to accurately decode common user interface interaction patterns, such as typing, gesturing and determining whether an action was intended or not by the user.Since crowdsourcing tasks in a high-fidelity VR environment is a new avenue of research, there are many fundamental questions that need to be answered. We believe this project could result in potential seminal work on the understanding of the design space for crowdsourcing in VR.Another potential impact is the data itself. Our internal work on building deep neural networks for decoding typing tasks on touchscreen and physical keyboards has revealed that deep neural networks (specifically, recurrent neural networks) output traditional hidden Markov model decoding. However, we have also found that the amount of data that needs to be collected is very large, in fact, we use our CrowdDesign task architecture as mentioned previously in our report to collect touchscreen data from hundreds of users. CrowdDesignVR can substantially widen the scope and let us tackle some of deep previously unsolved questions in user interface design, such as how we can build a gesture recogniser that is capable of learning to recognise both open-loop (direct recall from motor memory) and closed-loop (visually-guided motion) gestures on both the 2D plane and in 3D space. A large amount of data would allow us to train a recurrent neural network to learn this separation. The potential is large as users are always in a continuum between open-loop and closed-loop interaction. However, due to the fundamental differences in the underlying generative models that result in the observed behaviour, it is very difficult to collect sufficient training data in lab.

我们最初的建议CrowdDesign开始探索如何利用移动的手机的多功能传感器功能，帮助快速原型设计基于移动的传感器的用户界面。主要目的是调查，如果我们可以众包这种传感器依赖的任务，以帮助设计师在现场快速评估新的交互技术的移动的设备。我们已经确定了一个非常强大的研究轨迹，激励我们在今年之后继续CrowdDesign项目：CrowdDesignVR。在这个后续项目中，我们建议大幅扩展CrowdDesign项目的范围，并将其从智能手机平台提升到虚拟现实平台。为了探索一个非常有前途的人机交互众包研究轨迹，我们需要投入时间和精力来实现一个高质量的VR众包平台。CrowdDesignVR将是第一个虚拟现实众包系统。它将通过Steam VR分发网络分发任务，这使它能够接触到大量VR用户。之前的研究无法达到这种规模，因为研究仅限于对当地参与者进行机会抽样，然后训练他们使用特定的VR系统。相比之下，通过访问数千甚至数万名Steam用户，CrownDesignVR促进了用户交互数据收集，其规模要大几个数量级。这提供了一些更广泛的好处：1）我们可以在项目过程中创建更准确的人类行为模型; 2）我们可以收集足够的训练数据来训练机器学习模型，例如深度神经网络模型，以准确解码常见的用户界面交互模式，例如打字，手势并确定用户是否有意进行动作。由于高保真VR环境中的众包任务是一种新的研究途径，有许多基本问题需要回答。我们相信这个项目可能会对理解VR众包的设计空间产生潜在的开创性工作。另一个潜在的影响是数据本身。我们在构建深度神经网络以解码触摸屏和物理键盘上的打字任务方面的内部工作表明，深度神经网络（特别是递归神经网络）输出传统的隐马尔可夫模型解码。然而，我们也发现需要收集的数据量非常大，事实上，我们使用我们之前在报告中提到的CrowdDesign任务架构来收集数百名用户的触摸屏数据。CrowdDesignVR可以大大拓宽范围，让我们解决用户界面设计中一些以前未解决的深层次问题，例如我们如何构建一个手势识别器，能够学习识别2D平面和3D空间上的开环（从运动记忆中直接回忆）和闭环（视觉引导运动）手势。大量的数据将使我们能够训练一个循环神经网络来学习这种分离。潜力是巨大的，因为用户总是处于开环和闭环交互之间的连续体中。然而，由于导致观察到的行为的底层生成模型的根本差异，很难在实验室中收集足够的训练数据。