CONVER-SE: Conversational Programming for Smart Environments

智能环境旨在对访问，生活和工作的人的需求做出智能反应。例如，当客厅变暗时，灯光可以亮起;当博物馆参观者走近时，视频展览可以用正确的语言播放。然而，我们缺乏直观的方法让没有技术背景的用户理解和重新配置这些环境的行为，而且公众对自动化环境相当不信任。虽然有一些工具可以让用户在没有编程知识的情况下查看和更改定义智能环境行为的规则，但它们还没有被技术爱好者广泛采用。现有工具的一个缺点是，它们将注意力从所讨论的环境中转移，要求用户从真实的世界对象转换为基于屏幕的抽象表示。新的编程工具，使用户能够利用他们的理解和引用对象在真实的世界可以大大增加信任和吸收的智能environments.This研究将调查用户如何理解和描述智能环境的行为，而在situation. and使用的研究结果，开发更直观的编程工具。例如，一个工具可以让某人简单地说，他们希望在天黑时打开一盏灯，并指向它以识别它。语音接口现在广泛用于智能个人助理，但功能主要限于发布即时命令或设置简单的提醒。实际上，使用语音接口进行编程任务存在许多挑战，并且理想化的交互（例如灯示例）实际上一点也不简单。在许多情况下，用于为日常用户设计编程接口的研究是在研究实验室中进行的，而不是在真实的家庭或工作场所中进行的，被邀请参加设计和评估研究的人通常是大学生或工作人员，或者对技术有兴趣或背景的人。这些界面一旦脱离了它们被设计和测试并提供给日常用户的一小部分玩具使用场景，就经常会崩溃。这项研究调查了使用语音进行编程的挑战，并评估了缓解这些挑战的方法，包括对话提示，使用手势和接近数据以避免歧义，以及提供可以自定义的默认行为。在这个项目中，我们主要关注智能家居场景，我们将在真实的家庭环境中进行研究。语音界面越来越多地被用于这些场景，但不支持通过语音查询、调试和改变行为。我们将招募没有编程背景的参与者，包括老年人和残疾人用户，他们通常被强调为可以从智能家居技术中受益的人，但很少被纳入此类研究。我们将在人们的家中进行采访，以了解他们如何自然地描述智能环境的规则，同时考虑到语音，手势和位置。我们将寻找他们描述的规则中的任何错误或不明确的元素，并调查研究人员的提示可以帮助他们清楚地表达规则。我们还将探讨参与者可以在多大程度上定制呈现给他们的默认行为。这些数据将被用来创建一个对话界面，利用人类提示的方法，并在真实的世界环境中进行测试。该系统的一些元素将由人类研究人员控制，但该系统将模拟与智能会话界面交互的体验。这将使我们能够确定在开发功能齐全的对话式编程工具方面所追求的富有成效的领域，这些工具在博物馆，教育，农业和机器人技术中也可能有用。

Smart environments are designed to react intelligently to the needs of those who visit, live and work in them. For example, the lights can come on when it gets dark in a living room or a video exhibit can play in the correct language when a museum visitor approaches it. However, we lack intuitive ways for users without technical backgrounds to understand and reconfigure the behaviours of such environments, and there is considerable public mistrust of automated environments. Whilst there are tools that let users view and change the rules defining smart environment behaviours without having programming knowledge, they have not seen wide uptake beyond technology enthusiasts. One drawback of existing tools is that they pull attention away from the environment in question, requiring users to translate from real world objects to abstract screen-based representations of them. New programming tools that allow users to harness their understandings of and references to objects in the real world could greatly increase trust and uptake of smart environments.This research will investigate how users understand and describe smart environment behaviours whilst in situ, and use the findings to develop more intuitive programming tools. For example, a tool could let someone simply say that they want a lamp to come on when it gets dark, and point at it to identify it. Speech interfaces are now widely used in intelligent personal assistants, but the functionality is largely limited to issuing immediate commands or setting simple reminders. In reality, there are many challenges with using speech interfaces for programming tasks, and idealised interactions such as the lamp example are not at all simple, in reality. In many cases, research used to design programming interfaces for everyday users is carried out in research labs rather than in the real home or workplace settings, and the people invited to take part in design and evaluation studies are often university students or staff, or people with an existing interest or background in technology. These interfaces often fall down once taken away from the small set of toy usage scenarios in which they have been designed and tested and given to everyday users. This research investigates the challenges with using speech for programming, and evaluates ways to mitigate these challenges, including conversational prompts, use of gesture and proximity data to avoid ambiguity, and providing default behaviours that can be customised. In this project, we focus primarily on smart home scenarios, and we will carry out our studies in real domestic settings. Speech interfaces are increasingly being used in these scenarios, but there is no support for querying, debugging and alternating the behaviours through speech.We will recruit participants with no programming background, including older and disabled users, who are often highlighted as people who could benefit from smart home technology, but rarely included in studies of this sort. We will carry out interviews in people's homes to understand how they naturally describe rules for smart environments, taking into account speech, gesture and location. We will look for any errors or unclear elements in the rules they describe, and investigate how far prompts from researchers can help them to be able to express the rules clearly. We will also explore how far participants can customise default behaviours presented to them. This data will be used to allow us to create a conversational interface that harnesses the approaches that worked with human prompts, and test it in real world settings. Some elements of the system will be controlled by a human researcher, but the system will simulate the experience of interacting with an intelligent conversational interface. This will allow us to identify fruitful areas to pursue in developing fully functional conversational programming tools, which may also be useful in museums, education, agriculture and robotics.