Lifelearn: Unbounded activity and context awareness

可穿戴计算市场预计将呈爆炸式增长，正如2014年和2015年初在运动和健身领域出现大量新产品所证明的那样。2013年Business Insider估计，到2018年，ipad的出货量将达到3亿台。可穿戴设备（以及类似的手机）的独特之处在于它们的情境智能：它们使用传感器来推断用户的情境，比如位置、活动或社交互动。这种上下文智能允许健身追踪器自己检测用户正在跑步、散步或做俯卧撑。我们的动力来自于普及的“可穿戴智能助手”的愿景，这些助手可以在日常生活中提供情境支持。它们可以作为痴呆症患者的“记忆提醒器”，或通过在适当时间提供支持性提示（例如，与肥胖、糖尿病、心血管疾病作斗争）来鼓励健康行为。该项目涉及任何此类辅助技术的核心：从传感器识别一般人类活动和环境的能力。当前的方法只能识别预定义的或“封闭集”的活动和上下文。这对于上面概述的场景来说是不够的。在这样的应用程序中，相关活动的集合在设计时并不一定是已知的，因为不同的用户往往有不同的例程，例程可能随着用户兴趣的改变而改变，并且活动的执行可能不同，例如在受伤之后。因此，相关活动和环境的集合可能是无限的，被称为“开放式”。该项目研究了从现有的可穿戴设备（如智能手表和手机）中识别“开放式”活动和环境所需的方法，遵循终身学习原则。换句话说，系统应该发现用户参与了一项新的活动，即使它最初没有被编程为了解该活动。我们开发了新的开放式学习技术，可以在运行时对不断变化的类数量进行建模。这些方法在由可穿戴设备和服务器上的软件组成的识别基础设施上运行。基础设施将开放源代码，以使其他项目受益。我们开发了发现重复出现的可穿戴传感器模式的方法。重复的模式可能对应于新的活动或上下文。因此，他们使用开放式学习技术建模。最后，我们开发了一些方法来确定发现的模式是否有意义以及它代表什么。这是通过让用户参与进来，偶尔要求提供有关他/她当前活动的信息来实现的。我们比较不同的反馈选项，尽量减少中断的数量和查询的复杂性。总体而言，该系统是根据现有数据以及在该项目中收集的新的长期数据集进行评估的。我们的方法新颖而及时。活动识别的性能提高是渐进的，无法处理未知活动对于日常生活场景中的大规模部署是最关键的。这个项目解决了这个基本限制。考虑到医疗保健成本不断上升，以及依靠技术解决这一问题的呼声，这是及时的。这个项目的成果以及对人类日常行为的理解可能会导致新的智能助手，这些智能助手可以帮助支持独立生活或帮助用户遵循健康的行为改变。研究结果也可用于心理学研究和可持续创新领域，如消费者产品互动评估和行为改变倡议。因此，该项目具有明显的社会效益。该项目由我们的合作伙伴联合利华和普莱西半导体公司支持，他们分别对消费者行为研究和医疗保健领域的新产品感兴趣。

The Wearable Computing market is expected to explode, as evidenced in 2014 and early 2015 with a plethora of new products primarily in the sports and fitness domain. Business Insider in 2013 estimated that 300 million units would be shipped by 2018. What makes wearables (and similarly mobile phones) unique is their contextual intelligence: they use sensors to infer users' context, such as location, activities or social interactions. This contextual intelligence allows a fitness tracker to detect by itself that the user is running, walking, or doing push-ups. We are motivated by the vision of pervasive "wearable smart assistants" that provide situated contextual support in daily life. They may act as "memory reminders" for people with dementia, or encourage healthy behaviours through supportive prompts presented at the right time (e.g. to fight obesity, diabetes, cardiovascular diseases).This project deals with the heart of any such assistive technology: the ability to recognise general human activities and context from sensors. Current methods can only recognise pre-defined or "closed sets" set of activities and context. This is insufficient for the scenarios outlined above. In such applications, the set of relevant activities is not necessarily known at design-time, as different users tend to have different routines, routines may change as users change interests, and activities may be performed differently, for instance after an injury. Therefore the set of relevant activities and contexts is potentially unbounded and is said to be "open-endedThe project investigates the methods required to recognise an "open-ended" set of activities and contexts from existing wearables, such as a smartwatch and a mobile phone, following lifelong learning principles. In other words, the system should discover that a user engages in a new activity, even if it was not initially programmed with the knowledge of that activity.We develop new open-ended learning techniques that can model changing number of classes at runtime. These methods run on a recognition infrastructure comprising software on the wearable devices and on a server. The infrastructure will be made open-source to benefit other projects. We develop methods that discover reoccurring wearable sensor patterns. Repeating patterns may correspond to new activities or contexts. Therefore they are modelled using open-ended learning techniques. Finally, we develop methods to decide whether a discovered pattern is meaningful and what it represents. This is achieved by involving the user and occasionally requesting to provide information about his/her current activity. We compare different feedback options that minimise the number of interruptions and the complexity of the queries. Overall, the system is evaluated on existing data as well as on a new long-term dataset collected within this project.Our approach is novel and timely. Performance increases in activity recognition are incremental and the inability to deal with unknown activities is most critical for large-scale deployments in daily life scenarios. This project addresses this fundamental limit. This is timely given raising costs of healthcare and calls to rely on technology to address this issue. The outcomes of this project along understanding daily human behaviour may lead to new smart assistants that could help support independent living or assist users in following healthy behaviour change. The outcomes may also find their use in psychology research and in the area of sustainable innovation, such as the assessment of consumer-product interaction and behaviour change initiatives. As such the project has clear societal benefits.This project is supported by our partners Unilever and Plessey Semiconductors, respectively interested in consumer behaviour research and new products in the healthcare domain.