Lifelearn: Unbounded activity and context awareness

Lifelearn：无限活动和情境意识

• 批准号: EP/N007816/1
• 负责人: Daniel Roggen
• 金额: $ 12.55万
• 依托单位: University of Sussex
• 依托单位国家: 英国
• 项目类别: Research Grant
• 财政年份: 2016
• 资助国家: 英国
• 起止时间: 2016 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=EP%2FN007816%2F1
• 关键词: Lifelearn; Unbounded; activity; context; awareness

The Wearable Computing market is expected to explode, as evidenced in 2014 and early 2015 with a plethora of new products primarily in the sports and fitness domain. Business Insider in 2013 estimated that 300 million units would be shipped by 2018. What makes wearables (and similarly mobile phones) unique is their contextual intelligence: they use sensors to infer users' context, such as location, activities or social interactions. This contextual intelligence allows a fitness tracker to detect by itself that the user is running, walking, or doing push-ups. We are motivated by the vision of pervasive "wearable smart assistants" that provide situated contextual support in daily life. They may act as "memory reminders" for people with dementia, or encourage healthy behaviours through supportive prompts presented at the right time (e.g. to fight obesity, diabetes, cardiovascular diseases).This project deals with the heart of any such assistive technology: the ability to recognise general human activities and context from sensors. Current methods can only recognise pre-defined or "closed sets" set of activities and context. This is insufficient for the scenarios outlined above. In such applications, the set of relevant activities is not necessarily known at design-time, as different users tend to have different routines, routines may change as users change interests, and activities may be performed differently, for instance after an injury. Therefore the set of relevant activities and contexts is potentially unbounded and is said to be "open-endedThe project investigates the methods required to recognise an "open-ended" set of activities and contexts from existing wearables, such as a smartwatch and a mobile phone, following lifelong learning principles. In other words, the system should discover that a user engages in a new activity, even if it was not initially programmed with the knowledge of that activity.We develop new open-ended learning techniques that can model changing number of classes at runtime. These methods run on a recognition infrastructure comprising software on the wearable devices and on a server. The infrastructure will be made open-source to benefit other projects. We develop methods that discover reoccurring wearable sensor patterns. Repeating patterns may correspond to new activities or contexts. Therefore they are modelled using open-ended learning techniques. Finally, we develop methods to decide whether a discovered pattern is meaningful and what it represents. This is achieved by involving the user and occasionally requesting to provide information about his/her current activity. We compare different feedback options that minimise the number of interruptions and the complexity of the queries. Overall, the system is evaluated on existing data as well as on a new long-term dataset collected within this project.Our approach is novel and timely. Performance increases in activity recognition are incremental and the inability to deal with unknown activities is most critical for large-scale deployments in daily life scenarios. This project addresses this fundamental limit. This is timely given raising costs of healthcare and calls to rely on technology to address this issue. The outcomes of this project along understanding daily human behaviour may lead to new smart assistants that could help support independent living or assist users in following healthy behaviour change. The outcomes may also find their use in psychology research and in the area of sustainable innovation, such as the assessment of consumer-product interaction and behaviour change initiatives. As such the project has clear societal benefits.This project is supported by our partners Unilever and Plessey Semiconductors, respectively interested in consumer behaviour research and new products in the healthcare domain.

可穿戴计算市场预计将爆炸式增长，2014年和2015年初将出现大量新产品，主要是在体育和健身领域。Business Insider在2013年估计，到2018年将出货3亿台。可穿戴设备（以及类似的移动的手机）的独特之处在于它们的情境智能：它们使用传感器来推断用户的情境，如位置、活动或社交互动。这种上下文智能允许健身跟踪器自己检测用户正在跑步，走路或做俯卧撑。我们的动力来自于无处不在的“可穿戴智能助手”的愿景，这些助手在日常生活中提供情境支持。它们可以作为痴呆症患者的“记忆提醒器”，或者通过在正确的时间提供支持性提示来鼓励健康的行为（例如对抗肥胖、糖尿病、心血管疾病）。该项目涉及任何此类辅助技术的核心：从传感器识别一般人类活动和背景的能力。目前的方法只能识别预定义的或“封闭集”的活动和上下文。这不足以应付上述情况。在这样的应用中，相关活动的集合在设计时不一定是已知的，因为不同的用户倾向于具有不同的例程，例程可以随着用户改变兴趣而改变，并且活动可以不同地执行，例如在受伤之后。因此，相关活动和上下文的集合可能是无限的，并且被称为“开放式的”。该项目研究了从现有的可穿戴设备（例如智能手表和移动的电话）中识别“开放式”活动和上下文集合所需的方法，遵循终身学习原则。换句话说，系统应该发现，用户从事一个新的活动，即使它不是最初编程的知识，activity.We开发新的开放式学习技术，可以在运行时改变类的数量。这些方法在识别基础设施上运行，该识别基础设施包括可穿戴设备上和服务器上的软件。该基础设施将开放源代码，以使其他项目受益。我们开发的方法，发现重复出现的可穿戴传感器模式。重复模式可能对应于新的活动或环境。因此，他们使用开放式学习技术建模。最后，我们开发方法来决定发现的模式是否有意义以及它代表什么。这是通过让用户参与并偶尔请求提供有关其当前活动的信息来实现的。我们比较了不同的反馈选项，最大限度地减少中断的数量和查询的复杂性。总的来说，该系统是在现有的数据，以及在这个项目中收集的一个新的长期数据集进行评估。我们的方法是新颖和及时的。活动识别的性能提升是渐进式的，无法处理未知活动对于日常生活场景中的大规模部署至关重要。本项目解决了这一基本限制。考虑到医疗保健成本的提高以及依靠技术解决这一问题的呼声，这是及时的。该项目的成果沿着对人类日常行为的理解，可能会产生新的智能助手，帮助支持独立生活或帮助用户遵循健康的行为改变。研究结果还可用于心理学研究和可持续创新领域，如评估消费者与产品的互动和行为改变举措。该项目得到了我们的合作伙伴联合利华和Plessey Semiconductors的支持，这两家公司分别对消费者行为研究和医疗保健领域的新产品感兴趣。

项目成果

Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations

• DOI: 10.1145/2971763.2971764
• 发表时间: 2016-09
• 期刊: Proceedings of the 2016 ACM International Symposium on Wearable Computers
• 作者: Francisco Javier Ordonez;D. Roggen

Complex human gestures encoding from wearable inertial sensors for activity recognition

可穿戴惯性传感器对复杂的人体手势进行编码以进行活动识别

• 发表时间: 2018
• 作者: Ciliberto M

Demo: Complex human gestures encoding from wearable inertial sensors for activity recognition

演示：从可穿戴惯性传感器编码复杂的人体手势以进行活动识别

• 发表时间: 2018
• 作者: Ciliberto M

BlueSense: designing an extensible platform for wearable motion sensing, sensor research and IoT applications

BlueSense：为可穿戴运动传感、传感器研究和物联网应用设计可扩展平台

• 发表时间: 2018
• 作者: Roggen D

Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition.

• DOI: 10.3390/s16010115
• 发表时间: 2016-01-18
• 期刊: Sensors (Basel, Switzerland)
• 作者: Ordóñez FJ;Roggen D
• 通讯作者: Roggen D