Lifelearn: Unbounded activity and context awareness

点击翻译按钮获取中文摘要

The Wearable Computing market is expected to explode, as evidenced in 2014 and early 2015 with a plethora of new products primarily in the sports and fitness domain. Business Insider in 2013 estimated that 300 million units would be shipped by 2018. What makes wearables (and similarly mobile phones) unique is their contextual intelligence: they use sensors to infer users' context, such as location, activities or social interactions. This contextual intelligence allows a fitness tracker to detect by itself that the user is running, walking, or doing push-ups. We are motivated by the vision of pervasive "wearable smart assistants" that provide situated contextual support in daily life. They may act as "memory reminders" for people with dementia, or encourage healthy behaviours through supportive prompts presented at the right time (e.g. to fight obesity, diabetes, cardiovascular diseases).This project deals with the heart of any such assistive technology: the ability to recognise general human activities and context from sensors. Current methods can only recognise pre-defined or "closed sets" set of activities and context. This is insufficient for the scenarios outlined above. In such applications, the set of relevant activities is not necessarily known at design-time, as different users tend to have different routines, routines may change as users change interests, and activities may be performed differently, for instance after an injury. Therefore the set of relevant activities and contexts is potentially unbounded and is said to be "open-endedThe project investigates the methods required to recognise an "open-ended" set of activities and contexts from existing wearables, such as a smartwatch and a mobile phone, following lifelong learning principles. In other words, the system should discover that a user engages in a new activity, even if it was not initially programmed with the knowledge of that activity.We develop new open-ended learning techniques that can model changing number of classes at runtime. These methods run on a recognition infrastructure comprising software on the wearable devices and on a server. The infrastructure will be made open-source to benefit other projects. We develop methods that discover reoccurring wearable sensor patterns. Repeating patterns may correspond to new activities or contexts. Therefore they are modelled using open-ended learning techniques. Finally, we develop methods to decide whether a discovered pattern is meaningful and what it represents. This is achieved by involving the user and occasionally requesting to provide information about his/her current activity. We compare different feedback options that minimise the number of interruptions and the complexity of the queries. Overall, the system is evaluated on existing data as well as on a new long-term dataset collected within this project.Our approach is novel and timely. Performance increases in activity recognition are incremental and the inability to deal with unknown activities is most critical for large-scale deployments in daily life scenarios. This project addresses this fundamental limit. This is timely given raising costs of healthcare and calls to rely on technology to address this issue. The outcomes of this project along understanding daily human behaviour may lead to new smart assistants that could help support independent living or assist users in following healthy behaviour change. The outcomes may also find their use in psychology research and in the area of sustainable innovation, such as the assessment of consumer-product interaction and behaviour change initiatives. As such the project has clear societal benefits.This project is supported by our partners Unilever and Plessey Semiconductors, respectively interested in consumer behaviour research and new products in the healthcare domain.