Crowdsourcing Labels and Explanations to Build More Robust, Explainable AI/ML Activity Models

项目摘要/摘要 随着65岁以上的人口从5800万人增长到2050年的8800万人， 阿尔茨海默病和相关痴呆症(ADRDS)老年患者1.家长项目 介绍临床驱动的技术方法，以自动评估老年人的功能健康 从多模式传感器数据。社区和我们的父项目中所缺少的是 地道的智能手表活动标签。如果没有足够数量的标签数据，机器学习模型 无法学习健壮的行为模型并使用这些模型进行功能健康预测。此外， 具有对应标签的活动类别非常不对称，进一步限制了机器学习 由于经典的班级分布不均衡问题而产生的性能。在本补充申请中，我们 建议大幅提高我们的父项目和 菲尔德。为此，我们将创建一种机制，通过Amazon Machine Turk众包活动标签。 此外，我们将利用众包机会将父项目推向下一步 为可解释的机器学习模型奠定了基础。一旦我们的活动标签目标数量是 我们将启动第二轮众包，要求公民科学家创造一句话 对应于活动实例的可视化数据的解释。补充项目将包括 四项任务。首先，我们将创建一个可视化和数据点选择工具，用于Amazon Mechanical 土耳其(AMT)数据收集论坛。将使用基线主动学习策略收集一组初始的 标注并创建基线模型，之后，将采用主动学习和注释器选择策略 细化以收集剩余的活动标签。最后，来自每个建模类别的不同数据点 将显示为培训讲解模型收集一组文本标题。本补编的结果 该项目将是最大的一组活动之一，标签为智能手表数据收集“在野外”。已标记的 本附录创建的数据集将为大量健康研究提供基础，这些研究可以利用 通过在真实世界研究中收集的连续可穿戴传感器读数观察到的活动信息。对于 父项目，标签数据量将增加10000%以上。该副刊还将提供 创建可解释的移动健康AI/ML工具的起点。

PROJECT SUMMARY / ABSTRACT As the population of individuals 65+ grows from 58 million to 88 million by 2050, so too will the number of individuals who are aging with Alzheimer's disease and related dementias (ADRDs)1. The parent project introduces clinically-driven technological methods to automate assessment of an older adult's functional health from multimodal sensor data. What is lacking in the community, and in our parent project, is the availability of ground-truth smartwatch activity labels. Without a sufficient amount of labeled data, machine learning models cannot learn robust behavior models and use these models for functional health prediction. Additionally, the categories of activities that have corresponding labels are very skewed, further limiting machine learning performance because of the classical imbalanced class distribution problem. In this supplement request, we propose to dramatically increase the availability of labeled smartwatch data for our parent project and for the field. To do this, we will create a mechanism to crowdsource activity labels through Amazon Mechanical Turk. Additionally, we will capitalize on the crowdsourcing opportunity to push the parent project to the next step by laying a foundation for explainable machine learning models. Once our target number of activity labels is reached, we will initiate a second round of crowdsourcing by asking citizen scientists to create one-sentence explanations of the visualized data corresponding to an activity instance. The supplement project will contain four tasks. First, we will create a visualization and data point-selection tool for use in the Amazon Mechanical Turk (AMT) forum for data collection. A baseline active learning strategy will be used to collect an initial set of labels and create a baseline model, after which, the active learning and annotator selection strategies will be refined to collect the remainder of the activity labels. Finally, diverse data points from each modeled category will be displayed to collect a set of text captions for training explanation models. The outcome of this supplement project will be one of the largest sets of activity labeled smartwatch data collected “in the wild.” The labeled datasets created by this supplement will offer a foundation for a multitude of health studies that can utilize activity information observed by continuous wearable sensor readings collected in real-world studies. For the parent project, the amount of labeled data will increase by over 10,000%. The supplement will also offer a starting point for creating explainable mobile health AI/ML tools.