a long-term children's mental health assessment system using sensor-embedded block-shaped tangible user interfaces

本研究旨在开发可预测儿童心理健康的块状有形用户界面（TUIs）。今年的研究成果是提取儿童玩玩具积木时的运动特征和结构特征，用于案例研究，并构建多模态机器学习模型，以高精度和可解释性预测儿童的行为问题。我的工作观察并确定了儿童心理健康和行为问题的关键游戏类型：被动游戏、优柔寡断游戏、不活跃游戏和激烈游戏。基于这些发现，我的研究将上述观察到的风格量化为运动和结构特征，然后使用视频和嵌入传感器的玩具块的融合数据提取它们。运动数据包括时间序列中的加速度和旋转。结构数据包括块的堆叠层数、复杂度、宽/高长宽比、堆叠/拆卸等，均为时间序列。然后，我的工作提出了一种多模态机器学习方法，使用运动特征、结构特征和结构图像来实现准确和可解释的儿童心理健康预测。这项研究促成了一个涵盖人机交互（HCI）、心理健康和数据科学的跨学科领域。今年，它在国内和国际上向不同领域的广泛受众进行了介绍，并作为预测和监测儿童心理健康的有益方法获得了积极的反馈。

This research aims to develop block-shaped Tangible User Interfaces (TUIs) that robustly predict children's mental health. This year, the research achievement is extracting motion features and structural features when children are playing with toy blocks, for case studies and building Multimodal Machine Learning models to predict a child's behavior problems with high accuracy and interpretability.My work observed and identified crucial play styles that indicated children's mental health and behavioral problems: passive play, indecisive play, inactive play, and drastic play. Building on the findings, my research quantified the above-observed styles into motion and structural features, and then extracted them using the fused data from videos and sensor-embedded toy blocks. The motional data include the acceleration and rotation in the time series. The structural data include blocks' stacking layer count, complexity, width/height aspect ratio, and stacking/disassembly, all in time series. My work then proposed a Multimodal Machine Learning approach using motion features, structural features, and structural images to achieve accurate and interpretable child mental health predictions.This research contributed to an interdisciplinary field encompassing human-computer interaction (HCI), mental health, and data science. This year, it was presented domestically and internationally to broad audiences in different fields, and received positive feedback as a beneficial method for predicting and monitoring children's mental health.