a long-term children's mental health assessment system using sensor-embedded block-shaped tangible user interfaces

点击翻译按钮获取中文摘要

This research aims to develop block-shaped Tangible User Interfaces (TUIs) that robustly predict children's mental health. This year, the research achievement is extracting motion features and structural features when children are playing with toy blocks, for case studies and building Multimodal Machine Learning models to predict a child's behavior problems with high accuracy and interpretability.My work observed and identified crucial play styles that indicated children's mental health and behavioral problems: passive play, indecisive play, inactive play, and drastic play. Building on the findings, my research quantified the above-observed styles into motion and structural features, and then extracted them using the fused data from videos and sensor-embedded toy blocks. The motional data include the acceleration and rotation in the time series. The structural data include blocks' stacking layer count, complexity, width/height aspect ratio, and stacking/disassembly, all in time series. My work then proposed a Multimodal Machine Learning approach using motion features, structural features, and structural images to achieve accurate and interpretable child mental health predictions.This research contributed to an interdisciplinary field encompassing human-computer interaction (HCI), mental health, and data science. This year, it was presented domestically and internationally to broad audiences in different fields, and received positive feedback as a beneficial method for predicting and monitoring children's mental health.