SHB: Type II (INT): Synthesizing Self-Model and Mirror Feedback Imageries with Applications to Behavior Modeling for Children with Autism

该项目是一个跨学科的综合研究和教育计划，旨在开发处理镜像的新技术，旨在研究和启用自闭症谱系障碍(ASD)儿童的行为建模。这项研究的核心是一种“虚拟镜子”设备，它结合了校准的深度和视觉传感器网络，在房间大小的透明显示器上渲染任意形状虚拟镜子的视点相关的动态视图。通过多模式和空间多样化的传感器，该系统提供高保真、非侵入性的眼睛凝视、面部表情、身体姿势、身体运动和其他人类行为模式的捕获。将开发新的多媒体处理算法，用于利用要在常规显示器和虚拟镜子上呈现的有限目标训练数据，将2D和3D物理外观以及行为从源个体传输到目标个体。患有自闭症的儿童通常对社交缺乏兴趣，但似乎对镜子中自己的形象和其他人模仿他们的行为非常感兴趣。本项目开发的软硬件系统可以在传统的视觉媒体和沉浸式设备上提供前所未有的能力来创造新的自我行为。预计该系统将为治疗师、教师和照顾者提供更大的灵活性，为视频建模和自我建模治疗创建材料。在网络上的部署降低了技术障碍，增加了获得循证治疗的机会，并促进了家庭和诊所外行为建模的一致实践。通过结合视觉反馈和新行为的实时渲染，虚拟镜子有望为ASD儿童提供更有效的行为建模。成功将表明ASD的自身/其他系统可以被修改，并表明ASD的根本缺陷受到环境操纵的影响。拟议的研究计划是来自电气工程、心理学、医学和教育的PI的合作努力。教育目标是开发一个研究计划，在这个计划中，从一开始就合作地进行问题识别、数据共享和问题解决。计划开展的外展项目包括儿科诊所的设备演示、研究结果的电视纪录片和访问农村地区的高中，以促进利用技术解决重要的健康和社会问题，并扩大代表不足的群体参与STEM活动的范围。有关该项目的信息，包括但不限于描述、人员、表彰、最新结果和出版物，将张贴在http://www.vis.uky.edu/mialab/NSF_autism.上

This project is an interdisciplinary, integrated research and education program to develop novel technologies in manipulating mirror images, aimed at studying and enabling behavioral modeling of children with autism spectrum disorder (ASD). Central to the research is a "virtual-mirror" device that combines a network of calibrated depth and visual sensors to render a viewpoint-dependent dynamic view of an arbitrary-shaped virtual mirror on a room-size see-through display. Through multimodal and spatially-diverse sensors, the proposed system provides high-fidelity, non-intrusive capturing of eye gaze, facial expression, body pose, body movement, and other human behavioral patterns. New multimedia processing algorithms will be developed for transferring 2D and 3D physical appearances, as well as behaviors from a source individual to a target individual with limited target training data to be rendered on regular displays and the virtual mirror. Children with ASD typically lack interest in social interactions, but appear to be highly interested in their own image in mirrors and others imitating their actions. The software and hardware systems developed in this project can provide unprecedented capability in creating novel behaviors of self in both traditional visual medium and immersive devices. The system is expected to provide greater flexibility to therapists, teachers, and caretakers in creating material for video modeling and self-modeling therapy. Deployment over the web lowers the technology barrier, increases access to evidence-based treatments, and promotes consistent practice of behavioral modeling in home and beyond clinics. By combining visual feedback and real-time rendering of new behaviors, the virtual mirror is expected to deliver more effective behavioral modeling for children with ASD. Success will indicate that the self/other system in ASD can be modified, and suggest that a fundamental deficit in ASD is subject to environmental manipulation. The proposed research program is a collaborative effort of PIs from electrical engineering, psychology, medicine, and education. The educational objective is to develop a research program in which problem identification, data sharing, and problem solving occur collaboratively from the very beginning. Outreach programs including device demonstrations at pediatric clinics, television documentaries of research results, and visits to high schools in rural districts are planned to promote the use of technologies in solving important health and societal problems, as well as to broaden the participation of under-represented groups in STEM activities.Information about this project, including but not limited to the description, personnel, acknowledgements, latest results, and publications, will be posted at http://www.vis.uky.edu/mialab/NSF_autism.