EFRI-M3C: Mobility Skill Acquisition and Learning through Alternative and Multimodal Perception for Visually Impaired People

EFRI-M3C：通过替代和多模态感知为视障人士获取和学习移动技能

• 批准号: 1137172
• 负责人: Zhigang Zhu
• 金额: $ 199.97万
• 依托单位: CUNY City College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2011
• 资助国家: 美国
• 起止时间: 2011-09-01 至 2018-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1137172&HistoricalAwards=false
• 关键词: EFRI; M3C; Mobility; Skill; Acquisition

The objective of this EFRI-M3C project is to develop models of sensorimotor control in order to establish a set of design criteria for developing improved assistive technologies for visually impaired people. Multimodal sensory information obtained by wearable machine sensors will substitute for and/or augment impaired vision with somatosensory and other novel stimulation (?transducing?) methods. The project has the following three research threads: (1) Determination of the information needed by visually impaired people to perform wayfinding and arm reaching tasks and the impact on task performance when transducing information through visual, auditory and vibrotactile modalities. (2) Understanding of motor skill acquisition and control through in vivo measurements of brain activity and movement performance. (3) Development of a sensorimotor model through man-machine co-learning, which can be used for improving sensor/display designs and applied to machines. The intellectual merit of this research is that it will generate a theoretical foundation for a deeper understanding of the neural mechanisms of sensorimotor integration and motor learning, shedding new light on these functions in humans and machines. This will lead to new design concepts of alternative perception, formulation of required information necessary for successful orientation and wayfinding, and development of cost-effective and revolutionary mechatronic devices to assist visually impaired people in achieving mobility functions comparable to people with normal vision. The interdisciplinary team includes experts in engineering, computer science, psychology, and applied physiology from the City College of New York and Georgia Tech, tackling challenging problems on the boundaries of sensing, cognition, and action. Advisory board members, including experts in (neuro-)ophthalmology and human vision research, as well as counselors at the NYS Commission for the Blind and Visually Handicapped, will provide guidance for the study. The broader impacts of this research will be assistive technologies for individuals with sensory impairments, whose numbers have been rising due to the increasing population of older adults in the US and around the world. The project will also create new and expand existing academic programs in assistive technologies, brain computer interfaces and sensorimotor integration. It will increase cross-campus education opportunities for students particularly in under-represented groups of the two campuses.

EFRI-M3C项目的目标是开发感觉运动控制模型，以便为视障人士开发改进的辅助技术建立一套设计标准。可穿戴机器传感器获得的多模式感觉信息将用体感和其他新的刺激(？传感？)来替代和/或增强视力受损。方法：研究方法。该项目有以下三个研究思路：(1)确定视障人士执行寻路和手臂到达任务所需的信息，以及通过视觉、听觉和振动触觉方式传递信息时对任务绩效的影响。(2)通过活体测量大脑活动和运动表现，了解运动技能的获得和控制。(3)通过人机协同学习开发了一种传感电机模型，该模型可用于改进传感器/显示器的设计，并应用于机器。这项研究的智力价值在于，它将为更深入地理解感觉运动整合和运动学习的神经机制提供理论基础，为人类和机器的这些功能提供新的线索。这将导致另类感知的新设计概念，制定成功定向和寻路所需的必要信息，并开发具有成本效益和革命性的机电设备，以帮助视障人士实现与视力正常的人相媲美的行动功能。这个跨学科的团队包括来自纽约城市学院和佐治亚理工学院的工程、计算机科学、心理学和应用生理学方面的专家，解决感知、认知和行动边界上的挑战性问题。顾问委员会成员，包括(神经)眼科和人类视力研究的专家，以及纽约盲人和视障人士委员会的顾问，将为这项研究提供指导。这项研究的更广泛影响将是为有感觉障碍的人提供辅助技术，由于美国和世界各地老年人人口的增加，这些人的数量一直在上升。该项目还将在辅助技术、脑机接口和传感器运动集成方面创建新的和扩展现有的学术项目。它将增加学生的跨校园教育机会，特别是在两个校区代表不足的群体中。

项目成果

Crowd Counting with Minimal Data Using Generative Adversarial Networks for Multiple Target Regression

• DOI: 10.1109/wacv.2018.00131
• 发表时间: 2018-03
• 期刊: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV)
• 作者: Greg Olmschenk;Hao Tang;Zhigang Zhu

A Hybrid Indoor Positioning System for Blind and Visually Impaired Using Bluetooth and Google Tango

使用蓝牙和 Google Tango 为盲人和视障人士提供混合室内定位系统

• 发表时间: 2018
• 期刊: Journal on technology and persons with disabilities
• 作者: Nair, V.;Tsangouri, C.;Xiao, B.;Olmschenk, G.;Seiple, W. H.;Zhu, Z.
• 通讯作者: Zhu, Z.

ASSIST: Personalized Indoor Navigation via Multimodal Sensors and High-Level Semantic Information

ASSIST：通过多模态传感器和高级语义信息进行个性化室内导航

• DOI: 10.1007/978-3-030-11024-6_9
• 发表时间: 2019
• 期刊: Roth S. (eds
• 作者: Nair, V.;Budhai, M.;Olmschenk, G.;Seiple, W. H.;Zhu, Z.
• 通讯作者: Zhu, Z.

ASSIST: Evaluating the usability and performance of an indoor navigation assistant for blind and visually impaired people

• DOI: 10.1080/10400435.2020.1809553
• 发表时间: 2020-09-02
• 期刊: ASSISTIVE TECHNOLOGY
• 影响因子: 1.8
• 作者: Nair, Vishnu;Olmschenk, Greg;Zhu, Zhigang
• 通讯作者: Zhu, Zhigang

Transformation from arm joint coordinates to hand external coordinates explains non-uniform precision of hand position sense in horizontal workspace

从手臂关节坐标到手外部坐标的转换解释了水平工作空间中手位置感精度的不均匀

• DOI: 10.1016/j.humov.2022.103020
• 发表时间: 2022
• 期刊: Human Movement Science
• 影响因子: 2.1
• 作者: Oh, Kyunggeune;Prilutsky, Boris I.
• 通讯作者: Prilutsky, Boris I.