PFI:BIC: iWork, a Modular Multi-Sensing Adaptive Robot-Based Service for Vocational Assessment, Personalized Worker Training and Rehabilitation.

PFI：BIC：iWork，一种基于模块化多传感自适应机器人的服务，用于职业评估、个性化工人培训和康复。

• 批准号: 1719031
• 负责人: Fillia Makedon
• 金额: $ 99.96万
• 依托单位: University of Texas at Arlington
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2023-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1719031&HistoricalAwards=false
• 关键词: PFI; BIC; iWork; Modular; Multi

Automation, foreign competition, and the increasing use of robots replacing human jobs, stress the need for a major shift in vocational training practices to training for intelligent manufacturing environments, so-called "Industry 4.0". In particular, vocational safety training using the latest robot and other technologies is imperative, as thousands of workers lose their job or die on the job each year due to accidents, unforeseen injuries, and lack of appropriate assessment and training. The objective of this Partnerships for Innovation: Building Innovation Capacity (PFI:BIC) project is to develop iWork, a smart robot-based vocational assessment and intervention system to assess the physical, cognitive and collaboration skills of an industry worker while he/she performs a manufacturing tasks in a simulated industry setting and collaborating with a robot to do the task. The aim is to transform traditional vocational training and rehabilitation practices to an evidence-based and personalized system that can be used to (re)train, retain, and prepare workers for robotic factories of the future. The need for personalized vocational training, rehabilitation and accurate job-matching is essential to ensuring a strong manufacturing sector, vital to America's economic development and ability to innovate. The iWork service is "smart" because it can adjust and adapt to the individual's abilities as it assesses him/her and help decide on the type of tasks needed to test and train, based on the job's complexity, difficulty or familiarity to the worker. The iWork system integrates human expert knowledge to overcome or compensate for detected worker constraints. Research has shown that robot trainers can increase motivation and sustain interest, increase compliance and learning, and provide training for specific and individual needs. The iWork system aims to assess and train both the human and the work-assistive robot, as they collaborate on a manufacturing job. The projected outcome is low-cost vocational training solutions that can have substantial economic and societal benefits to diverse economic sectors. Most importantly, if successful, projected outcomes could impact how millions of persons seeking a manufacturing job are trained, including those facing a type of learning, physical or aging disability. The system's mobile, low cost methods accelerate recognizing a worker's specific needs and improve the ability of the vocational expert to make correlations between cognitive and physical assessments, thus empowering traditional practices with user-centric targeted training methods. In addition, the project's robot-based emphasis on safety and risk assessment, can reduce liability costs and productivity setbacks faced by industry, due to manufacturing accidents. The iWork system uses computational methods in reinforcement (machine) learning, data mining, collaborative filtering and human robot interaction to collect and analyze multi-sensing worker data during a manufacturing human-robot collaboration simulation. Data collected and analyzed come from sensors, wearables, and explicit user feedback measuring worker movements, eye gazes, errors made, performance delays, human-robot interactions, physiological metrics, and others, depending on the task. The system has a closed loop architecture composed of four phases: assessment, recommendation, intervention (or adjustment), and evaluation, with a human expert in the loop. The system generates recommendations for personalized interventions to the expert, at different loop intervals. Use of the latest developments in sensing technologies, robotics and intelligent communications, assess the ability to enhance the intelligence of a robot co-worker with more human-like learning and collaboration abilities to support the human in achieving a task. The system is modular and customizable to a particular manufacturing task, domain or worker robot. Two types of robots are used, socially assistive robots that provide non-contact user assistance through feedback and physically assistive robots that provide cognitive, physical and collaboration skill training. To predict risks of injury due to inattention, age, vision, or physical and mental issues, motion analysis and kinematics experiments are conducted to determine the type of safety training needed, to assess how well a human interacts with a collaborative robot, and how best to train the robot to help the human overcome identified physical and other deficiencies in performing a given task. The project integrates three main areas of expertise, engineered service system design, where assistive robots interact with and train each other to collaborate; computing, sensing, and information technologies, where machine learning, data mining and recommender algorithms are used to identify behavioral patterns of interest, and recommend targeted interventions; and human factors and cognitive engineering that deploy methods from the team's expertise in workplace assessment, personalized psychiatric intervention, and evaluation methods of vocational satisfaction, work habits, work quality, etc., as they relate to job preparation and retention.The project has an interdisciplinary team of experts from two collaborating universities, University of Texas Arlington (UTA) and Yale University, representing several fields, including human factors, psychology, computing, and industrial organization. The project deploys two primary industry partners, SoftBank Robotics (San Francisco, CA) manufacturer of humanoid service robots, and InteraXon (Canada), producing mobile EEG devices, who provides hardware, software and know-how to enhance iWork's functionality in cognitive activity monitoring. The broader context partners include, C8Sciences (USA), Assistive Technology Resources (USA), Barrett Technologies Inc. (USA), and the Dallas Veteran Affairs Research Corp. (USA).

自动化、国外竞争以及越来越多地使用机器人取代人类工作，强调了职业培训实践向智能制造环境培训的重大转变的必要性，即所谓的“工业4.0”。特别是，使用最新机器人和其他技术的职业安全培训势在必行，因为每年都有成千上万的工人因事故、意外伤害和缺乏适当的评估和培训而失去工作或死于工作。创新伙伴关系：建设创新能力（PFI:BIC）项目的目标是开发iWork，这是一个基于智能机器人的职业评估和干预系统，用于评估工业工人在模拟工业环境中执行制造任务并与机器人合作时的身体，认知和协作技能。其目的是将传统的职业培训和康复实践转变为基于证据和个性化的系统，可用于（重新）培训、留住和为未来的机器人工厂做好准备的工人。个性化的职业培训、康复和准确的工作匹配对于确保强大的制造业至关重要，这对美国的经济发展和创新能力至关重要。iWork服务之所以“智能”，是因为它可以根据个人的能力进行调整和适应，并根据工作的复杂性、难度或对员工的熟悉程度，帮助决定需要测试和培训的任务类型。iWork系统集成了人类专家知识，以克服或补偿检测到的工人约束。研究表明，机器人训练师可以增加动机和保持兴趣，提高依从性和学习能力，并为特定和个人需求提供培训。iWork系统旨在评估和培训人类和工作辅助机器人，因为他们在制造工作中合作。预计的结果是低成本的职业培训解决方案，可以为不同的经济部门带来巨大的经济和社会效益。最重要的是，如果成功，预计的结果可能会影响数百万寻求制造业工作的人的培训方式，包括那些面临学习、身体或老年残疾的人。该系统的移动、低成本方法加速了对工人特定需求的识别，并提高了职业专家在认知和身体评估之间建立关联的能力，从而使传统实践与以用户为中心的有针对性的培训方法相结合。此外，该项目以机器人为基础，强调安全和风险评估，可以减少由于制造事故而导致的责任成本和行业面临的生产力挫折。iWork系统使用强化（机器）学习、数据挖掘、协同过滤和人机交互的计算方法，在制造人机协作模拟过程中收集和分析多传感工人数据。收集和分析的数据来自传感器、可穿戴设备和明确的用户反馈，根据任务的不同，测量工人的动作、目光、错误、性能延迟、人机交互、生理指标等。该系统有一个闭环架构，由四个阶段组成：评估、建议、干预（或调整）和评估，在这个循环中有一个人类专家。该系统以不同的循环间隔为专家提供个性化干预建议。利用传感技术、机器人技术和智能通信的最新发展，评估提高机器人同事智能的能力，使其具有更多类似人类的学习和协作能力，以支持人类完成任务。该系统是模块化的，可针对特定的制造任务、领域或工人机器人进行定制。使用了两种类型的机器人，一种是通过反馈提供非接触用户帮助的社交辅助机器人，另一种是提供认知、身体和协作技能培训的物理辅助机器人。为了预测由于注意力不集中、年龄、视力或身体和精神问题而造成的伤害风险，进行了运动分析和运动学实验，以确定所需的安全训练类型，评估人类与协作机器人的互动程度，以及如何最好地训练机器人来帮助人类克服执行给定任务时确定的身体和其他缺陷。该项目整合了三个主要的专业领域：工程服务系统设计，其中辅助机器人相互交互并相互训练以进行协作；计算、传感和信息技术，其中使用机器学习、数据挖掘和推荐算法来识别感兴趣的行为模式，并推荐有针对性的干预措施；以及人因和认知工程，运用团队在工作场所评估、个性化精神病学干预、职业满意度、工作习惯、工作质量等方面的专业知识的方法，因为它们与工作准备和保留有关。该项目由来自德克萨斯大学阿灵顿分校（UTA）和耶鲁大学两所合作大学的跨学科专家团队组成，代表了几个领域，包括人为因素、心理学、计算机和工业组织。该项目部署了两个主要的行业合作伙伴，人形服务机器人制造商SoftBank Robotics（加利福尼亚州旧金山）和生产移动脑电图设备的InteraXon（加拿大），后者提供硬件、软件和技术来增强iWork在认知活动监测方面的功能。更广泛的合作伙伴包括C8Sciences（美国）、辅助技术资源公司（美国）、Barrett Technologies Inc.（美国）和达拉斯退伍军人事务研究公司（美国）。

项目成果

Towards predicting task performance from EEG signals

• DOI: 10.1109/bigdata.2017.8258478
• 发表时间: 2017-12
• 期刊: 2017 IEEE International Conference on Big Data (Big Data)
• 作者: Michalis Papakostas;K. Tsiakas;Theodoros Giannakopoulos;F. Makedon

Towards a Real-Time Cognitive Load Assessment System for Industrial Human-Robot Cooperation

面向工业人机合作的实时认知负荷评估系统

• DOI: 10.1109/ro-man47096.2020.9223531
• 发表时间: 2020
• 期刊: IEEE
• 作者: Rajavenkatanarayanan, Akilesh;Nambiappan, Harish Ram;Kyrarini, Maria;Makedon, Fillia
• 通讯作者: Makedon, Fillia

Edge-IoT framework for speech and mobile-based human-robot interaction

用于语音和基于移动的人机交互的边缘物联网框架

• DOI: 10.1145/3498361.3538767
• 发表时间: 2022
• 期刊: ACM
• 作者: Nambiappan, Harish Ram;Karim, Enamul;Saurav, Jillur Rahman;Srivastav, Anushka;Makedon, Fillia
• 通讯作者: Makedon, Fillia

Designing a Vocational Immersive Storytelling Training and Support System to Evaluate Impact on Working and Episodic Memory

设计职业沉浸式讲故事培训和支持系统，以评估对工作和情景记忆的影响

• DOI: 10.1145/3453892.3462216
• 发表时间: 2021
• 期刊: ACM
• 作者: Doolani, Sanika;Wessels, Callen;Makedon, Fillia
• 通讯作者: Makedon, Fillia

A Human Robot Interaction Framework for Robotic Motor Skill Learning

用于机器人运动技能学习的人机交互框架

• DOI: 10.1145/3197768.3197790
• 发表时间: 2018
• 期刊: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference on - PETRA '18
• 作者: Theofanidis, Michail;Cloud, Joe;Babu, Ashwin Ramesh;Brady, James;Makedon, Fillia
• 通讯作者: Makedon, Fillia