Intersubjective AI-driven multimodal interaction for advanced user-centric human robot collaborative applications (Jarvis)

为了加强欧盟工业的竞争力，越来越需要灵活的生产，利用机器和人力的能力。不断增长的机器人市场已经做出了反应，通过提供具有有效载荷，类型和移动性等各种特征的协作机器人解决方案来适应不断变化的情况。许多感知和安全的技术解决方案允许它们与人类共存。虽然在过去的5年中已经进行了一些研究活动，但由于以下限制，更广泛的工业应用仍然滞后：·有限的认知和智能·协作操作的低性能·协作流动性相当低，因为操作员必须适应机器人的特殊性。·机器人编程的复杂性，需要熟练的工程师参与，在执行阶段不提供灵活性，并且不能从经验丰富的操作员的隐性知识中受益。因此，尽管机电一体化相当先进，但需要机械和控制的完美匹配，以创造从重复和精确导向的任务发展成为智能和有用的同事的机器人。适当的人机交互是最重要的领域。因此，JARVIS的目标是开发一套可重复使用的工具，使人工智能驱动的多模式交互手段：a）涉及物理和远程信息交换，机器人控制和编程的接口，B）为各种机器人提供社交技能，以实现无缝的以用户为中心的交互，扩展人类完成复杂任务的能力，以及c）展示应用程序的可扩展性和实现规模经济的能力。

To strengthen the competitiveness of the EU industry, there is an increasing need for flexible production, exploiting the capabilities of both the machinery and human workforce. The growing market of robots has reacted, adapting to the changing situation by providing collaborative robotic solutions with a variety of characteristicsin terms of payload, type and mobility. A number of technologicalsolutions for perception and safety allow their coexistence with humans. While several research activities have been conducted over the past 5 years on this topic, wider industrial adoption is lagging due to the following restrictions: • Limited cognition and intelligence • Low performance of collaborative operations • Collaboration fluidity is rather low as the operators have to adapt to the particularities of the robots. • Complexity in robot programming which requires the involvement of skilled engineers, does not provide flexibility in the execution phase, and does not benefit from the tacit knowledge of experienced operators. Thus, despite the fact the mechatronics are quite advanced, a perfect matching of mechanics and control is required to create robots that advance from repetitive and precision oriented tasks to becoming intelligent and helpful co-workers. Proper human robot interaction is the most important ground to be covered. Thus, JARVIS aims to develop a reusable set of tools that enable AI driven multimodal means of interaction: a) involving interfaces for physical and remote information exchange, robot control and programming, b) providing social skills to a variety of robots to achieve seamless user-centric interaction that extends human ability for complex tasks and c) demonstrating scalability of application and ability to achieve economies at scale.