Surgical data science for intelligent guidance and control in image-guided and robotic interventions

外科数据科学的研究旨在通过利用来自不同数据源(例如，医学成像、传感器)的信息来提高介入性医疗保健的质量。通过分析这些数据集，我们将能够模拟最优的手术任务执行并设计基于技能的控制器，从而提高手术机器人的自主性，从而以最小的侵入性和创伤实现更有效的手术。该项目将研究基于模型和基于学习的混合控制方法，以提高手术机器人的机动性和灵活性。基于模型的视觉方法将确保满足安全约束，而基于学习的视觉方法将在手术过程中导航各种不确定性，并提高手术任务执行的性能。还将探索基于深度学习的视觉方法，用于使用手术摄像机定位和绘制环境，提供术中图像指导和增强临床决策。具体目标是：通过融合不同的数据集来建模最佳手术执行将这些模型转化为手术机器人的控制策略，从而提高自动化结合基于模型和基于学习的方法，以确保安全约束得到满足实施和评估术中指导/辅助系统，以提高手术性能从多模式数据(例如，图像、手术视频、触觉和位置传感器)分析和建立模型可以提高手术和机器人干预的效率。为了模拟最佳的手术任务执行，首先，需要使用来自非监督学习的工具对手术过程中的不同动作进行分段。然后，可以通过融合上述异类数据集来对每个动作进行建模。所得到的模型可用于在外科训练和模拟中开发新的解决方案，并通过将其转化为控制策略(例如，技能学习)来提高机器人的自主性。然而，机器人有效执行相同技能的能力既取决于从技能演示收集的数据的质量，也取决于建模方法本身。为了提高机器人的自主学习和仿真能力，人们提出了强化学习和深度学习两种方法来代替传统的建模方法。然而，这些强大的基于学习的方法需要与基于模型的控制相结合，以确保安全运行和高性能的同时。EPSRC的主要研究领域，如医学成像、人工智能技术、机器人学和控制工程是该项目的核心技术领域。该项目与EPSRC目前的研究主题组合很好地结合在一起，特别是与人工智能、机器人和医疗技术。它还通过关注数据驱动的方法和实时分析来解决EPSRC Healthcare Technologies在“物理干预的前沿”和“优化治疗”方面的两大挑战，以实现手术机器人的新功能(自主、基于性能的指导)。项目成果提供了通过更准确和安全的程序、更多的手术机会和个性化治疗来改善介入结果的广泛医疗影响的机会，同时降低了医疗成本和患者恢复时间。在项目期间，预计将出现手术机器人行业项目成果的整合和进一步发展的机会。受益者包括商业外科机器人系统的制造商以及新型介入性机器人平台的开发商。

Research in surgical data science aims at improving the quality of interventional healthcare by leveraging information from heterogeneous data sources (e.g., medical imaging, sensors). By analysing these datasets, we will be able to model optimal surgical task execution and design skill-based controllers that will increase the autonomy of surgical robots resulting in more efficient surgeries with minimum invasiveness and trauma caused. This project will investigate hybrid control approaches incorporating both model-based and learning-based ones to provide increased manoeuvrability and dexterity in surgical robots. The model-based ones will ensure that safety constraints are satisfied, while the learning-based ones will navigate the various uncertainties during the procedure and increase the performance of the surgical task execution.Vision methods based on Deep Learning, would also be explored for localization and mapping of the environment using surgical cameras, offering intra-operative image guidance and enhancing clinical decision-making.The specific objectives are to:Model optimal surgical execution by fusing heterogeneous datasetsTranslate these models into control policies for surgical robots leading to increased automationCombine model-based and learning-based approaches to ensure that safety constraints are satisfiedImplement and evaluate intra-operative guidance/assistance systems to enhance surgical performanceAnalysing and building models from multimodal data (e.g., images, surgical videos, haptic and position sensors) can enhance the efficiency of surgical and robotic interventions. To model optimal surgical task execution, first, segmentation of the different actions during the procedure needs to be done, using tools from unsupervised learning. Then, each action can be modelled by fusing the aforementioned heterogeneous datasets. The resulting models can be used to develop novel solutions in surgical training and simulation, and also to increase robot autonomy by translating them into control policies (e.g., skill learning). However, the ability of the robot to perform the same skill effectively depends both on the quality of the data collected from skill demonstration, and the modelling approach itself. To improve the autonomous learning and imitation ability of robots, Reinforcement and Deep Learning are proposed, instead of traditional modelling methods. However, these powerful learning-based approaches need be combined with model-based control to ensure safe operation and high performance at the same time. Key EPSRC's research areas such as Medical Imaging, Artificial Intelligence Technologies, Robotics and Control Engineering are the project's core technical areas. The project is well-aligned with the current portfolio of EPSRC's research themes and specifically with Artificial Intelligence and Robotics and Healthcare Technologies. It also addresses two EPSRC Healthcare Technologies Grand Challenges in "Frontiers of Physical Intervention" and "Optimising Treatment" by focusing on data-driven methods and real-time analytics to realise new capabilities (autonomy, performance-based guidance) in surgical robotics. Project outcomes present opportunities for broad healthcare impact towards improving interventional outcomes with more accurate and safe procedures, increased access to surgery and personalising treatment, while reducing healthcare costs and patient recovery times.Opportunity for integration and further development of project outcomes in the surgical robotics industry is expected to arise during the project duration. Beneficiaries include manufacturers of commercial surgical robotics systems as well as developers of novel interventional robotic platforms.