A Framework for Evaluating Diagnostic Imaging Artificial Intelligence Medical Tools

本项目的目标和目的1)设计、评估人工智能的工作方法，并将其引入现有的放射学工作流程2)临床工作人员的设计和临床评估工具包3)在设计过程中使用的提高产品信任度的良好做法4)采购前/部署评估工具包5)购买人工智能系统的良好做法指南6)采购后/部署工具包7)将人工智能系统部署到现有工作流程参与框架的良好做法指南针对学院：1)涉及NIAW作者的研究论文2)评估、服务实施和教育的增强工具包实际：至少两个研讨会和联合设计站点，C&V和CTM UHBs中心是几个放射科医生，但这可以扩大，这取决于我们讨论的参与。其他网站可能希望参与，这将是我们讨论的联合参与练习(其他健康委员会和信托)。可以考虑其他放射学报告人员的参与，并将加强NHS X项目的参与(报告放射技师、放射学实习生)-这将是有益的，可以讨论参与和/或合作。在第一年和第二年，学生将需要与不同的利益相关者合作，组织联合设计研讨会和其他形式的启发研究，为规划在这些框架中需要考虑的内容提供信息。这将需要将重点放在具体的案例研究上，然后以产生的方式开展工作，以确定可在其中建立共同框架的共性。所采用的具体方法将由学生与指导团队和博士合作伙伴共同决定，以回应学生的理解和他们确定的特定信息需求。作为该项目的一部分，已经提出了一些建议(例如，人种志、访谈、讲习班、设计小说)，很可能这些都是适当的。在第二年，学生应该能够介绍框架草案的多次迭代，同时继续使用以人为中心的方法来唤起利益相关者的反思和指导，以进一步发展他们的框架，以及制定如何使用它的指导。这项工作可以采取互动共同设计会议的形式，在那里框架草案被付诸实践(例如，利用学生开发的现有技术或原型)。通过这一点，开发的框架将得到发展。除了为其实际应用制作框架和指导外--这在英国目前是缺乏的--学生还将制作学术出版物，记录他们的研究实践并出版。该项目的可交付成果将对利益攸关方具有重大价值，该项目的学术和实践成果有可能影响威尔士内外许多医疗保健领域的采购实践。

The project intended aims and Objectives1) A working methodology for the design, evaluation & introduction of AI into existing Radiology workflow2) Design & clinical evaluation toolkit for clinical staff3) Good practice to be used during design that promotes trust in the product4) Pre-procurement/deployment evaluation toolkit5) Good practice guide to purchasing AI systems6) Post procurement/deployment toolkit7) Good practice guide to deploying an AI system into an existing workflow - engagement frameworkFor the Academy:1) Research papers involving NIAW based authors2) Enhanced toolkit for evaluation, service implementation and educationPractically:Minimum of two sites for workshop & co-design, C&V and CTM UHBs, centred on a couple of Radiologists but this can expand depending upon the engagement as we discussed.Other sites may wish to be involved and that would be a joint engagement exercise as we discussed (other Health Boards and Trusts). Other Groups of Radiology Reporter involvement could be considered and will enhance the project (Reporting Radiographers, Radiology Trainees) NHS X involvement - this would be beneficial, and involvement and/ or collaboration could be discussed.The PhD timelineIn years 1 and 2, the student will need to work with the various stakeholders to organise co-design workshops and other forms of elicitation studies to inform planning for what needsto be considered in such frameworks. This will require focusing on specific case-studies, and then working in a generative fashion to identify commonalities in which a common framework can be built upon. The specific methodologies employed will be decided by the student in partnership with the supervision team and PhD partner, responding to the student's understanding and the specific information needs they identify. A number have been proposed as part of this project (e.g., ethnographies, interviews, workshops, design fictions) and it is likely a mixture of these will be appropriate. During the second year, the student should be able to present multiple iterations of draft frameworks, while continuing to employ human-centred approaches to evoking reflections and guidance from stakeholders to further develop their framework, as well as developing guidance for how it can be used. This work may take the form of interactive co-design sessions where the draft framework is put into practice (e.g. with existing technologies or prototypes developed by the student). Through this the developed framework will evolve. PhD contributions In addition to producing the framework and guidance for its practical use-something that is lacking in the UK currently-the student will produce academic publications documenting their research practice for publication. The deliverables of the project will be of significant value for the stakeholder, and the academic and practical outcomes of the project hold the potential to shape procurement practice in many healthcare contexts in and outside of Wales.