Formalizing and Automating Technology Maturity Assessment

你会使用40年前的方法或工具来推进一个尖端的工程项目吗？你会在不了解方法结果的普遍性的情况下，使用另一个组织、不同领域开发的方法吗？目前技术成熟度评估的最佳实践是技术准备水平（TRL）量表，它是由美国宇航局在20世纪70年代开发的。今天，TRL评估被用于制造、能源和运输行业的组织以及世界各地的主要拨款机构和监管机构做出数百万美元的决策。TRL在已建立的应用领域没有得到充分的验证，在技术密集型初创企业蓬勃发展的新领域也很少得到研究。trl依赖于专家评估，而专家评估已被证明是高度主观的。我最近的进展已经确定了这种评估中当前最佳做法的局限性，作为拟议工作的基础。随着对数据的访问和自动分类技术的增加，存在着提高技术成熟度评估可靠性的重要机会。我的长期愿景是彻底改变技术成熟度评估过程，因此我提出了一个研究计划来形式化和自动化这个过程，有三个短期目标：1。建立对工程密集型组织中TRL评估的人际协议的正式理解。设计一个收集TRL评估的实验方法，以正式建立不同行业背景下专家分歧的衡量标准。2.生成TRL的自动分类模型。收集和分析TRL评估的公开可用数据集，包括来自NASA的数据集，以及科学出版物、专利和风险基金。建立技术成熟度的概念模型，为机器学习技术成熟度分类器的开发提供信息。3.为决策支持开发和验证一个新的技术成熟度评估分析工具。结合专家和数据驱动的知识来评估技术成熟度。将此工具包含在可公开访问的在线门户网站中，以便进行持续的数据收集，从而刺激未来的研究并激发未来的合作。为技术成熟度评估提出的新工具将通过在涉及：工程密集型开发、风险资本投资和公共资金授予的组织中做出更明智的决策来促进创新。最先进的机器学习技术的应用将影响系统工程决策的广泛领域，以重新思考关键过程，其中许多目前完全依赖于专家。HQP包括本科生暑期研究员和研究生，他们不仅将获得技术工程设计方面的技能，还将获得数据分析和行为研究方面的技能，这些技能目前在加拿大严重短缺。

Would you apply a 40-year old method or tool to advance a cutting-edge engineering project? Would you use a method developed at another organization, in a different field, without an understanding of the generalizability of the method's results? Current best practice in technology maturity assessment is the Technology Readiness Level (TRL) scale, which was developed by NASA in the 1970s. Today, TRL assessments are used to make multi-million-dollar decisions at organizations in the manufacturing, energy and transportation industries, and at major grant agencies and regulatory bodies worldwide. The TRL is not sufficiently validated in established fields of application, and is little studied in new fields where technology-intensive start-ups are blossoming. The TRLs rely on expert-assessments, which have been shown to be highly subjective. My recent progress has established the limitations of current best practices in such assessments, serving as a foundation for the proposed work. With increasing access to data and automated classification techniques, an important opportunity exists to improve the reliability of technology maturity assessments. My long-term vision is to revolutionize the technology maturity assessment process, and I therefore propose a research program to formalize and automate this process, with three short-term objectives: 1.Establish a formal understanding of inter-human agreement on TRL assessment in engineering-intensive organizations. Design an experimental method to collect TRL assessments, to formally establish a measure of expert disagreement in various industry contexts. 2.Generate an automated classification model for TRL. Collect and analyze publicly available data sets of TRL assessments, including one from NASA, as well as scientific publications, patents, and venture funding. Build a conceptual model of technology maturity to inform the development of a machine-learning technology-maturity classifier. 3.Develop and validate a novel Technology Maturity Assessment Analytics Tool for decision support. Incorporate expert and data-driven knowledge to assess technology maturity. Embody this tool in a publicly accessible online portal such that ongoing data collection can be conducted, stimulating future research and sparking future collaboration. The novel tool proposed for technology maturity assessment will boost innovation through better-informed decisions at organizations that involve: engineering-intensive development, venture capital investment, and the granting of public funds. The application of state-of-the-art machine learning techniques will influence the broad field of systems engineering decision-making to re-think critical processes, many of which currently rely solely on experts. HQP, including undergraduate summer researchers and graduate students, will gain skills not only in technical engineering design but also data analytics and behavioural studies, currently in critical shortage in Canada.