Formalizing and Automating Technology Maturity Assessment

你会使用40年前的方法或工具来推进一个尖端的工程项目吗？你会在不了解方法结果的普遍性的情况下，使用另一个组织、不同领域开发的方法吗？目前技术成熟度评估的最佳实践是技术准备水平（TRL）量表，它是由美国宇航局在20世纪70年代开发的。今天，TRL评估被用于制造、能源和运输行业的组织以及世界各地的主要拨款机构和监管机构做出数百万美元的决策。

Would you apply a 40-year old method or tool to advance a cutting-edge engineering project? Would you use a method developed at another organization, in a different field, without an understanding of the generalizability of the method's results? Current best practice in technology maturity assessment is the Technology Readiness Level (TRL) scale, which was developed by NASA in the 1970s. Today, TRL assessments are used to make multi-million-dollar decisions at organizations in the manufacturing, energy and transportation industries, and at major grant agencies and regulatory bodies worldwide. The TRL is not sufficiently validated in established fields of application, and is little studied in new fields where technology-intensive start-ups are blossoming. The TRLs rely on expert-assessments, which have been shown to be highly subjective. My recent progress has established the limitations of current best practices in such assessments, serving as a foundation for the proposed work. With increasing access to data and automated classification techniques, an important opportunity exists to improve the reliability of technology maturity assessments. My long-term vision is to revolutionize the technology maturity assessment process, and I therefore propose a research program to formalize and automate this process, with three short-term objectives: 1. Establish a formal understanding of inter-human agreement on TRL assessment in engineering-intensive organizations. Design an experimental method to collect TRL assessments, to formally establish a measure of expert disagreement in various industry contexts. 2. Generate an automated classification model for TRL. Collect and analyze publicly available data sets of TRL assessments, including one from NASA, as well as scientific publications, patents, and venture funding. Build a conceptual model of technology maturity to inform the development of a machine-learning technology-maturity classifier. 3. Develop and validate a novel Technology Maturity Assessment Analytics Tool for decision support. Incorporate expert and data-driven knowledge to assess technology maturity. Embody this tool in a publicly accessible online portal such that ongoing data collection can be conducted, stimulating future research and sparking future collaboration. The novel tool proposed for technology maturity assessment will boost innovation through better-informed decisions at organizations that involve: engineering-intensive development, venture capital investment, and the granting of public funds. The application of state-of-the-art machine learning techniques will influence the broad field of systems engineering decision-making to re-think critical processes, many of which currently rely solely on experts. HQP, including undergraduate summer researchers and graduate students, will gain skills not only in technical engineering design but also data analytics and behavioural studies, currently in critical shortage in Canada.