Measuring Theme Success: QA approaches to user-led NLP training

衡量主题成功：用户主导的 NLP 培训的 QA 方法

• 批准号: 10060709
• 金额: $ 1.93万
• 依托单位: NERDS WITH WORDS LTD
• 依托单位国家: 英国
• 项目类别: Collaborative R&D
• 财政年份: 2023
• 资助国家: 英国
• 起止时间: 2023 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=10060709
• 关键词: Measuring; Theme; Success; QA; approaches

Wordnerds is a SaaS platform offering a unique approach to text analytics, used by large organisations to derive value from their Voice of Customer programmes. We have attracted high profile customers across the retail, transport and housing sectors through our USP of 'context themes': allowing users to train their own AI-based categories on meaning rather than keywords.As a start-up with a small team we've stayed at the cutting edge of text analytics by taking an agile approach to product development - delivering value quickly and moving on to the next feature to stay ahead of customer needs. Our context themes are unique and effective but we feel now is the time to refine the user experience of training them, to make them easier to use, measure and prove.Working with the experts at TUV SUD National Engineering Laboratory (known as NEL) we will be undertaking research and development to:* Provide externally-validated benchmark metrics for the Wordnerds categorisation technology (including the number of samples required to reach an acceptable accuracy level, so that users know when 'training' is complete; and any variation across different types of categorisation - e.g. emotional, motivational, or topical categories).* Assess the potential for alternative data science approaches to enhance our categorisation process, e.g. SeTFT architecture or an ensemble model approach, by building and testing proof of concept software.* Explore ways of illustrating category accuracy in a visually impactful way.We hope to validate and implement a combination of measures that will markedly improve the user experience and confidence of our customers. This will in turn lead to more effective use of insights from our platform, and greater return on investment.

Wordnerds是一个SaaS平台，提供了一种独特的文本分析方法，被大型组织用来从他们的客户之声计划中获得价值。我们通过“情境主题”的USP吸引了零售、运输和住房行业的高端客户：允许用户根据含义而不是关键字来训练自己的基于AI的类别。作为一个拥有小团队的初创公司，我们通过采用敏捷的产品开发方法，一直处于文本分析的最前沿-快速提供价值，并转移到下一个功能，以领先于客户需求。我们的上下文主题是独特而有效的，但我们认为现在是时候改进培训它们的用户体验，使它们更容易使用，测量和证明。与TUV SUD国家工程实验室的专家合作（称为NEL），我们将进行研究和开发，以：* 为Wordnerds分类技术提供外部验证的基准指标（包括达到可接受的准确度水平所需的样本数量，以便用户知道“训练”何时完成;以及不同类型的分类之间的任何变化-例如情感，动机或主题类别）。通过构建和测试概念验证软件，评估替代数据科学方法的潜力，以增强我们的分类流程，例如SeTFT架构或集成模型方法。*探索如何以视觉上有影响力的方式说明类别准确性。我们希望验证并实施一系列措施，以显著改善用户体验和客户信心。这反过来将导致更有效地利用我们平台的见解，并获得更高的投资回报。