Searching Documents with Text and Mathematical Content Using a Pen-Based Interface

使用基于笔的界面搜索包含文本和数学内容的文档

• 批准号: 539433-2019
• 负责人: Labahn, George
• 金额: $ 8.32万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Collaborative Research and Development Grants
• 财政年份: 2020
• 资助国家: 加拿大
• 起止时间: 2020-01-01 至 2021-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=699558
• 关键词: Searching; Documents; Text; Mathematical; Content

Scientific and engineering documents typically contain both text and mathematics. However, when it comes to searching such documents, current commercial technology is able to match text only and so is not well-suited for tasks such as finding mathematical expressions; this must be extended to become math-aware. In this proposal we aim to search text and mathematical content together in order to get the best search results. It has been shown that search engines become more effective when they are able to infer some semantics for terms appearing in documents since, in such instances, terms with similar meanings can be matched. As a result we wish to ensure that math-aware search can infer semantics for mathematical formulas. In this way retrieval based on both text and mathematical formulas can become more effective. In order for a math-aware search engine to have broad impact, it is important that a user be able to enter their mathematical formulas in a natural handwritten manner rather than to use one-dimensional LaTeX-like commands. This can be natural on today's pen-based devices and so an important part of our project is to investigate how a pen-based interface can be used to retrieve documents that are rich in mathematical content. Converting handwritten mathematical expressions to be used in a math-aware search system has considerable difficulties. Challenges include recognition of individual characters, combining handwritten characters into math expressions, distinguishing among similar looking expressions, use of gestures, and determining wild cards. We plan to investigate data-driven methods to ensure good recognition of handwritten math. The implementation and evaluation of our math-aware retrieval system plays a central role in our work.

科学和工程文档通常包含文本和数学。 然而，在搜索此类文档时，目前的商业技术只能匹配文本，因此不太适合查找数学表达式等任务；这必须延伸到具有数学意识。在这个提案中，我们的目标是同时搜索文本和数学内容，以获得最佳搜索结果。 事实证明，当搜索引擎能够推断文档中出现的术语的某些语义时，它们会变得更加有效，因为在这种情况下，可以匹配具有相似含义的术语。 因此，我们希望确保数学感知搜索能够推断数学公式的语义。这样基于文本和数学公式的检索可以变得更加有效。 为了使数学搜索引擎产生广泛的影响，用户能够以自然的手写方式输入数学公式，而不是使用一维的类似 LaTeX 的命令，这一点很重要。这在当今基于笔的设备上是很自然的，因此我们项目的一个重要部分是研究如何使用基于笔的界面来检索富含数学内容的文档。 将手写数学表达式转换为用于数学感知搜索系统具有相当大的困难。挑战包括识别单个字符、将手写字符组合成数学表达式、区分相似的表达式、手势的使用以及确定通配符。我们计划研究数据驱动的方法，以确保手写数学的良好识别。 我们的数学感知检索系统的实施和评估在我们的工作中发挥着核心作用。