III: Small: Combining Algorithms for Recognition and Retrieval of Mathematics

三：小：数学识别与检索的组合算法

• 批准号: 1016815
• 负责人: Richard Zanibbi
• 金额: $ 39.25万
• 依托单位: Rochester Institute of Tech
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-09-01 至 2015-03-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1016815&HistoricalAwards=false
• 关键词: III; Small; Combining; Algorithms; Recognition

Currently there is a significant need for increasing mathematical literacy in the United States. Among other benefits, this would permit more Americans to pursue careers in STEM (Science, Technology, Engineering and Mathematics) disciplines. The most extensive resource for finding introductory information on math, the internet, primarily supports text-based search. Similarly, current Portable Document Format (.pdf) viewers support search for text, but not math. The goal of this research is to develop a query-by-expression mechanism, where users enter expressions using images, stylus/finger, mouse and keyboard. Users may then search using a combination of the expression appearance, symbols, structure, and mathematical semantics. Expression properties may be combined with text-based search, allowing queries for mathematical information to be more precise than current text-based methods.For query-by-expression methods to be viable, improvements in math recognition are needed, along with the development of efficient methods for indexing and retrieving mathematical expressions. In particular, the project seeks to improve optical character recognition (OCR) for handwritten and typeset mathematics, along with methods for parsing expression structure. Approach based on Graph Transformer Networks (GTN) and adaptations of boosting techniques is applied to intelligent combination of modules that locate, recognize and relate mathematical symbols with an aim to further improve recognition. Research focuses on identifying appropriate features, distance metrics, indexing and search methods for expression retrieval. Developed techniques are evaluated via user studies both in-lab settings and through the internet. Additional user studies to identify appropriate use cases for query-by-expression are also planned.This project is expected to produce new query-by-expression methods usable by both math experts and (perhaps more importantly) non-experts. These methods might be adapted to retrieving other non-textual document elements such as chemical diagrams, tables, and figures. Source code and experimental data developed for the project will be made public via the project web site (http://www.cs.rit.edu/~dprl/msearch.html). To promote mathematical literacy, the principal investigator and graduate students working on the project will visit middle schools and talk about the history, recognition and retrieval of mathematical notation. The PI also plans to participate in the McNair Scholars program at RIT, which seeks to provide research experiences to low-income, first-generation college students that are interested in pursuing doctoral studies.

目前，美国迫切需要提高数学素养。除其他好处外，这将使更多的美国人能够从事 STEM（科学、技术、工程和数学）学科的职业。 互联网是查找数学入门信息最广泛的资源，主要支持基于文本的搜索。 同样，当前的可移植文档格式 (.pdf) 查看器支持文本搜索，但不支持数学搜索。 这项研究的目标是开发一种按表达式查询的机制，用户可以使用图像、手写笔/手指、鼠标和键盘输入表达式。然后，用户可以使用表达外观、符号、结构和数学语义的组合进行搜索。 表达式属性可以与基于文本的搜索相结合，使数学信息的查询比当前基于文本的方法更加精确。为了使按表达式查询方法可行，需要改进数学识别，以及开发用于索引和检索数学表达式的有效方法。特别是，该项目旨在改进手写和排版数学的光学字符识别（OCR），以及解析表达式结构的方法。 基于图变换网络（GTN）和增强技术的改进的方法被应用于定位、识别和关联数学符号的模块的智能组合，以进一步提高识别能力。 研究重点是确定用于表达检索的适当特征、距离度量、索引和搜索方法。开发的技术通过实验室环境和互联网的用户研究进行评估。还计划进行其他用户研究，以确定按表达式查询的适当用例。该项目预计将产生可供数学专家和（也许更重要的是）非专家使用的新的按表达式查询方法。这些方法可能适用于检索其他非文本文档元素，例如化学图表、表格和图形。该项目开发的源代码和实验数据将通过项目网站（http://www.cs.rit.edu/~dprl/msearch.html）公开。为了提高数学素养，该项目的主要研究者和研究生将访问中学，谈论数学符号的历史、识别和检索。该 PI 还计划参加 RIT 的麦克奈尔学者计划，该计划旨在为有兴趣攻读博士学位的低收入第一代大学生提供研究经验。