Recognition of Handwritten Words in School Essays Using Conditional Random Fields

本研究是关于阅读理解测试中儿童手写反应中的单词再认问题。单词识别的方法将基于条件随机场(CRF)，这是一种区分方法，不对底层数据做出任何假设，因此在序列标记问题上优于隐马尔可夫模式(HMM)。首先，使用现有的基于神经网络的算法将学生的回答分割成单词图像。然后，将每个单词图像过度分割成多个小片段，使得片段的组合形成字符图像。分段被标记为具有从CRF模型估计的概率的字符。使用评估片段的最佳组合的动态规划算法来计算表示来自词典的词条的单词图像的总概率。从阅读文章、测试提示、答案和学生回答中提取的词典用于限制探索路径的数量，并根据笔迹样本估计CRF模型的状态和转换参数。状态参数对应于如下特征：位置(按长度归一化)、地点(在开始、中间或结尾)、高度、宽度、到原型的距离、高度偏差等。过渡参数对应于特征诸如：字符对的标签(TH、ER、QU等)、垂直重叠(候选字符图像的像素)、高度差、宽度差、纵横比差、双元组宽度等。研究测试台将包括对纽约布法罗一所市中心学校8年级和5年级的阅读理解提示的手写评分回答。共有300个8年级的回答和200个5年级的回答，每个回答大约有100-150个单词。参数估计的训练数据最初将由150名学生回答和1000篇成年人的半页文章组成。随着研究的进展，这些将得到额外的学校数据的补充。面向目标的复杂文档图像分析、自然语言处理和机器学习的集成将推动改进的手写识别方法。儿童？S笔迹识别在文件分析中从未被研究过。到目前为止，复杂文档的手写识别技术在很大程度上还不可用。成功将使全州范围内的测试能够在本学年晚些时候进行，结果将更早提供，从而对改善教育产生影响。

This research concerns recognition of words in handwritten responses of children in reading comprehension tests. The approach to word recognition will be based on conditional random fields (CRFs), which are discriminative methods that do not make any assumptions about the underlying data and hence are known to be superior to Hidden Markov Modes (HMMs) for sequence labeling problems.The student response is first segmented into word images using an existing neural network based algorithm. Each word image is then over- segmented into a number of small segments such that the combination of segments forms character images. Segments are labeled as characters with probability evaluated from the CRF model. The total probability of a word image representing an entry from the lexicon is computed using a dynamic programming algorithm which evaluates the optimal combination of segments. A lexicon derived from the reading passage, testing prompt, answer rubric and student responses is used to limit the number of paths to explore.The state and transition parameters of the CRF model are estimated from handwriting samples. State parameters correspond to features such as: position (normalized by length), place (in start, middle or end), height, width, distances to prototype, deviations of height, etc. Transition parameters correspond to features such as: label of character pair (th, er, qu, etc), vertical overlap (of pixels of candidate character images), height difference, width difference, aspect ratio difference, bigram width, etc.The research test-bed will consist of scored handwritten responses to reading comprehension prompts from Grades 8 and 5 of an inner city school in Buffalo, New York. There are 300 Grade 8 responses and 200 Grade 5 responses, with about 100-150 words in each response. Training data for parameter estimation will initially consist of 150 student responses and 1,000 half-page writings of adults. These will be supplemented with additional school data as research progresses.Goal-oriented integration of complex document image analysis, natural language processing and machine learning will drive improved handwriting recognition methods. Children?s handwriting recognition has never before been studied in document analysis. Handwriting recognition technology for complex documents is as yet largely unavailable. Success will allow statewide testing to be done later in the school year with results provided sooner thereby having an impact on improved education.