Camera-based Text Recognition from Complex Backgrounds for the Blind or Visually

今天，美国有超过1000万盲人和视力受损者。计算机视觉、数码相机和便携式计算机方面的最新技术发展使得有可能通过开发基于相机的 将联合收割机计算机视觉技术与其他现有产品相结合。 尽管许多阅读助手是专门为盲人或视力受损的人设计的，但是从复杂背景或非平面背景中阅读文本， 表面处理是非常具有挑战性的，并且尚未成功地解决。许多日常任务都涉及这些具有挑战性的条件，例如自动售货机上的阅读说明，书架上排列的书籍标题，药瓶上的说明或汤罐上的标签。 该建议的重点是开发新的计算机视觉算法来识别复杂背景中的文本：1）从具有多种不同颜色的背景（例如排列在书架上的书名）和2）从非平坦表面（例如药瓶或汤罐上的标签）。新开发的计算机视觉技术将与现成的光学字符识别（OCR）和语音合成软件产品相结合。视觉信息将通过头戴式 通过便携式计算机（PDA或手机）分析，而语音显示将通过迷你扬声器、耳机或蓝牙设备输出。一个实用的阅读系统原型将产生从复杂的背景和非平面阅读文本。该系统将具有成本效益，因为它只需要一个头戴式摄像头（100万分辨率<100美元），一台可穿戴计算机（<300美元）和两个迷你扬声器或耳机。“ReadIRlS”[74] OCR软件的价格低于150美元，“TextAloud”语音合成软件约为30美元[75]。 该项目将在纽约城市学院和纽约灯塔国际学院执行两年。CCNY位于纽约市的哈莱姆社区，被指定为少数民族机构和西班牙裔服务机构（37%西班牙裔和27%非洲裔美国人）。Lighthouse International是一家领先的非营利组织，致力于保护视力，并提供急需的视力和康复服务，以帮助所有年龄段的人克服视力丧失的挑战。在这两年内，我们将1）开发新的算法，从多种不同颜色的背景中识别文本; 2）开发新的算法，从非平面上识别文本;以及3）通过整合现成的光学字符识别（OCR）和语音合成软件产品，为盲人用户开发一个具有成本效益的原型阅读系统。原型和算法的有效性将由视力正常的人和视力受损的人进行评估。将创建复杂背景（多种颜色和非平面）上的文本数据库，用于算法和系统评估。该数据库将提供给计算机视觉和视力康复科学领域的研究团体。总之，这项工作将提供一个基于研究的基础，为下一代盲人阅读助手的设计提供信息，并产生一个实用的原型，以帮助盲人用户在现实世界的环境中从复杂的背景中阅读文本。

There are more than 10 million blind and visually impaired people living in America today. Recent technology developments in computer vision, digital cameras, and portable computers make it possible to assist these individuals by developing camera-based products that combine computer vision technology with other existing products. Although a number of reading assistants have been designed specifically for people who are blind or visually impaired, reading text from complex backgrounds or non-flat surfaces is very challenging and has not yet been successfully addressed. Many everyday tasks involve these challenging conditions, such as reading instructions on vending machines, titles of books aligned on a shelf, instructions on medicine bottles or labels on soup cans. This proposal focuses on the development of new computer vision algorithms to recognize text from complex backgrounds: 1) from backgrounds with multiple different colors (e.g .. the titles of books lined up on a shelf) and 2) from non-flat surfaces (e.g .. labels on medicine bottles or soup cans). The newly developed computer vision techniques will be integrated with off-the-shelf optical character recognition (OCR) and speech-synthesis software products. Visual information will be captured via a head-mounted camera (on sunglasses or hat) and analyzed by a portable computer (PDA or cell phone), while the speech display will be outputted via mini speakers, earphones, or Bluetooth device. A practical reading system prototype will be produced to read text from complex backgrounds and non-flat surfaces. The system will be cost-effective since it requires only a head mounted camera (<US$100 for 1M resolution), a wearable computer (<US$300), and two mini-speakers or earphones. The price of "ReadIRlS" [74] OCR software is under $150 and the "TextAloud" speech synthesis software is about $30 [75]. This project will be executed over two years at the City College of New York (CCNY) and Lighthouse International, New York. CCNY, located in the Harlem neighborhood of New York City, is designated as both a Minority Institution and a Hispanic-serving Institution (37% Hispanic and 27% African American). Lighthouse International is a leading non-profit organization dedicated to preserving vision and to providing critically needed vision and rehabilitation services to help people of all ages overcome the challenges of vision loss. During the two years, we will 1) develop new algorithms to recognize text from backgrounds with multiple different colors; 2) develop new algorithms to recognize text from non-flat surfaces; and 3) develop a cost-effective prototype reading system for blind users by integrating with off-the-shelf optical character recognition (OCR) and speech-synthesis software products. The effectiveness of the prototype and algorithms will be evaluated by people with normal vision and people with vision impairment. A database of text on complex backgrounds (multiple colors and non-flat surfaces) will be created for algorithm and system evaluation. The database will be made available to research communities in the areas of computer vision and vision rehabilitation science. In summary, this effort will provide a research-based foundation to inform the design of next generation reading assistants for blind persons, as well as produce a practical prototype to help the blind user read text from complex backgrounds in real-world environments.