Space-Compacting Magnification Augmented with Natural Gestures and Keyboardless Text Entry for Low Vision Smartphone Interaction

通过自然手势和无键盘文本输入增强空间压缩放大倍率，实现低视力智能手机交互

• 批准号: 9886627
• 负责人: I.V. Ramakrishnan
• 金额: $ 40.82万
• 依托单位: STATE UNIVERSITY NEW YORK STONY BROOK
• 依托单位国家: 美国
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-03-01 至 2023-02-28
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9886627
• 关键词: Accelerometer; Acute; Address; Affect; Age; Algorithms; Android; Area; Awareness; Cellular Phone; Complex; Controlled Study; Cues; Dependence; Development; Devices; Disabled Persons; Disadvantaged; Elderly; Elements; Employment Opportunities; Eye; Fingers; Friends; General Population; Gestures; Hand; In Situ; Information Technology; Learning; Manuals; Microscope; Motion; Outcome; Plant Roots; Population; Productivity; Psyche structure; Reader; Research; Self-Help Devices; Services; Sum; Surface; Taxes; Techniques; Technology; Telephone; Text; Touch sensation; Travel; Use Effectiveness; Vision; Vision Disorders; Visual; Visual impairment; barometric pressure; base; disability; experience; falls; graphical user interface; next generation; novel; operation; preservation; pressure sensor; recruit; sensor; synergism; usability; virtual; vision aid

Project Summary For people with low-vision impairments, the smartphone has become inextricably tied to their daily lives just like the general population. It is their go-to assistive device, relying on the smartphone’s built-in screen magnifier - Zoom on the iPhone and Magnifier on Android - to interact with it. But the usability of these screen magnifiers falls woefully short, adversely affecting productivity. First, the magnifier indiscriminately magnifies the raw screen pixels, including whitespace, as a blanket operation, causing occlusion of important contextual information such as visual cues from the user’s viewport. This necessitates panning over these occluded portions and mentally reconstructing the contextual information necessary for interacting with the content elements. Second, magnification gestures such as the bimanual multi-tap and multi-finger touch gestures tend to be more complex than the basic 1-finger swipe. The complexity of these touch gestures makes them cumbersome and tiring to use. Remembering the entire repertoire of gestures is also difficult. Since all these touch gestures involve some subset of finger combinations, it is easy to mix up one for the other. Third, virtual keyboards, which takes up significant screen real estate as-is, is difficult to use for text entry and editing in magnified view. Either the entire screen area is magnified including the virtual keyboard or only the display area. In the former, some of the keys are occluded from the view whereas in the latter the keys remain unmagnified. Regardless, key presses are hard to do in either cases. In sum, all these usability issues contribute to a vastly disproportionate gap in user experience and productivity between people with and without low-vision impairments. This proposal seeks to develop the next generation screen magnifier that will bridge this wide gap in user experience. It is rooted on three novel ideas. First, instead of indiscriminately magnifying the screen content as is done now, it will do object-aware magnification by identifying the objects in the graphical interface and compacting the space between the objects so as to keep contextually related objects close together in the magnified view. Second, by leveraging the untapped built-in sensors such as accelerometer, geometric field and barometric pressure sensors, it will expand the default surface gestures to include surfaceless natural gestures for magnification operations that can be done with only one hand, thus freeing the other hand for other tasks. More importantly, these gestures will be easy-to-do and easy-to-learn and recall. Third, it will incorporate a novel keyboardless gesture-based text entry and editing technique to eliminate the difficulties that arise with virtual keyboards for text entry in magnification mode. These three ideas will inform the development of CxZoom, a transformative, next-generation smartphone screen reader for low vision. CxZoom will make interaction with smartphones far more usable for people with low vision, thereby eliminating any barriers to their productivity and empower them to utilize the power and connectivity of these devices to fully participate in this digitized economy.

项目摘要 对于低视力障碍的人来说，智能手机已经与他们的日常生活密不可分， 就像普通人一样。它是他们的首选辅助设备，依靠智能手机的内置屏幕 放大镜-iPhone上的放大镜和Android上的放大镜-与之交互。但这些屏幕的可用性 放大镜福尔斯的功能严重不足，对生产力产生了不利影响。第一，放大镜不分青红皂白地放大 原始屏幕像素，包括空白，作为覆盖操作，导致重要上下文的遮挡， 信息，例如来自用户视口的视觉提示。这需要在这些被遮挡的部分上进行平移 以及在精神上重构与内容元素交互所必需的上下文信息。 第二，诸如双手多次敲击和多手指触摸手势的放大手势倾向于更多地被使用。 比基本的单指滑动复杂。这些触摸手势的复杂性使得它们笨重， 使用起来很累。记住所有的手势也很困难。由于所有这些触摸手势都涉及到 手指组合的某个子集，很容易将一个与另一个混淆。第三，虚拟键盘， 上显着的屏幕真实的房地产是，很难用于文本输入和编辑放大的看法。要么 整个屏幕区域被放大，包括虚拟键盘或仅显示区域。在前一种情况下， 键从视图中被遮挡，而在后者中键保持未放大。无论如何，按键是 在任何情况下都很难做到。总之，所有这些可用性问题都导致了用户之间不成比例的差距。 有和没有低视力障碍的人之间的经验和生产力。 这项建议旨在开发下一代屏幕放大器，将弥合这一巨大的差距，在用户 体验.它基于三个新颖的理念。首先，不要不加选择地放大屏幕内容， 现在完成，它将通过识别图形界面中的对象来进行对象感知放大， 压缩对象之间的空间，以使上下文相关的对象在 放大视图。其次，通过利用未开发的内置传感器，如加速度计，几何场和 气压传感器，它将扩大默认的表面手势，包括表面自然 一种仅用一只手就能完成放大操作的手势，从而解放了另一只手 手做其他任务。更重要的是，这些手势将易于操作，易于学习和回忆。三是 将采用一种新颖的基于手势的无键盘文本输入和编辑技术， 在放大模式下用于文本输入的虚拟键盘出现的困难。这三个想法将告知 CxZoom的开发，这是一种变革性的下一代智能手机屏幕阅读器，用于低视力。 CxZoom将使智能手机的交互更适用于视力低下的人，从而消除 消除影响其工作效率的任何障碍，并使他们能够利用这些设备的功能和连接性， 全面参与数字化经济。