GARDE: Trackable Interactive Multimodal Manipulatives: Towards a Tangible Learning Environment for the Blind

PI: Pontelli， E. & Cook, j .建议编号：1401639标题：GARDE：可追踪交互式多模态操作：为盲人提供一个有形的学习环境更广泛的意义和重要性本研究针对从事数学课程的视障学生，从基本的数感到基本代数和图形和图表建模。这个群体传统上是通过使用教具来教学的，教具是有形的物体，可以用来创建代数、几何、测量和科学中重要概念的心理模型或“图像”。随着触摸和手指操作的可用性，视觉障碍的数学学习得到了改善。然而，最先进的技术仍然缺乏一种能够独立探索、与同行合作（两者都被证明可以增强学习）以及对一系列数学概念和相关领域科学的普遍适用性的设备。我们建议发展timm（可跟踪交互式多模态操作器）。timm平台独立于盲文，支持与同行协作，并提供多模式和实时反馈，以促进抽象数学概念的独立探索和深入学习。该平台由智能活动模块（包括各种传感器和反馈设备）以及整体多点触控跟踪基础设施组成。TIMM平台包括用于访问其特性和api的中间件，以促进交互式应用程序的开发。众所周知，美国在技术上的领先地位目前正受到我们的教育系统所培养的科学家和工程师数量减少的威胁，这是一个全国性的问题。许多研究表明，学生进入大学时缺乏数学准备会影响他们决定攻读STEM学位，并与他们在STEM领域的成功完成学位相关。虽然在2007年，大约33%的八年级学生在数学素养上达到或超过熟练水平，但在同一时期，只有7%的视觉障碍学生达到了这一水平。统计数据证实，无论是在工业界还是学术界，视障人士在STEM职业中的代表性都明显不足。虽然技术在“传统”课堂的数学和科学教学中无处不在，但同样的技术并没有帮助有视觉障碍的学生。传统课堂教学的推荐课程通常依赖于游戏和基于计算机的解决方案，这些解决方案高度可视化，对盲人学生来说非常难以理解。与此同时，为视障学生推荐的数学课程仍然主要基于繁琐和原始的手工操作（钉板，磁性盲文砖，盲文块）。这项工作的智力价值在于拟议基础设施的独特特征，以及视障人士的教学和学习研究将通过timm的可用性而得以实现。与最先进的解决方案不同，timm实现了带有音频和触觉反馈的物理、多模态操作，结合了物理和数字世界的优点，通过实时支持和辅导促进抽象学习。通过简单的可编程接口，timm可以用来教授一系列可以应用于多个科学领域的数学概念（大多数解决方案都集中在单个概念或问题上）。由于所建议的timm设备的通用性，因此受此工作影响的潜在用户社区非常大。2010年有2150万美国人报告视力丧失；在美国，新墨西哥州报告视力障碍的人口比例排名第六。此外，研究人员已经与新墨西哥州的几个项目建立了关系（学生无障碍服务，新墨西哥盲人和视障学校，数学采用技术实验室），为至少1000名视障学生提供服务，这些学生将受到这项工作的影响。

PI: Pontelli, E. & Cook, J.Proposal Number: 1401639Title: GARDE: Trackable Interactive Multimodal Manipulatives: Towards a Tangible Learning Environment for the BlindBroader Significance & Importance This work targets visually impaired students engaged in math courses ranging from basic number sense to basic algebra and graph and chart modeling. This group is traditionally taught through the use of manipulatives, which are tangible objects that can be used to create mental models or "images" of important concepts in algebra, geometry, measurements, and science. With the availability of touch and finger manipulatives, learning math for the visually impaired has improved. However, the state-of-the-art still lacks a device that enables independent exploration, collaboration with peers (both proven to enhance learning), and general applicability to a range of mathematical concepts and related domain sciences. We propose the development of, TIMMs (Trackable Interactive Multi-modal Manipulatives). The TIMMs platform is independent of Braille, enabling collaboration with peers, and provides multimodal and real-time feedback to promote independent exploration and deeper learning of abstract mathematical concepts. The platform is composed of intelligent active blocks (that include a variety of sensors and feedback devices) along with an overall multi-touch tracking infrastructure. The TIMM platform includes middle-ware to access its features and APIs to facilitate the development of interactive applications.Technical Description It is well-known, and a national concern, that the future of the United States' lead in technology is presently threatened by the decreasing number of scientists and engineers that our educational system is producing. Much research indicates that the lack of math preparation of students entering college weighs into their decision to pursue a STEM degree and correlates to their successful degree completion in STEM fields. Although in 2007, about 33% of 8th grade students achieved at or above proficient rank in math literacy, for the same period only 7% of students with visual disabilities achieved such rank. The statistics confirm that visually impaired individuals are significantly underrepresented in STEM careers, both in industry as well as academia. While technology has become ubiquitous in the teaching of mathematics and science in "traditional" classroom setting - the same technology has not come to the aid of students with visual disabilities. Recommended curricula for traditional classroom teaching often rely on games and computer-based solutions that are highly visual and highly inaccessible to blind students. At the same time, the recommended math curricula for visually impaired students are still predominantly based on cumbersome and primitive manual manipulatives (peg-boards, magnetic Braille tiles, Braille blocks). The intellectual merit of this work lies in the unique characteristics of the proposed infrastructure, and the research in teaching and learning of the visually impaired that will be enabled through the availability of TIMMs. Unlike state-of-the-art solutions, TIMMs implements physical, multi-modal manipulatives with audio and haptic feedback, combining the best of both the physical and the digital worlds to promote abstract learning with real-time support and tutoring. Through simple, programmable interfaces, TIMMs can be used to teach a range of mathematical concepts (most solutions focus on a single concept or problem) that can be applied to several scientific domains. Because of the generality of the proposed TIMMs device, the potential user community impacted by this work is very large. 21.5 million Americans reported vision loss in 2010; New Mexico has the 6th highest percentage of population reporting visual disabilities in the US. Furthermore, the investigators have established relationships with several programs (Student Accessibility Services, New Mexico School for the Blind and Visually Impaired, Math Adoptive Technology Lab) within New Mexico that serve at least 1,000 visually impaired students that will be impacted by this work.