GARDE: Trackable Interactive Multimodal Manipulatives: Towards a Tangible Learning Environment for the Blind

PI：Ponelli，E.&amp；Cook，J.Proposal Numbers：1401639标题：Garde：Trackable Interactive多模式操纵器：为盲人打造有形的学习环境这项工作的目标是学习从基本数感到基本代数以及图形和图表建模等数学课程的视障学生。这一组传统上是通过使用操纵器来教授的，操纵器是有形的物体，可以用来创建心理模型或代数、几何、测量和科学中重要概念的“图像”。随着触摸和手指操作的普及，视障人士的数学学习能力有所提高。然而，最先进的设备仍然缺乏一种能够实现独立探索、与同行合作(都被证明可以增强学习)以及对一系列数学概念和相关领域科学的普遍适用性的设备。我们提出了TIMMS(Trackable Interactive多通道操纵器)的开发。TIMMS平台独立于盲文，支持与同行合作，并提供多模式和实时反馈，以促进对抽象数学概念的独立探索和更深入的学习。该平台由智能活动块(包括各种传感器和反馈设备)以及整体多点触摸跟踪基础设施组成。TIMM平台包括用于访问其功能的中间件和促进交互应用程序开发的API。技术描述众所周知，美国在技术方面的领先地位的未来正受到我们教育系统培养的科学家和工程师数量减少的威胁，这是众所周知的，也是一个全国性的问题。许多研究表明，进入大学的学生缺乏数学准备影响了他们攻读STEM学位的决定，并与他们在STEM领域成功完成学位相关。虽然在2007年，约33%的八年级学生数学能力达到或超过熟练水平，但在同一时期，只有7%的视力障碍学生达到了这一水平。统计数据证实，无论是在工业界还是学术界，视障人士在STEM职业生涯中的比例都明显偏低。虽然科技已经在“传统”课堂环境下的数学和科学教学中变得无处不在，但同样的科技并没有帮助视力残疾的学生。传统课堂教学的推荐课程通常依赖于游戏和基于计算机的解决方案，而这些游戏和基于计算机的解决方案非常直观，盲人学生很难接触到这些解决方案。与此同时，为视障学生推荐的数学课程仍然主要基于繁琐和原始的手工操作(钉板、磁性盲文瓷砖、盲文积木)。这项工作的学术价值在于拟议的基础设施的独特特点，以及将通过提供TIMM实现对视障人士的教学和学习的研究。与最先进的解决方案不同，TIMMS实现了具有音频和触觉反馈的物理、多模式操作，结合了物理和数字世界的最佳效果，通过实时支持和辅导来促进抽象学习。通过简单、可编程的界面，TIMM可用于教授可应用于多个科学领域的一系列数学概念(大多数解决方案侧重于单个概念或问题)。由于建议的TIMMS设备的通用性，这项工作影响的潜在用户群体非常大。2010年有2150万美国人报告视力丧失；新墨西哥州的视力障碍人口比例在美国排名第六。此外，调查人员还与新墨西哥州的几个项目(学生无障碍服务、新墨西哥州盲人和视障学校、数学采用技术实验室)建立了关系，这些项目至少为1,000名将受到这项工作影响的视障学生提供服务。

PI: Pontelli, E. & Cook, J.Proposal Number: 1401639Title: GARDE: Trackable Interactive Multimodal Manipulatives: Towards a Tangible Learning Environment for the BlindBroader Significance & Importance This work targets visually impaired students engaged in math courses ranging from basic number sense to basic algebra and graph and chart modeling. This group is traditionally taught through the use of manipulatives, which are tangible objects that can be used to create mental models or "images" of important concepts in algebra, geometry, measurements, and science. With the availability of touch and finger manipulatives, learning math for the visually impaired has improved. However, the state-of-the-art still lacks a device that enables independent exploration, collaboration with peers (both proven to enhance learning), and general applicability to a range of mathematical concepts and related domain sciences. We propose the development of, TIMMs (Trackable Interactive Multi-modal Manipulatives). The TIMMs platform is independent of Braille, enabling collaboration with peers, and provides multimodal and real-time feedback to promote independent exploration and deeper learning of abstract mathematical concepts. The platform is composed of intelligent active blocks (that include a variety of sensors and feedback devices) along with an overall multi-touch tracking infrastructure. The TIMM platform includes middle-ware to access its features and APIs to facilitate the development of interactive applications.Technical Description It is well-known, and a national concern, that the future of the United States' lead in technology is presently threatened by the decreasing number of scientists and engineers that our educational system is producing. Much research indicates that the lack of math preparation of students entering college weighs into their decision to pursue a STEM degree and correlates to their successful degree completion in STEM fields. Although in 2007, about 33% of 8th grade students achieved at or above proficient rank in math literacy, for the same period only 7% of students with visual disabilities achieved such rank. The statistics confirm that visually impaired individuals are significantly underrepresented in STEM careers, both in industry as well as academia. While technology has become ubiquitous in the teaching of mathematics and science in "traditional" classroom setting - the same technology has not come to the aid of students with visual disabilities. Recommended curricula for traditional classroom teaching often rely on games and computer-based solutions that are highly visual and highly inaccessible to blind students. At the same time, the recommended math curricula for visually impaired students are still predominantly based on cumbersome and primitive manual manipulatives (peg-boards, magnetic Braille tiles, Braille blocks). The intellectual merit of this work lies in the unique characteristics of the proposed infrastructure, and the research in teaching and learning of the visually impaired that will be enabled through the availability of TIMMs. Unlike state-of-the-art solutions, TIMMs implements physical, multi-modal manipulatives with audio and haptic feedback, combining the best of both the physical and the digital worlds to promote abstract learning with real-time support and tutoring. Through simple, programmable interfaces, TIMMs can be used to teach a range of mathematical concepts (most solutions focus on a single concept or problem) that can be applied to several scientific domains. Because of the generality of the proposed TIMMs device, the potential user community impacted by this work is very large. 21.5 million Americans reported vision loss in 2010; New Mexico has the 6th highest percentage of population reporting visual disabilities in the US. Furthermore, the investigators have established relationships with several programs (Student Accessibility Services, New Mexico School for the Blind and Visually Impaired, Math Adoptive Technology Lab) within New Mexico that serve at least 1,000 visually impaired students that will be impacted by this work.