GARDE: Trackable Interactive Multimodal Manipulatives: Towards a Tangible Learning Environment for the Blind

PI：Pontelli，E. &Cook，J.提案编号：1401639标题：GARDE：可跟踪的交互式多模态操作：为盲人提供一个灵活的学习环境& 这项工作的目标是视障学生从事数学课程，从基本的数字感，基本代数和图形和图表建模。这一组传统上是通过使用操纵器来教授的，操纵器是有形的物体，可以用来创建代数，几何，测量和科学中重要概念的心理模型或“图像”。随着触摸和手指操作的可用性，视力受损者的数学学习得到了改善。然而，最先进的技术仍然缺乏一种设备，可以独立探索，与同行合作（两者都被证明可以增强学习），以及对一系列数学概念和相关领域科学的普遍适用性。我们提出了开发，TINUS（可跟踪的交互式多模态操纵器）。TIBRATION平台独立于盲文，能够与同行合作，并提供多模式和实时反馈，以促进对抽象数学概念的独立探索和深入学习。该平台由智能活动块（包括各种传感器和反馈设备）沿着整体多点触摸跟踪基础设施组成。TIMM平台包括访问其功能的中间件和API，以促进交互式应用程序的开发。技术描述 众所周知，美国在技术方面的领先地位目前正受到我们教育系统培养的科学家和工程师数量减少的威胁，这是一个全国性的问题。许多研究表明，进入大学的学生缺乏数学准备会影响他们攻读STEM学位的决定，并与他们在STEM领域成功完成学位相关。虽然在2007年，大约33%的八年级学生在数学素养方面达到或超过熟练水平，但同期只有7%的视力残疾学生达到这一水平。统计数据证实，视障人士在STEM职业中的代表性显着不足，无论是在工业界还是学术界。 虽然技术在“传统”课堂环境中的数学和科学教学中无处不在，但同样的技术并没有帮助视力残疾的学生。 为传统课堂教学推荐的课程往往依赖于游戏和基于计算机的解决方案，这些解决方案非常直观，盲人学生很难接触到。与此同时，为视障学生推荐的数学课程仍然主要基于繁琐和原始的手工操作（钉板、磁性盲文砖、盲文块）。 这项工作的智力价值在于拟议基础设施的独特特征，以及将通过TIM的可用性实现的视障人士教学研究。与最先进的解决方案不同，TIMENTOS通过音频和触觉反馈实现了物理的多模态操作，结合了物理和数字世界的优点，通过实时支持和辅导来促进抽象学习。通过简单的可编程接口，TIMENTOS可用于教授一系列可应用于多个科学领域的数学概念（大多数解决方案侧重于单个概念或问题）。 由于所提出的TIRECT设备的通用性，受这项工作影响的潜在用户社区非常大。21.5 2010年有100万美国人报告视力丧失;新墨西哥州报告视力残疾的人口比例在美国排名第六。此外，研究人员还与新墨西哥州的几个项目（学生无障碍服务、新墨西哥州盲人和视障者学校、数学辅助技术实验室）建立了关系，这些项目将为至少1,000名受这项工作影响的视障学生提供服务。

PI: Pontelli, E. & Cook, J.Proposal Number: 1401639Title: GARDE: Trackable Interactive Multimodal Manipulatives: Towards a Tangible Learning Environment for the BlindBroader Significance & Importance This work targets visually impaired students engaged in math courses ranging from basic number sense to basic algebra and graph and chart modeling. This group is traditionally taught through the use of manipulatives, which are tangible objects that can be used to create mental models or "images" of important concepts in algebra, geometry, measurements, and science. With the availability of touch and finger manipulatives, learning math for the visually impaired has improved. However, the state-of-the-art still lacks a device that enables independent exploration, collaboration with peers (both proven to enhance learning), and general applicability to a range of mathematical concepts and related domain sciences. We propose the development of, TIMMs (Trackable Interactive Multi-modal Manipulatives). The TIMMs platform is independent of Braille, enabling collaboration with peers, and provides multimodal and real-time feedback to promote independent exploration and deeper learning of abstract mathematical concepts. The platform is composed of intelligent active blocks (that include a variety of sensors and feedback devices) along with an overall multi-touch tracking infrastructure. The TIMM platform includes middle-ware to access its features and APIs to facilitate the development of interactive applications.Technical Description It is well-known, and a national concern, that the future of the United States' lead in technology is presently threatened by the decreasing number of scientists and engineers that our educational system is producing. Much research indicates that the lack of math preparation of students entering college weighs into their decision to pursue a STEM degree and correlates to their successful degree completion in STEM fields. Although in 2007, about 33% of 8th grade students achieved at or above proficient rank in math literacy, for the same period only 7% of students with visual disabilities achieved such rank. The statistics confirm that visually impaired individuals are significantly underrepresented in STEM careers, both in industry as well as academia. While technology has become ubiquitous in the teaching of mathematics and science in "traditional" classroom setting - the same technology has not come to the aid of students with visual disabilities. Recommended curricula for traditional classroom teaching often rely on games and computer-based solutions that are highly visual and highly inaccessible to blind students. At the same time, the recommended math curricula for visually impaired students are still predominantly based on cumbersome and primitive manual manipulatives (peg-boards, magnetic Braille tiles, Braille blocks). The intellectual merit of this work lies in the unique characteristics of the proposed infrastructure, and the research in teaching and learning of the visually impaired that will be enabled through the availability of TIMMs. Unlike state-of-the-art solutions, TIMMs implements physical, multi-modal manipulatives with audio and haptic feedback, combining the best of both the physical and the digital worlds to promote abstract learning with real-time support and tutoring. Through simple, programmable interfaces, TIMMs can be used to teach a range of mathematical concepts (most solutions focus on a single concept or problem) that can be applied to several scientific domains. Because of the generality of the proposed TIMMs device, the potential user community impacted by this work is very large. 21.5 million Americans reported vision loss in 2010; New Mexico has the 6th highest percentage of population reporting visual disabilities in the US. Furthermore, the investigators have established relationships with several programs (Student Accessibility Services, New Mexico School for the Blind and Visually Impaired, Math Adoptive Technology Lab) within New Mexico that serve at least 1,000 visually impaired students that will be impacted by this work.