SBIR Phase I: Virtual Music helper, next-generation educational social platform for conservatory music practice and performance empowered by AI and AR

这个SBIR第一阶段项目的重点是为儿童开发一个虚拟音乐助手(VMH)的原型。研究证实，音乐和音乐学习可以提高儿童的智力、学习能力和智商。即使是患有注意力缺陷/多动障碍的儿童，也能从数学考试中事先听音乐中受益。学习或练习音乐的孩子在STEM科目上表现出更好的成绩。然而，研究也表明，孩子们因为错误地认为他们缺乏音乐能力而选择退出音乐课，许多孩子经常发现音乐练习没有吸引力，而且很无聊，最终失去了兴趣并退出了。父母发现要引诱孩子坐下来练习很有挑战性，或者他们无法帮助孩子，因为他们要么没有时间，要么自己不懂音乐来帮助他们。虚拟音乐助手能读懂孩子们的音乐吗？音乐作业，指导他们完成练习，并立即批改。这个帮手不是代替老师，而是在家里帮助家长，鼓励孩子多练习。从更广泛的角度来看，这个项目可能会推动重大研究，寻找有效的方法来辅导数学或语言等音乐以外的科目的孩子，或者为有特殊需求的孩子提供辅导。这个SBIR第一阶段项目提出了一个由人工智能和增强现实为孩子们提供支持的虚拟音乐助手(VMH)。VMH是一个3D移动虚拟人(化身)，可以为每个孩子进行个性化，并从视觉和行为角度与孩子进行现场对话。VHM为教师提供有关每个学生的进步和弱点的智能内容。该项目将解决的技术挑战是检测和纠正孩子们犯下的错误，包括从原声乐器中检测复调音调，以高精度和高速度将笔记转换为机器可读的音乐格式，赋予化身决策算法以提供适当的口头和视觉纠正和反馈。该平台还旨在支持多种流行工具和多种语言。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This SBIR Phase I project focuses on development of a prototype for a Virtual Music Helper (VMH) for children. Research confirms that music and music learning enhances intelligence, learning and IQ specially in kids. Even children with attention deficit/ hyperactivity disorder benefit from listening to music beforehand in mathematics tests. Children who learn or practice music have shown better results in STEM subjects. However, studies also suggest children opt out of music class based on false belief that they lack musical ability and many kids often find the music practice non-engaging, and boring eventually losing interest and quitting. Parents find it challenging to entice the kids to sit down and practice or they cannot help their kids because either they don?t have time or are not music savvy themselves to help them. The virtual music helper can read the kids? music homework, guide them through the exercise and provide immediate correction. This helper is not to replace teachers, but help the parents at home and encourage kids to practice more. In broader terms this project may spur significant research in finding effective methods for tutoring kids in subjects other than music such as math or language, or for kids with special needs.This SBIR Phase I project proposes a Virtual Music Helper (VMH) that is empowered by Artificial Intelligence and Augmented Reality for kids. VMH is a 3D mobile-virtual-human (avatar) that can be personalized for each kid and conduct live dialog with kids both from visual and behavioral perspectives. VHM offers smart content for teachers about each student's progress and weak points. The technical challenges this project will address is the detection and correction of mistakes kids make including polyphonic pitch detection from acoustic instrument, conversion of note sheets to machine readable music format with high accuracy and speed, empowering the avatar with decision making algorithms to provide proper correction and feedback verbally and visually. This platform is also intended to support multiple popular instruments and multiple languages.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.