Talkback: Voice Control of Software for the Creative Industries

对讲：创意产业软件的语音控制

• 批准号: 73371
• 金额: $ 6.06万
• 依托单位: DBS MUSIC HOLDINGS LIMITED
• 依托单位国家: 英国
• 项目类别: Study
• 财政年份: 2020
• 资助国家: 英国
• 起止时间: 2020 至 无数据
• 项目状态: 已结题
• 来源: https://gtr.ukri.org/projects?ref=73371
• 关键词: Talkback; Voice; Control; Software; Creative

We will combine cutting edge Artificial Intelligence and Natural Language Processing with the industry standard method of digital control in music production, MIDI (Musical Instrument Digital Interface). For 40 years, MIDI has been the standard computer 'language' for passing instructions between digital music hardware/software such as "Play Note Middle C". It can also send commands to DAWs, to add a track or change the values of controls. Since all DAWs communicate with MIDI, Talkback can be implemented with whichever DAW a user is most familiar.Thus, with Talkback, a visually impaired user could be playing their guitar and simply ask the software to "Create a Guitar track with my favourite settings, and start recording, now."In addition, because of its widespread adoption and simplicity, using MIDI means that similar non-musical applications could potentially access the same opportunities for voice control, like film editing and graphics software.Talkback is a project to improve accessibility and workflow in the creative industries by providing voice command functionality to the specialised digital music software millions of professional and educational users employ. It also addresses important accessibility concerns with this software which at present relies exclusively on screen-based interaction and the attendant visual and mobility requirements of operating keyboard and mouse.Music production employs specialised Digital Audio Workstation (DAW) software which combines music composition/generation and audio recording/mixing into one screen-based system. A project comprises separate 'tracks' for instruments or sounds, with modern techniques requiring upwards of 20 and sometimes as many as 200 tracks. Each track may be a microphone recording or a digital instrument generated within the software, but each have many different controls, thus many thousands of separate parameters must be correctly set within the entire project.DAWs use drop-down menus and graphical controls which are highly visually reliant so without seeing the screen they are virtually impossible to use. Even commercially available physical control surfaces require excellent physical mobility and fine motor control so support for people with visual and/or mobility impairments is highly limited. Even non-disabled users cannot operate these systems while playing their instrument since doing so precludes using the mouse.Talkback returns the act of musical creation to its origins in human communication, by placing DAW controls back into the domain of naturally spoken requests. When playing in a band we simply say "the guitars should be louder": DAW control should operate the same way.

我们将把尖端的人工智能和自然语言处理与音乐制作中数字控制的行业标准方法MIDI(乐器数字接口)结合起来。40年来，MIDI一直是标准的计算机语言，用于在数字音乐硬件/软件之间传递指令，例如“Play Note MidC”。它还可以向DAWS发送命令，以添加轨迹或更改控件的值。因为所有DAW都与MIDI通信，所以Talkback可以用用户最熟悉的DAW来实现。因此，使用Talkback，视障用户可以在弹吉他时，简单地要求软件“用我最喜欢的设置创建吉他曲目，现在就开始录制”。此外，由于它的广泛采用和简单，使用MIDI意味着类似的非音乐应用程序可能会获得同样的语音控制机会，比如电影编辑和图形软件。Talkback是一个项目，通过向数百万专业和教育用户使用的专业数字音乐软件提供语音命令功能，来改善创意行业的可访问性和工作流程。它还解决了该软件的重要可访问性问题，该软件目前完全依赖于基于屏幕的交互以及随之而来的操作键盘和鼠标的视觉和移动性要求。音乐制作使用专门的数字音频工作站(DAW)软件，该软件将音乐创作/生成和音频记录/混音结合到一个基于屏幕的系统中。一个项目包括乐器或声音的单独“轨道”，现代技术需要20多个轨道，有时甚至多达200个轨道。每首曲目可以是麦克风录音或软件中生成的数字乐器，但每首曲目都有许多不同的控件，因此必须在整个项目中正确设置数千个单独的参数。DAW使用视觉上高度依赖的下拉菜单和图形控件，因此在看不到屏幕的情况下，他们几乎不可能使用。即使是商用的物理操纵面也需要良好的物理机动性和精细的运动控制，因此对视力和/或行动不便的人的支持非常有限。即使非残障用户也不能在演奏乐器时操作这些系统，因为这样做排除了使用鼠标的可能性。Talkback通过将Daw控件放回自然发声的请求域中，将音乐创作行为返回到人类交流的起源。当我们在乐队里演奏时，我们简单地说“吉他应该更响亮”：DAW CONTROL应该以同样的方式操作。