CAREER: Exploring Creative Expression through Music and Audio Technology

职业：通过音乐和音频技术探索创意表达

• 批准号: 0644151
• 负责人: Youngmoo Kim
• 金额: $ 49.99万
• 依托单位: Drexel University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-05-01 至 2013-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0644151&HistoricalAwards=false
• 关键词: CAREER; Exploring; Creative; Expression; through

Music contributes substantially to every culture on Earth, and the enjoyment of music is nearly universal. The appeal of music is that it provides expression to our own emotions, and it follows logically that individuals would want some creative input to that expression. Historically the primary exposure to music has been through live performance, providing audiences an opportunity for interaction with the musicians and music, but today the vast majority of music is experienced through recordings. And though recent digital audio technologies have had a tremendous impact on the world of recorded music, its fundamental nature remains unchanged: once a recording is made, that single "performance" is forever fixed, preventing any true interaction with the listener. The proposed work integrates research in digital audio technology with educational activities under a common vision of transforming the act of listening to "recorded" music into an interactive experience in which the "performance" responds to the creative input of the listener. Central to this vision is a concept known as Structured Audio, a semantic object-based representation of sound. The specific work activity consists of the following efforts: . Structured Sound Modeling (SSM) of musical instruments, which uses a novel signal processing and machine learning framework to facilitate a greater degree of musically expressive control than can be achieved with existing models for sound synthesis. . Development of the Structured Audio Platform (SAP), consisting of devices and software that employ these new instrument models to provide an expanded artistic palette for music producers and interfaces enabling nonmusicians to interact with their music by controlling creative aspects of the "performance." An outreach program that uses music technology to attract students to science, technology, engineering, and mathematics (STEM) by demonstrating the contributions of these disciplines to modern music production as well as the creativity inherent within STEM and related fields. With respect to intellectual merit, there are many methods of modeling musical instrument sounds, but most are limited in their ability to capture the full expressive range of those instruments. The project will employ a novel framework for modeling time-varying sounds using jointly estimated parameters of an excitation-resonance model to derive a HiddenMarkovModel (HMM) of expressive sound. This framework employs an analysis/synthesis scheme: input sound is parameterized by the model and can be regenerated from the model parameters. This process has been used to model the singing voice and can be applied to other instruments, resulting in high sound quality while retaining a full range of musical expression. Knowledge gained through this project will also benefit other machine listening applications, such as audio identification and sound sensing. The research activity includes the development of a custom platform of devices and interfaces that offer artists new creative options and allow non-musicians to also provide creative input to their music. A key area of research will investigate appropriate mappings from the expressive controls of SSM instruments to a simple, low-dimensional interface for musical novices. Creativity is a difficult subject to study quantitatively, but this project presents a framework from which a small subset of creative expression can be assessed and analyzed. The educational activities of this project are designed to broadly impact students at many levels. The proposed outreach program will introduce high school students, particularly from underrepresented groups, to the technology of digital music, revealing the influence of science and engineering on music performance, recording, and listening. By engaging students' affinity for popular music the program seeks to foster curiosity in science and technology. The research component of the proposal will also be directly integrated into this program via custom music devices that allow students to creatively interact with music in Structured Audio format. This confluence of engineering and art has the potential to incubate interest in both fields. Given the diminishing support for music education in the public school system, this multi-disciplinary approach may be a solution to sustaining creative arts in the curriculum. The proposed research integrates knowledge from a wide range of areas. Participating graduate students will receive interdisciplinary training in fields such as acoustics, music theory, signal processing, performance practice, and machine learning.

音乐对地球上的每一种文化都有很大贡献，对音乐的享受几乎是普遍的。音乐的吸引力在于它为我们自己的情感提供了表达，从逻辑上说，每个人都想要一些创造性的投入到这种表达中。在历史上，接触音乐的主要方式是现场表演，为观众提供了与音乐家和音乐互动的机会，但今天，绝大多数音乐都是通过录音体验的。尽管最近的数字音频技术对唱片音乐世界产生了巨大的影响，但它的基本性质仍然没有改变：一旦录制好，那一首《演奏》永远是固定的，阻止了与听众的任何真正互动。拟议的工作将数字音频技术研究与教育活动结合起来，在一个共同的愿景下，将收听“录制的”音乐的行为转变为一种互动体验，在这种互动体验中，“表演”对听者的创造性投入做出反应。这一愿景的核心是一个称为结构化音频的概念，它是声音的基于语义对象的表示。具体工作活动包括以下努力：乐器的结构化声音建模(SSM)，它使用一种新的信号处理和机器学习框架来促进比现有声音合成模型更大程度的音乐表现力控制。。开发结构化音频平台(SAP)，由采用这些新乐器模型的设备和软件组成，为音乐制作人提供扩展的艺术调色板，并提供接口，使非音乐家能够通过控制“表演”的创造性方面与他们的音乐互动。通过展示科学、技术、工程和数学(STEM)学科对现代音乐制作的贡献以及STEM和相关领域固有的创造力，利用音乐技术吸引学生学习这些学科的外展计划。关于智力的价值，有许多方法来模拟乐器的声音，但大多数方法都局限于捕捉这些乐器的全部表达范围的能力。该项目将采用一种新的框架来建模时变声音，使用激励-共振模型的联合估计参数来推导出表达声音的HiddenMarkovModel(HMM)。该框架采用分析/合成方案：输入声音由模型参数化，并可由模型参数重新生成。这一过程已被用于模拟歌唱的声音，并可应用于其他乐器，从而在保留全方位音乐表达的同时获得高音质。通过该项目获得的知识也将有益于其他机器收听应用，如音频识别和声音传感。研究活动包括开发一个定制的设备和界面平台，为艺术家提供新的创意选择，并允许非音乐家也为他们的音乐提供创意投入。一个关键的研究领域将研究从SSM乐器的表达控制到音乐新手的简单、低维界面的适当映射。创造力是一个很难定量研究的主题，但这个项目提供了一个框架，可以从这个框架中评估和分析创造性表达的一小部分。该项目的教育活动旨在广泛影响多个层次的学生。拟议的推广计划将向高中生，特别是来自代表人数不足的群体的高中生介绍数字音乐技术，揭示科学和工程对音乐表演、录音和收听的影响。通过激发学生对流行音乐的亲和力，该项目试图培养学生对科学和技术的好奇心。该提案的研究部分还将通过允许学生创造性地与结构化音频格式的音乐互动的定制音乐设备直接整合到该项目中。工程和艺术的这种融合有可能培养人们对这两个领域的兴趣。鉴于公立学校系统对音乐教育的支持正在减少，这种多学科的方法可能是在课程中保持创造性艺术的一个解决方案。拟议的研究整合了来自广泛领域的知识。参与培训的研究生将接受声学、音乐理论、信号处理、表演实践和机器学习等领域的跨学科培训。