CAREER: A Multimodal Mixed-Initiative Research Notebook for Information Discovery

虽然搜索引擎解决了帮助人们找到特定信息的问题，但它们并不是直接为支持人们开发创造性想法而构建的。这项研究支持人们从事持续的信息发现任务，如发明和论文开发，其目标是通过在发现和收集信息的同时形成想法来创造新知识。将开发新的人机交互模式，以支持人们利用信息语义、动态组织和可视化来组装他们的发明积木。这种方法是基于组成的形式，它在视觉上和概念上整合元素来表示一个集合。研究笔记本将支持持续创新，人们需要在他们创建的集合中连接和整合关键发现，同时随着时间的推移进行研究和发明。基于物理的技术，如质量弹簧和植绒模型，将被应用和扩展，以促进人类操纵的集合作为组成。新的互动方式，如基于笔的互动，将通过使参与者能够真正接触数字信息来改善感官反馈和参与。基于物理的技术和新的交互方式将与combinFormation混合主动组合平台集成，为增强桌面隐喻之外的研究笔记本提供基础。 我们的主要假设是，混合主动组成和物理为基础的多模态交互将提高个人和团队为基础的信息发现任务的新想法的出现。每年有1,000多名本科生在设计过程课程中开发新发明，使用研究笔记本进行作业，将促进创新教育，同时也产生信息发现的评估数据。拟议的研究在跨学科方法上具有创新性，预计将产生重大成果：（1）基于物理的信息交互新技术;（2）用于与信息交互的新的多模态技术;（3）用于支持人类发现思想之间的联系的新的集成语义、视觉和交互方法;（4）用于支持学生和研究人员与数字信息的持续交互的研究笔记本基础设施;（5）关于学生和研究人员如何在实践中随着时间的推移发展想法的新数据;（6）对构成研究和发明的创造性过程的新理解;（7）支持教育中的研究和发明的新方法。本提案开发的高级学习技术将通过研究笔记本和combinFormation平台实施和分发。软件技术将通过网络提供给学生和公众。多模式交互实验室将使集成的软件/硬件系统提供给一些学生。研究笔记本的方法、技术和网络基础设施可以提高科学家的生产力。用于发现和语义的新交互技术可以改变学生、研究人员、知识工作者和公众体验互联网和数字存储库的方式。以学习者为中心的方法，将研究问题，设计和实施融入教育，将有助于培养新一代以创造力为导向的以人为本的计算研究人员和实践者。

While search engines address the problem of helping people find particular information, they are not directly built to support people developing creative ideas. This research supports people engaged in sustained information discovery tasks, such as invention and thesis development, in which the goal is to create new knowledge by forming ideas while finding and assembling information. New modalities of human computer interaction will be developed to support people engaged in assembling their building blocks of invention, utilizing information semantics, dynamic organization, and visualization. The approach is based on the form of composition, which visually and conceptually integrates elements to represent a collection. The research notebook will support sustained innovation, in which people need to connect and integrate key findings in the collections they create while conducting research and invention over time. Physically-based techniques, such as mass-spring and flocking models, will be applied and extended to facilitate human manipulation of collections as compositions. New modalities of interaction, such as pen-based, will improve sensory feedback and engagement by enabling participants to literally get their hands into digital information. The physically-based techniques and new interaction modalities will be integrated with the combinFormation mixed-initiative composition platform to provide a basis for augmenting the research notebook beyond the desktop metaphor. Our principal hypothesis is that mixed-initiative composition and physically-based multimodal interaction will improve the emergence of new ideas during individual and team-based information discovery tasks. Using the research notebook on assignments by over 1,000 undergraduate students developing new inventions each year in The Design Process course will catalyze innovative education, while also generating evaluation data on information discovery.The proposed research is innovative in its interdisciplinary approach, expected to yield significant outcomes: (1) new techniques for physically-based interaction with information; (2) new multimodal techniques for interaction with information; (3) new integrated semantic, visual, and interactive methods for supporting human discovery of connections among ideas; (4) a research notebook infrastructure for supporting sustained interaction with digital information by students and researchers; (5) new data about how students and researchers develop ideas over time in practice; (6) new understanding of creative processes that make up research and invention; and (7) new methods for supporting research and invention in education. Advanced learning technologies developed by this proposal will be implemented and distributed through the Research Notebook and the combinFormation platform. Software technologies will be made available to the students and the public through the web. A multimodal interaction lab will make integrated software/hardware systems available to some of the students. The Research Notebook methods, techniques, and cyberinfrastructure can amplify the productivity of scientists. The new interactive techniques for discovery and semantics can transform how students, researchers, knowledge workers, and the public experience the internet and digital repositories. The integration into education of research issues, designs, and implementations, with a learner-centered approach, will contribute to the development of a new generation of creativity-oriented human-centered computing researchers and practitioners.