Data Driven Techniques for 3D Reconstruction, Motion Generation and Authoring Interactive Spaces in Media Arts

40年来，我的长期研究目标一直是发展强大而有效的三维建模、动画和交互计算技术。尽管在这一领域取得了巨大的进展，但实现这一目标仍面临许多科学挑战。2002年加入Concordia，我的第一个发现基金是研究从感知数据处理大型3D模型的有效计算技术。从那时起，我所有的研究活动，通过各种联邦和省级计划资助上述研究的核心。现在流行的术语数据驱动是对上述研究计划的最佳描述。它是建立在使用多模态数据（图像，视频，深度，运动），其中大部分可以便宜地捕获这些天。具体来说，我的研究目标在这个建议是开发新的计算技术，准确和强大的3D重建虚拟对象/世界，动画和3D交互，并开发新的计算机视觉/图形为基础的技术，增强公共空间的动态视觉效果。* 本研究将寻求以下问题的答案：1）需要哪些新技术来最好地使用多模态数据来重建复杂人造物体和场景（室内，城市地区，交通结构）的形状？2)我们如何将联合收割机程序解决方案技术与机器/深度学习和机器推理相结合，以更好地分析和合成3D形状和运动？3)我们如何创造动态的视觉效果，创造性地提高戏剧表演？* 4)什么样的编程模型和工具可以使艺术家创作交互式媒体空间？* 我的研究方法基本上是由研究，分析，新想法，实施，测试和验证的循环驱动的。具体来说，我计划研究解决方案方法（分析，机器学习和机器推理）的组合来解决3D图形中的问题。我将与我的3D图形组的学生进行持续的研究，该小组拥有多模态数据采集所需的基础设施和行业合作，并继续与同事和艺术学者成功合作。这项研究的主要目标是使产品开发人员能够轻松地生成具有逼真外观和动画的3D模型，而无需过多的用户输入，编程或人工努力。该项目将支持平均每年3名博士和2名硕士学生在视觉数据计算的前沿研究和技术方面的培训。* 这项研究为数据驱动的3D图形开辟了新的问题和研究途径。基础研究成果将使图形研究界受益，所产生的工具将使加拿大的工业、艺术家、娱乐和虚拟仿真部门受益。

The development of robust and efficient computational techniques for modelling, animation and interaction in three dimensions (3D) has been my long term research objective for 4 decades now. In spite of vast developments in this field, achieving this objective continues to pose many scientific challenges. Joining Concordia in 2002, my first discovery grant was on investigation of efficient computational techniques for processing large 3D models from sensed data. Since then, all my research activities, funded through various federal and provincial programs have the above research at their core. The now popular term data-driven is what describes the above research program best. It is built on the use of multi-modal data (image, video, depth, motion), most of which can be inexpensively captured these days. Specifically my research objective in this proposal is to develop new computational techniques for accurate and robust 3D reconstruction of virtual objects/worlds, with animations and 3D interactions, and to develop new computer vision/graphics based techniques for augmenting public spaces with dynamic visual effects. ***This research will seek answers to some of the following questions: 1) What are new techniques needed to best use multi-modal data to reconstruct shapes of complex man-made objects and scenes (interiors, urban areas, transportation structures)? 2) How can we combine procedural solution techniques with machine/deep learning and machine inference to better inform analysis and synthesis of 3D shape and motion? 3) How do we create dynamic visual effects which creatively enhance theatre performances?***4) What programming models and tools can empower artists in authoring of interactive media spaces? ***My research methodology is essentially driven by the cycle of research, analysis, new ideas, implement, test and verify. Specifically, I plan to investigate a combination of solution approaches (analytical, machine learning and machine inference) to solve problems in 3D graphics. I will build upon ongoing research with students in my 3D graphics group which has the required infrastructure and industry collaborations for multi-modal data acquisition, and continue the successful collaboration with colleagues and with arts scholars. The principal goal of this research is to make it easy for product developers to produce 3D models with realistic appearance and animations for novel situations without requiring excessive user input, programming or human effort. The project will support training on an average 3 PhD and 2 MSc students per year in leading edge research and technologies of visual data computing. ***This research opens up new problems and research avenues in data-driven 3D graphics. Fundamental research results will benefit the graphics research community and the resulting tools will benefit industry, artists, and the entertainment and virtual simulation sectors in Canada.**