Identifying User Requirements for Digitally Enabling Off-Site Fabrication with Mixed Reality and Gamification

在非现场制造中，民用和建筑基础设施组件设计的定制性质导致了复杂的配置，这些配置很难完全由自动化设备制造。劳动力大量参与大多数任务，使用的任务和流程与传统的现场施工非常相似。这就导致了在非现场生产这些组件所带来的最低生产率效益。混合现实系统的出现，如HoloLens 2及其与Trimble的硬帽加软件的集成，创造了一个机会，通过彻底重新思考工人的任务和流程来解决这个问题，从而实现大量的生产力和减少浪费的好处。项目目标是利用混合现实系统产生的点云、图像和系统定位数据，在构建位置上自动注册虚拟组件，确定最佳构建顺序，引导工人完成构建，并进行实时维度控制，对构建质量进行自我认证。其愿景是建立一个与工人互动的网络物理系统。博士项目将期望学生：a)开发一种自动注册方法，用于将设计模型注册到工作面上，允许手动调整；B)开发一种接近实时的组件捕获方法，利用设计模型的存在来创建对象的初始假设，并在存在杂波和空间误差的情况下将真实世界捕获的几何形状与设计模型相匹配；C)利用生成对抗网络从设计模型中学习装配过程；d)为装配案例研究开发游戏化+检验原型，以有效指导工人。作为起点，MRes组件将通过与FIBE2合作伙伴协商，包括广泛的最终用户需求分析，以映射用例的关键活动及其约束，以及优化构建序列的初步尝试。项目的预期结果是a)装配注册、组件捕获、游戏化和检查的方法；B)概念验证原型；C)出版物和会议报告；d)开发计划。这项研究将首次颠覆基于模型的装配、实时指导和渐进式自动化检测的现场制造。它还将生产一个工作原型。

In off-site manufacturing, the bespoke nature of components' design for civil and building infrastructure leads to complex configurations that are difficult to manufacture purely with automation equipment. Labour is heavily involved in most tasks, using tasks and processes very similar to traditional, on site construction. This leads to minimal productivity benefits from producing such components off site. The advent of mixed reality systems, such as HoloLens 2 and its integration with a hardhat plus software by Trimble, creates an opportunity to address this issue off site by completely rethinking the worker tasks and processes to achieve substantial productivity and waste reduction benefits. The project objectives are to leverage point cloud, image and system positioning data produced by mixed reality systems to automatically register virtual components on the build location, determine the optimal build sequence, guide the worker to complete the build, and perform real time dimensional control to self-certify the quality of the build. The vision is to build a cyber-physical system that interacts with the worker.The PhD project will expect from the student to: a) develop an automatic registration method for registering the design model to the work surface, allowing for manual adjustments; b) develop a near real time component capture method, leveraging the presence of the design model to create an initial hypothesis of the objects and matching the real world captured geometry with the design model in the presence of clutter and spatial errors; c) exploit generative adversarial networks to learn the assembly process from the design model alone; and d) develop a gamification + inspection prototype for an assembly case study for effective worker guidance. As a starting point, the MRes component will involve an extensive end-user requirements analysis through consultations with the FIBE2 partners to map the key activities of the use case and their constraints, along with an initial attempt at optimising build sequences.The expected outcomes of the project are a) Methods for assembly registration, component capture, gamification and inspection; b) a proof-of-concept prototype; c) publications and presentations at conferences; and d) an exploitation plan. This research will be the first to disrupt off site manufacturing with model-based assembly, real time guidance and progressive, automated inspection. It will also produce a working prototype.