Towards Evolvable and Sustainable Multimodal Machine Learning

机器学习通常限于单一的操作模式。要同时实现图像、声音和语言理解，需要机器重用知识，并从多模态数据中理解概念。该项目旨在构建一个稀疏模型，并提出一套创新算法，以增强模型泛化，解决分布和语义变化，并最大限度地减少训练多模式系统的计算和标记成本。其成果将使模型的可演化学习能够在部署后适应不同的测试场景，同时减少能源消耗和碳排放。这些技术的应用可使电子商务、农业和运输等部门受益。

Machine learning is commonly limited to a single operational modality. To enable image, sound and language comprehension simultaneously would require machines to reuse knowledge and understand concepts from multimodal data. The project aims to build a sparse model and present a set of innovative algorithms to enhance model generalisation for addressing distributional and semantic shifts and minimise the computational and labelling costs for training multimodal systems. Its outcomes will enable evolvable learning of models to suit varying testing scenarios after deployment and whilst reducing energy consumption and carbon emission. The application of these techniques could benefit sectors such as E-commerce, agriculture and transport.