Revisit Multimodal Meta-Learning through the
Lens of Multi-Task Learning (NeurIPS 2021)
Milad Abdollahzadeh
Touba Malekzadeh
Ngai-Man Cheung


Multimodal meta-learning is a recent problem that extends conventional few-shot meta-learning by generalizing its setup to diverse multimodal task distributions. This setup mimics how humans make use of a diverse set of prior skills to learn new skills. Previous work has achieved encouraging performance. In particular, in spite of the diversity of the multimodal tasks, previous work claims that a single meta-learner trained on a multimodal distribution can sometimes outperform multiple specialized meta-learners trained on individual unimodal distributions. The improvement is attributed to knowledge transfer between different modes of task distributions. However, there is no deep investigation to verify and understand the knowledge transfer between multimodal tasks. Our work makes two contributions to multimodal meta-learning. First, we propose a method to analyze and quantify knowledge transfer across different modes at a micro-level. Our quantitative, task-level analysis is inspired by the recent transference idea from multi-task learning. Second, inspired by hard parameter sharing in multi-task learning and a new interpretation of related work, we propose a new multimodal meta-learner that outperforms existing work by considerable margins. While the major focus is on multimodal meta-learning, our work also attempts to shed light on task interaction in conventional meta-learning.


[Slides & Talk - Coming Soon]


The PyTorch implementation of our Kernel ModuLation (KML) scheme for mulimodal meta-learning.


Paper and Supplementary Material

Milad Abdollahzadeh, Touba Malekzadeh, Ngai-Man Cheung
Revisit Multimodal Meta-Learning through the Lens of Multi-Task Learning.
35th Conference on Neural Information Processing Systems (NeurIPS 2021).



This project was supported by SUTD project PIE-SGP-AI-2018-01. This research was also supported by the National Research Foundation Singapore under its AI Singapore Programme [Award Number: AISG-100E2018-005].