An accessible introduction to the few-shot and meta-learning thread of my dissertation research.

Suppose we want a virtual assistant to recognize a user’s intent in a language with very little annotated training data. A conventional approach might fine-tune a model on whatever labeled examples are available. But when the dataset is tiny, the model can overfit quickly.

This is where few-shot learning and meta-learning become interesting.

Learning how to adapt

Few-shot learning focuses on learning from a small number of examples. Meta-learning asks a related but deeper question: can training across different tasks teach a model how to adapt to a new task efficiently?

One influential framework is model-agnostic meta-learning (MAML). It separates learning into two interacting stages: an inner loop, where the model adapts to a sampled task, and an outer loop, where it updates shared parameters to improve future adaptation.

For cross-lingual NLP, the tasks may involve different languages, domains, or data subsets. The hope is that experience across them will produce a more useful starting point for low-resource adaptation.

X-METRA-ADA

In X-METRA-ADA (NAACL 2021), my collaborators and I investigated meta-transfer learning for cross-lingual natural language understanding. The research considers how adaptation stages can help models generalize to target languages with limited supervision.

The broader dissertation work also explored meta-distillation for multilingual semantic search: using a teacher–student learning perspective to support adaptation and multilingual alignment. In the 2023 proposal, that line of work was described as MAML-Align; its status there should not be interpreted as a claim about its later publication outcome.

The evaluation challenge

Low-resource evaluation is not just a matter of reporting a single score. Language similarity, training-data balance, and whether datasets were built through translation can all affect what a result means. My proposal emphasizes carefully constructed meta-tasks and evaluation across typologically diverse languages.

The important question is not whether a model can memorize a few examples. It is whether those examples help it generalize to language settings it has not seen in the same way before.

Read more: X-METRA-ADA paper · Research code · Cross-lingual event trigger extraction (CoNLL 2019).