Unit 4: Current research
In this unit, you will see several examples of current research on large language models. The unit features both lecture-style reviews of recent developments and videos from research presentations.
Lectures
The lectures start by exploring how LLMs might store factual information. You will then learn about efficient fine-tuning techniques, retrieval-augmented generation, multilingual transformer architectures, and the issue of data contamination. The series concludes by reflecting on the “stochastic parrots” debate.
| Section | Title | Video | Slides | Quiz |
|---|---|---|---|---|
| 4.1 | How might LLMs store facts? | video | none | quiz |
| 4.2 | Efficient fine-tuning | video | paper | quiz |
| 4.3 | Retrieval-augmented generation (until 17:50) | video | none | quiz |
| 4.4 | Multilinguality and modular transformers | video | paper | quiz |
| 4.5 | Data contamination | video | paper | quiz |
| 4.6 | LLMs as stochastic parrots (until 15:00) | video | none | quiz |
To earn a wildcard for this unit, you must complete the quizzes no later than 2025-11-25.
Online meeting
During the online meeting, we will see additional examples of current research on LLMs.
The meeting will take place on 2025-11-26 between 18:00–20:00. A Zoom link will be sent out via the course mailing list.
Lab
In this lab, you will implement LoRA, one of the most well-known methods for parameter-efficient fine-tuning of large language models. Along the way, you will earn experience with Hugging Face Transformers, a state-of-the-art library for training and deploying language models, as well as with several related libraries.