Unit 4: Alignment and current research
In this unit, you will learn more about the alignment stage of LLM training. You will also see several examples of current research in this and related areas. The unit features both lecture-style reviews of recent developments and videos from research presentations.
Lectures
The lectures start by exploring LLMs alignment. You will then learn about current research on how LLMs store facts, efficient fine-tuning techniques, retrieval-augmented generation, and how tokenisation relates to LLM privacy and security. The series concludes by reflecting on the “stochastic parrots” debate.
| Section | Title | Video | Slides | Quiz |
|---|---|---|---|---|
| 4.1 | LLM alignment | video | slides | quiz |
| 4.2 | LLMs for fact completion | video | slides | quiz |
| 4.3 | Efficient fine-tuning | video | paper | quiz |
| 4.4 | Retrieval-augmented generation (until 17:50) | video | none | quiz |
| 4.5 | Adversarial tokenization | video | slides | quiz |
| 4.6 | LLMs as stochastic parrots (until 15:00) | video | none | quiz |
To earn a wildcard for this unit, you must complete the quizzes before the teaching session on Unit 4.
Additional materials
Lab
In this lab, you will implement LoRA, one of the most well-known methods for parameter-efficient fine-tuning of large language models. Along the way, you will earn experience with Hugging Face Transformers, a state-of-the-art library for training and deploying language models, as well as with several related libraries.
In the advanced lab, you will combine what you have learned in the previous labs to fine-tune a GPT-2 model for sentiment classification using LoRA. However, in contrast to the basic lab, you will not use Hugging Face Transformers but the code from labs 1–3.