Unit 3: Developing LLMs
In this unit, you will get an overview of different issues related to the development of large language models. In particular, the unit covers training strategies, the issue of data, emergent abilities of LLMs, and LLM alignment. It also discusses scaling laws, an important empirical finding about the relationship between the size and performance of large language models.
| Section | Title | Video | Slides | Quiz |
|---|---|---|---|---|
| 3.1 | Introduction to LLM development | video | slides | quiz |
| 3.2 | Training LLMs | video | slides | quiz |
| 3.3 | Data for LLM pretraining | video | slides | quiz |
| 3.4 | Scaling laws | video | slides | quiz |
| 3.5 | Emergent abilities of LLMs | video | slides | quiz |
| 3.6 | LLM alignment | video | slides | quiz |
Lab 3 will be published on 2025-02-10.