Week 5
Dear all,
We are approaching the end of course week 5 – and with it, the end of the first half of the course. Many things are happening this week, but I hope my weekly newsletter helps you to stay on track.
Do you have any questions about the lectures, lab, or project? Do not hesitate to book a short meeting with me! Schedule a 15-min chat
Best, Marco
This week: Alignment and current research
In this final unit of the course, you have learned about the alignment stage of LLM training. You have also seen several examples of current research in this and related areas. The material for this unit included lectures, lecture-style reviews of recent developments, and videos from research presentations. In the lab, you implement LoRA, one of the most well-known methods for parameter-efficient fine-tuning of LLMs, and gain experience with Hugging Face Transformers.
Also this week: Project seminar 4
On Tuesday, we had the fourth and final project seminar. The goal of the seminar was to “get you into project mode”. During the seminar, you prepared a two-minute pitch for your project, reflected on the resources (data, compute, evaluation methods, etc.) a project may need, and identified potential risks. At the end of the seminar, each group announced the next concrete step they would take to advance the project.
Make use of our feedback!
We strive to provide prompt, constructive, and meaningful feedback on your work in the course. We focus on formative feedback, which you can use to improve your learning (and we can use to improve our teaching!) while the course is ongoing. At the same time, we expect you to take responsibility for your learning by actively seeking feedback and reflecting on how to use it.
Here is a reminder about three forms of feedback that we encourage you to make use of:
- Lab sessions. Attending lab sessions and speaking with lab assistants are the primary ways to receive feedback on your lab work.
- Portfolio review. After lab 4, you can request a second “sanity check” of your portfolio and feedback on specific tasks from your lab assistant.
- 15-minute meeting. At any time during the course, you can book 15 minutes with me to get feedback on your learning.
Next week: Theory test and start of the project phase
On Monday, we will finish off the theory part of the course with a test. As previously announced, the test will consist of 40 questions taken from the quizzes (and sometimes lightly modified). This means revisiting the quiz questions and reviewing the sample solutions (available on the course website) is an excellent way to prepare for the test.
You do not need to register for the test; just bring a pen.
With the theory part of the course checked off, you can focus on your project. Many of you have already booked meetings with me to pitch project ideas and get suggestions and help. To schedule a meeting, use the following link: https://doodle.com/bp/marcokuhlmann/15min