Transformer-based models

Published

February 5, 2024

Transformer-based models

Level
Basic (22 points)

In this lab, you will implement the encoder–decoder architecture of Sutskever et al., 2014, including the attention-based extension presented of Bahdanau et al., 2015, and evaluate this architecture on a machine translation task.

Link to the basic lab

BERT for Natural Language Inference

Level
Advanced (22 points)

One of the main selling points of pre-trained language models is that they can be applied to a wide spectrum of different tasks in natural language processing. In this lab you will test this by fine-tuning a pre-trained BERT model on a benchmark task in natural language inference.

Link to the advanced lab