Lecture notes

1. Introduction to natural language processing
2. Text normalization, units and edit distance
3. Language modelling with N-grams
4. Entropy and perplexity (advanced)
5. Hidden Markov models
6. Expectation maximization (advanced)
7. Word embeddings
8. An introduction to neural networks
9. Recurrent neural networks
10. Encoder-decoder models and attention

Videos

• 10. Encoder-decoder models and attention

  1. A basic encoder-decoder model for machine translation
  2. Training and loss for encoder-decoder models
  3. Encoder-decoder models in general
  4. Greedy decoding
  5. Beam search
  6. Basic attention
  7. Attention - More general
  8. Evaluating machine translation with BLEU

Acknowledgements

With permission, I have used content from the NLP courses taught by Jan Buys (University of Cape Town) and Sharon Goldwater (University of Edinburgh).

License

Herman Kamper, 2022–2023.
This work is released under a Creative Commons Attribution-ShareAlike license (CC BY-SA 4.0).