Link Search Menu Expand Document

Natural Language Processing

CS 6340/5340, Spring 2024
MoWe / 11:50am - 1:10pm , WEB L102, Recordings

Instructor: Ana Marasović (she/her)

ana.marasovic@utah.edu

Office Hours: Tue 11:30-12:30 (MEB 2166)

TA: Purbid Bambroo (he/him)

u1472659@utah.edu

Office Hours: Thur 11am-12pm; Fri 1-2pm (MEB 3105)

TA: Rishanth Rajendhran (he/him)

u1419542@utah.edu

Office Hours: Mon/Wed 1:15-2:15pm (MEB 3105)


Through this website we will share:

Syllabus, announcements, communicating grades, policies, etc. are shared on Canvas. The current page you are on is NOT the syllabus. Please review this syllabus carefully – students are responsible for understanding everything there!

Communication will be done through Piazza. You will turn in your assignments to Gradescope.

Many links on this website can be accessed only with your gcloud.utah.edu account.

Calendar

The calendar and readings are tentative and subject to change.

Date Theme Topic Lecture Materials Work due
1/8 Course Logistics; Intro to NLP [recording] [slides] [readings]
1/10 Elements of a Simple NLP System Machine learning basics; Perceptron [recording] [slides] [readings]
1/15 MLK Day (no class)
1/17 cont. Logistic regression; Tokenization; Morphology [recording] [slides] [readings] 1/19 drop deadline
1/22 cont. Word embeddings [recording] [slides] [readings]
1/24 cont. Neural networks basics [recording] [slides] [readings] HW1 due (by 1/25)
1/29 Language Modeling (LM) n-gram LM; RNNs and their shortcomings [recording] [slides] [readings]
1/31 Machine Translation (MT) MT; Conditional generation; Decoding methods [recording] [slides] [readings]
2/5 Modern (Large) Language Models Attention; Self-attention (SA); Multi-headed SA; Positional embeddings; Transformers [recording] [slides] [readings]
2/7 cont. Pretraining; Decoders (GPT); MLM; Encoders (BERT); Encoder-Decoders (T5); Finetuning (FT) [recording] [slides] [readings] HW2 due (by 2/8)
2/12 cont. In-context learning; Instruction FT (FLAN-T5) [recording] [slides] [readings]
2/14 cont. Instruction FT (cont.); Learning from human feedback (LLaMA-2) [recording] [slides] [readings]
2/19 Presidents Day (no class)
2/21 Applications Taskification and Valid Benchmarks [recording] [slides] [readings] HW3 due (by 2/22)
2/26 Midterm course feedback; Midterm review [recording] [slides]
2/28 Midterm exam
3/4 Spring Break (no class)
3/6 Spring Break (no class)
3/11 Linguistic Structure Prediction Motivation [recording] [slides] [readings]
3/13 cont. Syntax; Context free-grammars; Constituency parsing [recording] [slides] [readings]
3/18 cont. Dependency parsing [recording] [slides] [readings]
3/20 cont. Part-of-speech tagging, Named entity recognition; Hiden Markov models [recording] [slides] [readings]
3/25 cont. Viterbi algorithm; Forward/backward probabilities; Conditional Random Fields [recording] [slides] [readings]
3/27 cont. Logical semantics; Predicate-argument semantics [recording] [slides] [readings] HW4 due (by 3/28)
4/1 cont. Coreference resolution; Discourse [recording] [slides] [readings]
4/3 Beyond Text Pragmatics [recording] [slides] [readings]
4/8 cont. Vision and language; Speech [recording] [slides] [readings]
4/10 Responsible NLP Data transparency; Reliable evaluation; Policy [recording] [slides] [readings]
4/15 cont. (Mechanistic) interpretability; Explainability; Safety [recording] [slides] [readings]
4/17 cont. TBD [recording] [slides] [readings] FP due (by 4/18)
4/22 Final course feedback; Final review [recording] [slides]
4/24 Reading day (no class)
4/26 Final exam (10:30 am – 12:30 pm)

FAQ Before Enrolling

How will you be evaluated? [Click to expand!]

Your performance in this course will be evaluated by:


Formal pre-requisites? Expected background? [Click to expand!]

The formal pre-requisite is C- or better in CS 3505.

This course is designed for computer science majors. We expect that you:

  • …are experienced with programming in Python – we won’t be reviewing your your code directly and we expect you to figure out the implementation side on your own; we are here to help you with understanding concepts and translating ideas into pseudocode,
  • …are comfortable with basic calculus, probability, and linear algebra,
  • …can quickly pick up machine learning and deep learning,
  • …can quickly pick up coding in pytorch,
  • …are interested in language and don’t aim to just deepen your machine learning knowledge.

If you completed CS 5353/6353 (Deep Learning), CS 5350/6350 (Machine Learning), CS 6957 (NLP with Neural Networks), or CS 6966/5966 (Local Explanations for Deep Learning Models), we expect you will be able to keep up.

Revisiting/polishing your knowledge. You can prepare by:

  1. There are a ton of Python resources for people with some programming experience. Check them out here. I highly recommend these slides/colab (caution: there are some typos) and my colleagues suggest these: 1, 2, 3, and 4.

  2. Math and machine learning basics are nicely covered in the first part of the Deep Learning book. Obviously, you can use the same book to familiarize yourself with deep learning. If you learn by coding then you will find this resource helpful Practical Deep Learning for Coders by Fast.ai (3: Neural net foundations; 5: From-scratch model, 13: Backpropagation & MLP, 14: Backpropagation)