Skip to main content Link Search Menu Expand Document (external link)
CSCI 2590

Calendar

Supervised learning

Jan 24
Text classification [recording]
HW 1 out[zip]
  1. Course overview
  2. Supervised learning basics
  3. Feature-based text classification
Jan 31
Word embedding [recording]
  1. Distributed representation of words
  2. Learning word vectors
Section Operations on word vectors [slides][notebook]
Feb 7
Sequence modeling [recording]
HW 1 due
HW 2 out[zip]
  1. Neural network basics
  2. RNN and its variants
  3. Attention and Transformers
SectionHPC tutorial and Pytorch tutorial
Feb 14
Sequence generation [recording]
  1. Encoder-decoder models
  2. Decoding algorithms
  3. Machine translation and evaluation
Feb 21
Tasks and applications [recording]
  1. The landscape of NLP tasks
  2. Evaluation
  3. Final project tips
Section NLP benchmarks [slides][notebook]

Representation learning

Feb 28
Pretrain then finetune [recording]
HW 2 due
  1. Data
  2. Self-supervised learning
  3. Adaptation / finetuning
SectionHuggingface transformer
Mar 7
Online midterm (no lecture)
  1. [2020 midterm] [2020 midterm solutions]
  2. [2021 midterm] [2021 midterm solution]
Mar 14
Spring break (no lecture)
Mar 21
Advanced pretraining and finetuning techniques [recording]
proposal due
HW 3 out[zip]
  1. Efficient pretraining
  2. Efficient finetuning
  3. Additional reading: Efficient transformers: a survey
Mar 28
Holistic evaluation
  1. Behavioral tests
  2. Fairness, robustness, privacy
  3. Additional reading: Holistic evaluation of language models

NLP via language modeling

Apr 4
Language models [recording]
  1. N-gram language models
  2. Neural autoregressive language models
  3. Language models as general task solvers
SectionPrompt engineering
Apr 11
Scaling language models (by Jason Wei) [recording]
HW 3 due
  1. Scaling laws
  2. Emergent abilities
  3. Reasoning
Apr 18
Aligning language models (basics)
  1. Why do we need alignment?
  2. Approaches to alignment
  3. Tips on project presentation and report
Apr 25
Aligning language models (advanced) (by Hyung Won Chung) [recording]
  1. Instruction tuning
  2. RLHF
May 2
Project presentation
Report due