Calendar

Week 1

Aug 25

Introduction, Applications: E 1, Probabilities (refresher only)

HW0 out (due 8/29)

paper selection out (due 9/8)

project assignment out (due 9/22)

Aug 27

Data Processing. Data Resources, Evaluation, Annotation: E 4.4-4.5, JM 2, 4.7-4.9, Nathan Schneider’s unix notes, Unix for poets, sculpting text, Suleyman Reading 1, Suleyman Reading 2, Hank Green Video Berg-Kirkpatrick on statistical significance tests

Aug 29

HW0 due

Week 2

Sep 1

LABOR DAY NO CLASS

HW1 out (due 9/19)

Sep 3

Linear Classifiers: E 2.2, 2.3, 2.4. JM 4, app. B, Thumbs up? Sentiment Classification using Machine Learning Techniques, Goldwater probability tutorial. The Perceptron (Rosenblatt 1958) (optional)

Week 3

Sep 8

Non-linear Classifiers, Backprop, Gradient Descent: E 3. JM 6 :

Sep 10

Distributional Feature Representations: PPMI, LSA, word2vec

E 14.3, 14.5–6. JM 5, LSA via SVD, Linguistic regularities in continuous space word representations, Efficient Estimation of Word Representations in Vector Space, Distributed Representations of Words and Phrases and their Compositionality

Sep 12

Early Drop (no W, refund)

Week 4

Sep 15

N-Gram Language Models, Feed Forward and Recurrent Language Models (RNNs): E 6.1–2, 6.4. 7.5, 7.7. JM 3, 13 Exploring the limits of language modeling LM notebook

Sep 17

Attention, Transformer Language Models: E 6.3, JM 8. Attention is all you need

Sep 19

HW1 due

Week 5

Sep 22

Pretrained language models (ELMo, BERT, and sentence similarity): JM 10 ELMo paper BERT paper Zoph Fine-Tuning paper Fine-Tuning demo

project proposal due

Sep 24

NO CLASS

Week 6

Sep 29

Prompting and Large Language Models: JM 7 T5 LoRA Prefix Tuning T0

HW2 out (due 10/18)

Oct 1

Reinforcement Learning with Human Feedback: Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO): JM 6, Ziegler RLHF Paper, DPO Paper

Week 7

Oct 6: Efficient Inference

Oct 8

MEGA (Guest Lecture by Xuezhe Ma): Mega Paper Megalodon

Oct 10: Mid Drop (No W, No refund)

Week 8

Oct 13

Agents (Guest Lecture by Tenghao Huang): WebArena, ToolLLM, Narrative Discourse, ReAct

Oct 15

Ethics (Guest Lecture by Katy Felkner): The Social Impact of Natural Language Processing, Energy and Policy Considerations for Deep Learning in NLP, Model Cards for Model Reporting

Oct 17: HW 2 due

Week 9

Oct 20

Information Retrieval (IR) and Question Answering (QA): JM 11

Oct 22

Machine Translation (MT)/Multilinguality slides1 slides2: JM12 Weaver, Translation (1952)

Week 10

Oct 27

Dialogue: JM 25, Appendix K

HW3 out (due 11/21)

Oct 29

Information Extraction: JM17.3, 20

Week 11

Nov 3: Multimodal NLP (Guest Lecture by Xuezhe Ma)

Nov 5

Spoken Language Processing (SLP) (Guest Lecture by Sudarsana Reddy Kadiri): JM 15

Nov 7

Project Report Version 1 due

Week 12

Nov 10

Mind Reading (Guest Lecture by Sam Nastase)

Nov 12

TBD (Guest Lecture by Robin Jia): -

Nov 14

Late Drop (W, No refund)

Week 13

Nov 17: TBD

Nov 19: Discourse Slides

Nov 21: HW 3 due

Week 14

Nov 24: TBD Guest Lecture

Nov 26

THANKSGIVING BREAK; NO CLASS

Week 15

Dec 1

Project Presentations: (10:00) TBD; Questions by: TBD; (10:22) TBD; Questions by: TBD; (10:44) TBD; Questions by: TBD; (11:06) TBD; Questions by: TBD; (11:28) TBD; Questions by: TBD

Dec 4

Project presentations: (10:00) TBD; Questions by: TBD; (10:22) TBD; Questions by: TBD; (10:44) TBD; Questions by: TBD; (11:06) TBD; Questions by: TBD; (11:28) TBD; Questions by: TBD