Introduction, Applications

E 1, Probabilities (refresher only)

Data Processing. Data Resources, Evaluation, Annotation

E 4.4-4.5, JM 2, 4.7-4.9, Nathan Schneider’s unix notes, Unix for poets, sculpting text, Suleyman Reading 1, Suleyman Reading 2, Hank Green Video

Linear Classifiers

E 2.2, 2.3, 2.4. JM 4, 5, Thumbs up? Sentiment Classification using Machine Learning Techniques, Goldwater probability tutorial. The Perceptron (Rosenblatt 1958) (optional)

Non-linear Classifiers, Backprop, Gradient Descent

E 3. JM 7.2–7.5 :

Distributional Feature Representations: PPMI, LSA, word2vec

E 14.3, 14.5–6. JM 6, LSA via SVD, Linguistic regularities in continuous space word representations, Efficient Estimation of Word Representations in Vector Space, Distributed Representations of Words and Phrases and their Compositionality

N-Gram Language Models, Feed Forward and Recurrent Language Models (RNNs)

E 6.1–2, 6.4. 7.5, 7.7. JM 3 Exploring the limits of language modeling

Transformer Language Models

E 6.3, JM 9, 10. Attention is all you need LM notebook

Pretrained language models (Elmo, BERT, and sentence similarity)

JM 11.1–11.3

Pretraining and Fine-Tuning

JM 11.4

Reinforcement Learning with Human Feedback: Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO)

Ziegler RLHF Paper, [DPO Paper] (/USC-CS662/assets/files/dpo.pdf)

ethics (Guest Lecture by Katy Felkner)

The Social Impact of Natural Language Processing, Energy and Policy Considerations for Deep Learning in NLP, Model Cards for Model Reporting

Machine Translation (MT)

Multilingual

Information Retrieval (IR) and Question Asking (QA)

Dialogue

Information Extraction (IE)

HW3 out (due 11/20)

Syntax

Multimodal NLP (Guest Lecture by Xuezhe Ma)

Spoken Language Processing (SLP) (Guest Lecture by TBD)

Project Report Version 1 due

VETERANS DAY NO CLASS

EMNLP NO CLASS

-

Agents (Guest Lecture by Tenghao Huang)

Discourse (Guest Lecture by Alexander Spangher)

Evaluation/Benchmarks

THANKSGIVING BREAK; NO CLASS

Project presentations

Project presentations