Introduction, Applications

E 1, Probabilities (refresher only)

Data Processing. Data Resources, Evaluation, Annotation

E 4.4-4.5, JM 2, 4.7-4.9, Nathan Schneider’s unix notes, Unix for poets, sculpting text, Suleyman Reading 1, Suleyman Reading 2, Hank Green Video Berg-Kirkpatrick on statistical significance tests

Linear Classifiers

E 2.2, 2.3, 2.4. JM 4, app. B, Thumbs up? Sentiment Classification using Machine Learning Techniques, Goldwater probability tutorial. The Perceptron (Rosenblatt 1958) (optional)

Non-linear Classifiers, Backprop, Gradient Descent

E 3. JM 6 :

Distributional Feature Representations: PPMI, LSA, word2vec

E 14.3, 14.5–6. JM 5, LSA via SVD, Linguistic regularities in continuous space word representations, Efficient Estimation of Word Representations in Vector Space, Distributed Representations of Words and Phrases and their Compositionality

N-Gram Language Models, Feed Forward and Recurrent Language Models (RNNs)

E 6.1–2, 6.4. 7.5, 7.7. JM 3, 13 Exploring the limits of language modeling, LM notebook, Fast and Robust Neural Network Joint Models for Statistical Machine Translation

Slides, Transformer Language Models

E 6.3, JM 8. Attention is all you need, Neural Machine Translation of Rare Words with Subword Units

Pretrained language models (ELMo, BERT, and sentence similarity); took 1.5 lectures

JM 10 ELMo paper BERT paper Zoph Fine-Tuning paper Fine-Tuning demo

project proposal due

Prompting and Large Language Models; took 1.5 lectures

JM 7 T5 LoRA Prefix Tuning T0

Jinyi Ye - What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspective

Questions by: Narges Ghasemi Ghaleh Bahmani

Saba Hashemi Safaei - Byte Latent Transformer: Patches Scale Better Than Tokens

Questions by: Yuxin Yang

Reinforcement Learning with Human Feedback: Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO); started but did not finish

JM 6, Ziegler RLHF Paper, DPO Paper

Sadra Sabouri Halestani - HUMT DUMT: Measuring and controlling human-like language in LLMs

Questions by: Nikunj Gupta

Feiyu Zhu - Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

Questions by: Sichang (Stephen) He

MEGA/Megalodon/Gecko (Guest Lecture by Xuezhe Ma)

Mega Paper Megalodon

Narges Ghasemi Ghaleh Bahmani - LLMs know their vulnerabilities: Uncover Safety Gaps through Natural Distribution Shifts

Questions by: Tianming Guo

Saeed Hedayatian - TreeRL: LLM Reinforcement Learning with On-Policy Tree Search

Questions by: Zhiyuan Gao

Reinforcement Learning with Human Feedback: Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO); almost finished

Daniel Ruiz - TokAlign: Efficient Vocabulary Adaptation via Token Alignment

Questions by: Abhinav Vadhera

Ardysatrio Haroen - Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling

Questions by: Chufan Shi

Agents (Guest Lecture by Tenghao Huang)

WebArena, Proactive Info Gathering, ToolLLM, Narrative Discourse, ReAct

Kiarash Vaziri Goodarzi - TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters

Questions by: Matthew Finlayson

Ethics (Guest Lecture by Katy Felkner)

The Social Impact of Natural Language Processing, Energy and Policy Considerations for Deep Learning in NLP, Model Cards for Model Reporting

Kaicheng Wang - MAIN-RAG: Multi-Agent Filtering Retrieval-Augmented Generation

Questions by: Ardysatrio Haroen

Zhiyuan Gao - OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use

Questions by: Naga Vamsi Ramana Dinavahi

Efficient Inference

LM-Infinite

Speculative Decoding

Keep the Cost Down: A Review on Methods to Optimize LLM’s KV Cache Consumption, Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding

Faith Baca - Large Language Models Are Biased Because They Are Large Language Models

Questions by: Sajjad Shahabi

Ruth-Ann Armstrong - Toward Automatic Discovery of a Canine Phonetic Alphabet

Questions by: Saba Hashemi Safaei

Information Retrieval (IR) and Question Answering (QA)

JM 11

Tianwen Fu - Improving Factuality with Explicit Working Memory

Questions by: Kaicheng Wang

Nikunj Gupta - Reinforced IR: A Self-Boosting Framework For Domain-Adapted Information Retrieval

Questions by: Faith Baca

Machine Translation (MT)/Multilinguality slides1 slides2

JM12 Weaver, Translation (1952)

Shixuan Li - Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms

Questions by: Jinyi Ye

Dialogue

JM 25, Appendix K

Sajjad Shahabi - Integrating Audio, Visual, and Semantic Information for Enhanced Multimodal Speaker Diarization on Multi-party Conversation

Questions by: Daniel Ruiz

Tianming Guo - HotelMatch-LLM: Joint Multi-Task Training of Small and Large Language Models for Efficient Multimodal Hotel Retrieval

Questions by: Shixuan Li

Multimodal NLP (Guest Lecture by Xuezhe Ma)

Anzhe Cheng - SHuBERT: Self-Supervised Sign Language Representation Learning via Multi-Stream Cluster Prediction

Questions by: Feiyu Zhu

Gonglin Chen - SpaRE: Enhancing Spatial Reasoning in Vision-Language Models with Synthetic Data

Questions by: Wenbin Teng

Spoken Language Processing (SLP) (Guest Lecture by Sudarsana Reddy Kadiri) speech, asr, e2e asr, tts, wav2vec2 tutorial, tacotron tutorial

JM 15

Wenbin Teng - Improve Vision Language Model Chain-of-thought Reasoning

Questions by: Kiarash Vaziri Goodarzi

Chufan Shi - ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generation

Questions by: Tianwen Fu

Project Report Version 1 due

Mind Reading (Guest Lecture by Sam Nastase)

Lydia Ignatova - Dehumanizing Machines: Mitigating Anthropomorphic Behaviors in Text Generation Systems

Questions by: Gonglin Chen

Sichang (Stephen) He - Learning to Rewrite: Generalized LLM-Generated Text Detection

Questions by: Anzhe Cheng

Information Extraction

JM17.3, 20

Abhinav Vadhera - JailbreakRadar: Comprehensive Assessment of Jailbreak Attacks Against LLMs

Questions by: Ruth-Ann Armstrong

Yuxin Yang - A Troublemaker with Contagious Jailbreak Makes Chaos in Honest Towns

Questions by: Sadra Sabouri Halestani

Discourse Slides

Danny Deng - LocAgent: Graph-Guided LLM Agents for Code Localization

Questions by: Saeed Hedayatian

Matthew Finlayson - Geometric Signatures of Compositionality Across a Language Model’s Lifetime

Questions by: Lydia Ignatova

Auditing, Dissecting, and Evaluating Large Language Models (Guest Lecture by Robin Jia)

Naga Vamsi Ramana Dinavahi - Sliding Windows Are Not the End: Exploring Full Ranking with Long-Context Large Language Models

Questions by: Danny Deng

Project Presentations

(10:00) Danny Deng, Feiyu Zhu - Adaption-of-Thought: Learning Question Difficulty Improves Large Language Models for Reasoning

Questions by: Ruth-Ann Armstrong, Yuxin Yang

(10:18) Kiarash Vaziri Goodarzi, Saba Hashemi Safaei, Saeed Hedayatian - Unveiling Multi-level and Multi-modal Semantic Representations in the Human Brain using Large Language Models

Questions by: Wenbin Teng, Naga Vamsi Ramana Dinavahi, Gonglin Chen

(10:36) Narges Bahmani, Sajjad Shahabi - Evaluating the Prompt Steerability of Large Language Models

Questions by: Nikunj Gupta, Chufan Shi, Ardysatrio Haroen

(10:54) Jinyi Ye, Sichang (Steven) He - Zero-Shot Detection of LLM-Generated Text using Token Cohesiveness

Questions by: Faith Baca, Abhinav Vadhera

(11:12) Matthew Finlayson, Daniel Ruiz, Tianming Guo - Learning from Natural Language Explanations for Generalizable Entity Matching

Questions by: Lydia Ignatova, Zhiyuan Gao

(11:30) Anzhe Cheng, Kaicheng Wang, Shixuan Li - Prompts have evil twins

Questions by: Matthew Finlayson, Tianwen Fu, Sadra Sabouri Halestani

THANKSGIVING BREAK; NO CLASS

Project Presentations

(10:00) Gonglin Chen, Zhiyuan Gao - Hello Again! LLM-powered Personalized Agent for Long-term Dialogue

Questions by: Danny Deng, Saeed Hedayatian, Feiyu Zhu

(10:18) Chufan Shi, Tianwen Fu, Wenbin Teng - Vision-Language Models Can Self-Improve Reasoning via Reflection

Questions by: Daniel Ruiz, Kaicheng Wang, Saba Hashemi Safaei

(10:36) Nikunj Gupta, Yuxin Yang - FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document

Questions by: Anzhe Cheng, Kiarash Vaziri Goodarzi

(10:54) Faith Baca, Lydia Ignatova, Ruth-Ann Armstrong - BiasWipe: Mitigating Unintended Bias in Text Classifiers through Model Interpretability

Questions by: Sichang (Steven) He, Narges Bahmani

(11:12) Abhinav Vadhera, Ardysatrio Haroen - Ranking Manipulation for Conversational Search Engines

Questions by: Jinyi Ye, Shixuan Li

(11:30) Naga Vamsi Ramana Dinavahi, Sadra Sabouri Halestani - Sneaking Syntax into Transformer Language Models with Tree Regularization

Questions by: Sajjad Shahabi, Tianming Guo