CSCI 544 Fall 2018 course page

Jonathan May and Nanyun (Violet) Peng

Website https://www.isi.edu/~jonmay/cs544_fa18_web/

Lectures SAL 101, Wednesdays and Fridays 8:00-9:50 am

Instructor office hours Jonathan May or Nanyun (Violet) Peng, RTH 512, Wednesdays and Fridays 10-11am or 11am-12pm with appointment

Textbook None required; Speech and Language Processing, 2nd Edition¹ is optional but badly out of date. The new version (called ``jm'' in reading selection notes below) is incomplete but preferred². All required readings from jm and elsewhere will be listed in the syllabus.

TA Office hours Xusen Yin, 10:30am-12:30pm Mon., SAL Lab

Ramesh Manuvinakurike, 11:30am-1:30pm Wed., SAL Lab

Sarik Ghazarian, 3pm-5pm Wed., SAL Lab

Quizzes (3% of grade) approximately 25 x 0.12% of grade each. Multiple choice online tests taken mid-class to break up the monotony. Open note, open internet, open discussion.

Homeworks (48.6% of grade) 6 x 8.1% of grade each. A mix of programming and written assignments, electronic submission due 11:59 PM local Pacific time per table below.

Project (8.1% of grade) Can be done in groups of up to 4. Preliminary part due mid-class, due on last class day. (this may be replaced with a seventh homework)

Late days Four (4) cumulative with no more than two (2) per assignment. 50% penalty first day; thereafter, no credit afterward.

Midterm (15% or 25% of grade, whichever is more advantageous) Friday, October 5, 8:00-9:45 am. Covers lectures, readings, and homeworks from 8/22-10/3

Final exam (15% or 25% of grade, whichever is more advantageous) Wednesday, December 5, 8:00-10:00 am. Covers lectures, readings, and homeworks from 8/22-11/30 (i.e. the entire class)

Contact us On Piazza or in class/office hours. Please do not email (unless notified otherwise).

A note about the midterm/final grading: whichever you do better on will count as 25% of your grade and the other one will count as 15% of your grade. Together the tests will count as 40% of your grade.

Syllabus/schedule - subject to change - latest version is always on the website!

instructor	date	material	reading	HW out	HW due
JM	8/22	intro, applications	jm v2 c1 ³, hirschberg & manning ⁴
JM	8/24	corpora, text processing, words, regular expressions (lec 2 starting code ⁵)	NLTK ch.2 ⁶, jmv3 c2⁷, nathan schneider's notes ⁸, Unix for poets⁹, sculpting text¹⁰,	HW 1 (due 9/7)
JM	8/29	Morphology, Finite-State Automata, FSA relationship to regex, finite state transducers	jm v2 c3 ¹¹
JM	8/31	probability	Goldwater probability tutorial ¹²,	HW 2 (due 9/14)
JM	9/5	Classifiers, features, naive Bayes, perceptron, logistic regression	jm v3 c6¹³, Eisenstein notes pp. 21-48¹⁴
JM	9/7	pos tagging, hmm, search	jm v3 c9¹⁵, blunsom notes¹⁶, collins notes (optional, more detailed)¹⁷		HW1
NP	9/12	parsing and syntax 1: treebanks, evaluation, cky, grammar induction	penn treebank ¹⁸, jm v3 c11 ¹⁹ jm v3 c12²⁰, jm v3 c13²¹	HW3=project proposal (due 9/26)
NP	9/14	parsing and syntax 2: pcfgs, restructuring, lexicalization, smoothing, beaming	chiang notes²²		HW2
NP	9/19	parsing and syntax 3: dependencies, mst and shift reduce algorithms	jm v3 c14²³	HW 4 (due 10/10)
NP	9/21	ngram LMs	jm v3 c4²⁴
NP	9/26	smoothing, interpolation	(optional) chen and goodman²⁵		HW 3(pp)
NP	9/28	feed forward lm, rnn lm	jm v3 c8²⁶
JM	10/3	REVIEW
	10/5	MIDTERM
NP	10/10	classical lexical semantics	jm v3 c17²⁷		HW 4
NP	10/12	distributional lexical semantics	jm v3 c15²⁸, jm v3 c16²⁹	HW 5 (due 10/26)
NP	10/17	RNNs, LM and semantics	jmv3 c7³⁰ c9³¹
NP	10/19	Information Extraction	jmv3 c21³²	HW 6 (due 11/9)
NP	10/24	Information Extraction: CRF	Collins's Note³³ Sutton and McCallum'a Note³⁴		project rewrite (moved to 10/31)
NP	10/26	Neural CRF	Lample et. al.³⁵		HW 5
JM	10/31	MT evaluation and alignment	arcturan and centauri³⁶, Bleu³⁷, koehn slides³⁸		project rewrite
JM	11/2	Statistical MT	Brown et al. ³⁹ (mathy bits of models 3+ can be skimmed), Knight workbook⁴⁰, mert⁴¹
JM	11/7	Neural MT	neubig tutorial⁴² or Koehn Chapter (section 13.5) ⁴³
JM	11/9	Summarization	jmv2 c23.3-23.7 (see blackboard), abi see blog post⁴⁴		HW 6
JM	11/14	Task-Oriented Dialogue	jmv3 c24⁴⁵, jmv3 c25⁴⁶
JM	11/16	Chit-Chat Dialogue	ritter et al.⁴⁷
	11/21	NO CLASS
	11/23	NO CLASS
JM	11/28	Leftovers: annotation, common sense, Entailment, generation	overview of kappa: ⁴⁸, bowman snli paper: ⁴⁹, csr: davis and marcus⁵⁰
JM/NP	11/30	Review			project
	12/5	FINAL

Footnotes

... Edition ¹: https://www.amazon.com/Speech-Language-Processing-Daniel-Jurafsky/dp/0131873210/
... preferred ²: https://web.stanford.edu/~jurafsky/slp3
... c1 ³: http://www.cs.colorado.edu/~martin/SLP/Updates/1.pdf
... manning ⁴: https://cs224d.stanford.edu/papers/advances.pdf
... code ⁵: https://www.isi.edu/~jonmay/cs544_fa18_web/lec2.startercode.zip
... ch.2 ⁶: http://www.nltk.org/book/ch02.html
... c2 ⁷: https://web.stanford.edu/~jurafsky/slp3/2.pdf
... notes ⁸: https://github.com/nschneid/unix-text-commands
... poets ⁹: https://www.cs.upc.edu/~padro/Unixforpoets.pdf
... text ¹⁰: http://matt.might.net/articles/sculpting-text/
... c3 ¹¹: http://ling.umd.edu/~idsardi/620/Jurafsky/jurafsky2000-3.pdf
... tutorial ¹²: http://homepages.inf.ed.ac.uk/sgwater/teaching/general/probability.pdf
... c6 ¹³: https://web.stanford.edu/~jurafsky/slp3/6.pdf
... 21-48 ¹⁴: https://github.com/jacobeisenstein/gt-nlp-class/blob/master/notes/eisenstein-nlp-notes.pdf
... c9 ¹⁵: https://web.stanford.edu/~jurafsky/slp3/9.pdf
... notes ¹⁶: http://digital.cs.usu.edu/~cyan/CS7960/hmm-tutorial.pdf
... detailed)¹⁷: http://www.cs.columbia.edu/~mcollins/courses/nlp2011/notes/hmms.pdf
... treebank ¹⁸: http://aclweb.org/anthology/J93-2004
... c11 ¹⁹: https://web.stanford.edu/~jurafsky/slp3/11.pdf
... c12 ²⁰: https://web.stanford.edu/~jurafsky/slp3/12.pdf
... c13 ²¹: https://web.stanford.edu/~jurafsky/slp3/13.pdf
... notes ²²: https://www3.nd.edu/~dchiang/teaching/nlp/2016/notes/chapter13v2.pdf
... c14 ²³: https://web.stanford.edu/~jurafsky/slp3/14.pdf
... c4 ²⁴: https://web.stanford.edu/~jurafsky/slp3/4.pdf
... goodman ²⁵: https://people.eecs.berkeley.edu/~klein/cs294-5/chen_goodman.pdf
... c8 ²⁶: https://web.stanford.edu/~jurafsky/slp3/8.pdf
... c17 ²⁷: https://web.stanford.edu/~jurafsky/slp3/17.pdf
... c15 ²⁸: https://web.stanford.edu/~jurafsky/slp3/15.pdf
... c16 ²⁹: https://web.stanford.edu/~jurafsky/slp3/16.pdf
... c7 ³⁰: https://web.stanford.edu/~jurafsky/slp3/7.pdf
... c9 ³¹: https://web.stanford.edu/~jurafsky/slp3/9.pdf
... c21 ³²: https://web.stanford.edu/~jurafsky/slp3/21.pdf
... Note ³³: http://www.cs.columbia.edu/~mcollins/crf.pdf
... Note ³⁴: https://people.cs.umass.edu/~mccallum/papers/crf-tutorial.pdf
... al.³⁵: https://www.aclweb.org/anthology/N16-1030
... centauri ³⁶: https://www.isi.edu/natural-language/mt/aimag97.pdf
... Bleu ³⁷: http://www.aclweb.org/anthology/P02-1040.pdf
... slides ³⁸: http://www.statmt.org/book/slides/04-word-based-models.pdf
... al.³⁹: http://www.aclweb.org/anthology/J93-2003
... workbook ⁴⁰: https://www.isi.edu/natural-language/mt/wkbk-rw.pdf
... mert ⁴¹: http://www.aclweb.org/anthology/P03-1021
... tutorial ⁴²: https://arxiv.org/pdf/1703.01619.pdf
... 13.5)⁴³: https://arxiv.org/abs/1709.07809
... post ⁴⁴: http://www.abigailsee.com/2017/04/16/taming-rnns-for-better-summarization.html
... c24 ⁴⁵: https://web.stanford.edu/~jurafsky/slp3/24.pdf
... c25 ⁴⁶: https://web.stanford.edu/~jurafsky/slp3/25.pdf
... al.⁴⁷: http://aclweb.org/anthology/D11-1054
... kappa:⁴⁸: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3900052/
... paper:⁴⁹: https://nlp.stanford.edu/pubs/snli_paper.pdf
... marcus ⁵⁰: https://cs.nyu.edu/davise/papers/CommonsenseFinal.pdf

jonmay@isi.edu

Website	`https://www.isi.edu/~jonmay/cs544_fa18_web/`
Lectures	SAL 101, Wednesdays and Fridays 8:00-9:50 am
Instructor office hours	Jonathan May or Nanyun (Violet) Peng, RTH 512, Wednesdays and Fridays 10-11am or 11am-12pm with appointment
Textbook	None required; Speech and Language Processing, 2nd Edition¹ is optional but badly out of date. The new version (called ``jm'' in reading selection notes below) is incomplete but preferred². All required readings from jm and elsewhere will be listed in the syllabus.
TA Office hours	Xusen Yin, 10:30am-12:30pm Mon., SAL Lab
	Ramesh Manuvinakurike, 11:30am-1:30pm Wed., SAL Lab
	Sarik Ghazarian, 3pm-5pm Wed., SAL Lab
Quizzes (3% of grade)	approximately 25 x 0.12% of grade each. Multiple choice online tests taken mid-class to break up the monotony. Open note, open internet, open discussion.
Homeworks (48.6% of grade)	6 x 8.1% of grade each. A mix of programming and written assignments, electronic submission due 11:59 PM local Pacific time per table below.
Project (8.1% of grade)	Can be done in groups of up to 4. Preliminary part due mid-class, due on last class day. (this may be replaced with a seventh homework)
Late days	Four (4) cumulative with no more than two (2) per assignment. 50% penalty first day; thereafter, no credit afterward.
Midterm (15% or 25% of grade, whichever is more advantageous)	Friday, October 5, 8:00-9:45 am. Covers lectures, readings, and homeworks from 8/22-10/3
Final exam (15% or 25% of grade, whichever is more advantageous)	Wednesday, December 5, 8:00-10:00 am. Covers lectures, readings, and homeworks from 8/22-11/30 (i.e. the entire class)
Contact us	On Piazza or in class/office hours. Please do not email (unless notified otherwise).