L645 / B659
Advanced Natural Language Processing (NLP)
Course goals In recent years, statistical methods have become the standard in the field of Natural Language Processing (NLP). This course gives an introduction to statistical models and machine learning paradigms in NLP. Such methods are helpful for reaching wide coverage, reducing ambiguity, automatic learning, increasing robustness, etc.
In this course, we will cover basic notions in probability and information theory, focusing on the concepts needed for NLP. Then we will discuss (Hidden) Markov Models, exemplified by an approach to POS tagging. The following sessions will be dedicated to probabilistic approaches to parsing, focusing on probabilistic context-free grammars.
Additionally, we will cover topics such as word sense disambiguation, text categorization, and statistical alignment methods and their use in machine translation. What we cover in the last third of the class will depend in part upon student interest. We will be focusing on statistical methods in the context of particular tasks, but all of the methods we will use are applicable to a range of tasks in NLP. Thus, this course provides an essential platform for further work in NLP.
Course website: http://cl.indiana.edu/~md7/15/645/
Assignments, slides, etc. will be posted here. (I only use oncourse for emails & semi-restricted data.)
Recommended course prerequisites: Some knowledge of either linguistics or computer science is extremely beneficial, with Computation and Linguistic Analysis (L545) being the strongest background. Some programming experience is expected.
|or by appointment|
Additional Opportunities I would encourage you to check out the weekly colloquium series on Computational Linguistics (CLingDing): see http://cl.indiana.edu/wiki/. Note, too, that there is an email listserv (COMPLING-L) which has announcements for talks, internships, etc.
Also, as it so happens, the Workshop on Methods will be bringing in David Blei (on October 16, 2–4pm), just as we will be about to start discussing topic modeling. I encourage you to attend. More information is at: http://ssrc.indiana.edu/seminars/wim.shtml
Academic Integrity (from the Dean for Academic Standards and Opportunities): As a student at IU, you are expected to adhere to the standards and policies detailed in the Code of Student Rights, Responsibilities, and Conduct (http://www.iu.edu/~code/). When you submit an assignment with your name on it, you are signifying that the work contained therein is all yours, unless otherwise cited or referenced. Any ideas or materials taken from another source for either written or oral use must be fully acknowledged. If you are unsure about the expectations for completing an assignment or taking a test or exam, be sure to seek clarification beforehand. All suspected violations of the Code will be handled according to University policies. Sanctions for academic misconduct may include a failing grade on the assignment, reduction in your final course grade, a failing grade in the course, among other possibilities, and must include a report to the Dean of Students who may impose additional disciplinary sanctions.
Students with Disabilities: Students who need an accommodation based on the impact of a disability should contact me to arrange an appointment as soon as possible to discuss the course format, to anticipate needs, and to explore potential accommodations.
I rely on Disability Services for Students for assistance in verifying the need for accommodations and developing accommodation strategies. Students who have not previously contacted Disability Services are encouraged to do so (812-855-7578; http://www.indiana.edu/~iubdss/).
|Aug.||24||Intro to class (.pdf, 2x3.pdf)|
MS, ch. 1
|26||Probability Theory (.pdf, 2x3.pdf)|
|Sep.||2||Information Theory (.pdf, 2x3.pdf)|
MS, 2.2; KS, 2.2
|7||Labor Day, no classes|
|9||Corpora and Linguistic Annotation (.pdf, 2x3.pdf)|
MS, ch. 3, 4
|14||Markov Chains (.pdf, 2x3.pdf)|
|21||N-gram POS tagging (.pdf, 2x3.pdf)|
|23||Smoothing (.pdf, 2x3.pdf)|
JM, 4.5; MS, 6.2
|28||Hidden Markov Models (.pdf, 2x3.pdf)|
MS, 9.2–9.3; KS, 2.1.4
|30||Calculating P(O) (.pdf, 2x3.pdf)|
MS, 9.3.1; KS, 2.1.5
|Oct.||5||Finding the Optimal State Sequence|
MS, 9.3.2; KS, 2.1.6
|7||Parameter Estimation (.pdf, 2x3.pdf)|
MS, 9.3.3; KS, 2.1.7
|12||Topic Modeling (tutorial slides)||HW5 due|
|16||[David Blei talk, SSRC, 2–4pm]|
|19||Sentiment Analysis (tutorial slides)||HW6 due|
|21||Sentiment Analysis (handout)|
|26||CYK parsing (.pdf, 2x3.pdf)|
|28||Probabilistic Context-Free Grammars (.pdf, 2x3.pdf)|
|Nov.||1||PCFGs (.pdf, 2x3.pdf)|
JM, 14.2, 14.4–14.5
MS, ch. 12
|9||Estimating PCFGs (.pdf, 2x3.pdf)|
|16||Practical Parsing (.pdf, 2x3.pdf)||
|18||Word Sense Disambiguation (WSD) (.pdf, 2x3.pdf)|
MS, ch. 7
|23||Thanksgiving break, no classes||
|25||Thanksgiving break, no classes||
|30||Deep learning (tutorial slides)|
|7||Semantic role labeling (SRL)|
|16||Final HW/Project due|
|Final HW due|
Disclaimer This syllabus is subject to change and likely will change. All important changes will be made in writing, with ample time for adjustment.