Statistical analysis of Lingustic data

The main books will be:

• Speech and Language processing by Jurafsky and Martin. (main text)
• Foundations of statistical natural language processing by Manning and Schutze. (Suggested if you have a weaker statistics background.)

Homework:

Schedule:

• Introduction (.pdf)

• Regular expressions (.pdf)
  • Chapter 2 of JM,
  • chapter 2 and chapter 4 of the Orily book on Regular Expressions.
• Ngrams (.pdf)
  • N-grams (Chapter 4 of JM)

• Backoff (.pdf) and information theory.
• Part of Speach tagging
  • Chapter 5 of JM
  • standford tagger
  • Penn tree bank

• HMM (.pdf) (chapter 6)
  • HMM (Chapter 6 of JM)
  • Kalman filters
  • Estimation via CCA (see Sham's paper)
• HMM (part 2) (.pdf)

• Speech: encoding (.pdf) (chapters 7 and 8)
  • Limits on D to A: Shannon limit
  • Limits on A to D: Nyquist rate
  • IPA (fancy alphabet: read along as various people speak words.)
• Speech: decoding (.pdf) (chapters 9 and 10)

• Context Free Grammars
  • .pdf
  • read chapters 12 and 16.of JM

• Parsing
  • .pdf
  • read chapters 13 and 14 of JM.
• Statistical parsing
  • .pdf
  • ``Parameter Estimation for Statistical Parsing Models: Theory and Practice of Distribution-Free Methods'' by Michael Collins.

• First Order Predicate Calculus
  • .pdf
  • Read chapters 17 and 18
• Word sense disambiguation
  • .pdf
  • Read chapters 19 and 20

• CCA
  • slides for today's lecture
  • paper with Sham
  • CCA goes back to the 1930's, so there should be pleanty of web material to look over. I won't put it up. But if you find something nice, email it to me and I'll post it.
• Other disambiguation solutions
  • pdf
  • Read the rest of chapter 20

• Nov 9th: Look at distributional vectors of words

• Nov 16th: NER
• Nov 18th: mFDR
  • lecture notes in .pdf (as given at JSM in 2008)
  • mFDR martingal paper: Martingale
  • We started this research studying bankruptcy.
  • A good introduction to the ideas of risk ratios (Dongyu Lin)
  • The original Risk inflation paper (with Edward George)

• Nov 23rd: Proof of martingale for mFDR

• Nov 30th: NO CLASS! GO SEE DAN ROTH!!
• Dec 2nd: Question answering
  • notes

• Dec 7th: Double lecture
• First lecture: Translation (chapter 25)
• Second lecture: Extensions to streaming feature selection (3-4:30)
  • JASA paper: VIF (Dongyu Lin)
  • JMLR paper: auction
• Dec 9th: in class presentations
• Dec 10th: out of class presentations (maybe?)