CS772: Advanced Natural Language Processing Concepts
Spring 2010


Information
Syllabus
Selected Readings

Suggested Reading List

Language Modeling

[CJ00] C. Chelba and F. Jelinek (2000). Structured language modeling. Computer Speech and Language, 14(4):283-332.

[R01] B. Roark (2001). Probabilistic top-down parsing and language modeling. Computational Linguistics, 27(2):249-276.

[C01] E. Charniak (2001). Immediate-head parsing for language models. Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics.

[C00] E. Charniak (2000). A maximum-entropy inspired parser. Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics.

Search Algorithms

[OUN01] F. Och, N. Ueffing and H. Ney (2001). An efficient A* search algorithm for statistical machine translation. Data-Driven Machine Translation Workshop. 55-62.

[K99] K. Knight (1999). Decoding complexity in word-replacement translation models. Computational Linguistics, 25(4):607-615.

[K04] P. Koehn (2004). Pharaoh: A beam search decoder for phrasebased statistical machine translation models. In Proceedings of AMTA 2004.

[NO00] H. Ney and S. Ortmanns (2000). Progress in dynamic programming search for LVCSR. Proceedings of the IEEE, 88(8):1224-1240.

[TN03] C. Tillmann and H. Ney (2003). Word reordering and a dynamic programming beam search algorithm for statistical machine translation. Computational Linguistics, 29:97-133.

[KM03] D. Klein and C. Manning (2003). A* parsing: Fast exact Viterbi parse selection. HLT-NAACL.

[HC07] L. Huang and D. Chiang (2007). Forest rescoring: faster decoding with integrated language models. ACL.

[H08] L. Huang (2008). A search in the forest: Efficient algorithms for parsing and machine translation based on packed-forests. Dissertation Proposal, University of Pennsylvania. Video

[HM09] Mark Hopkins and Greg Langmead (2009). Cube pruning as heuristic search. EMNLP.

Statistical Machine Translation

[BDDM93] P. Brown, S. Della Pietra, V. Della Pietra, and R. Mercer (1993). The mathematics of statistical machine translation: parameter estimation. Computational Linguistics, 19(2), 263-311.

[ON04] F. Och and H. Ney (2004). The alignment template approach to statistical machine translation. Computational Linguistics, 30(4):417-449.

[ON03] F. Och and H. Ney (2003). A systematic comparison of various statistical alignment models. Computational Linguistics, 29(1):19-51.

[YK01] Yamada and K. Knight (2001). A syntax-based statistical translation model, ACL 2001.

[K03] P. Koehn, F. Och, and D. Marcu (2003). Statistical phrase based translation. Proceedings of the Joint Conference on Human Language Technologies and the Annual Meeting of the North American Chapter of the Association of Computational Linguistics (HLT/NAACL).

[C05] D. Chiang (2005). A hierarchical phrase-Based model for statistical machine translation. Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics.

[C07] D. Chiang (2007). Hierarchical phrase-based translation. Computational Linguistics, 33(2):201-228.

[YK01] Yamada and K. Knight (2001). A decoder for syntax-based statistical MT, ACL 2002.

[W97] D. Wu (1997). Statistical inversion transduction grammars and bilingual parsing of parallel corpora. Computational Linguistics, 23(3).

[N00] H. Ney, S. Nieben, F. Och, H. Sawaf, C. Tillmann and S. Vogel (2000). Algorithms for statistical translation of spoken language. IEEE Transactions on Speech and Audio Processing. 8(1):24-36.

[WW97] Y. Wang and A. Waibel (1997). Decoding algorithm in statistical machine translation. Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics.

Topic and nonparametric Bayesian models

[BNJ03] D. Blei, A. Ng and M. Jordan (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3:993-1022.

[BL05] D. Blei and J. Lafferty (2005). Correlated topic models. Advances in Neural Information Processing Systems 18, 147-154.

[BL06] D. Blei and J. Lafferty (2006). Dynamic topic models. ICML, 113-120.

[J05] M. Jordan (2005). Dirichlet processes, Chinese restaurant processes and all that. Tutorial presentation at the NIPS Conference

[LK07] P. Liang and D. Klein (2007). Bayesian nonparametric structured models. Tutorial presentation at the ACL Conference.

[T06] Y. Teh (2006). A hierarchical Bayesian language model based on Pitman-Yor processes. The Joint 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, (COLING/ACL).