Probabilistic Approaches to Language Learning and Processing

At the engineering end of speech and natural language understanding research, the field has been transformed by the adoption of Bayesian probabilistic approaches with generative models such as Markov models, hidden Markov models, and probabilistic context-free grammars being standard tools of the trade, and people increasingly using more sophisticated models. More recently, there has also started to be use of these models as cognitive models to explore issues in psycholinguistic processing and how humans approach the resolution problem of combining evidence from numerous sources during the course of processing. Much of this work has been in a supervised learning paradigm where models are built from hand-annotated data; but probabilistic approaches also open interesting new perspectives on formal problems of language learning. ]

After surveying the broader field of probabilist approaches in natural language processing, I'd like to focus in on unsupervised approaches to learning language structure, show why it's a difficult problem, and present some recent work that I and others have been doing using probabilistic models, which shows considerable progress on tasks such as word class and syntactic structure learning.