Bayesian Methods for Unsupervised Learning

Many models used in machine learning and neural computing can be understood within the unified framework of probalistic graphical models. These include clustering models (k-means, mixture of Gaussians), dimensionality reduction models (PCA, factor analysis), time series models (hidden Markov models, linear dynamical systems), independent component analysis (ICA), hierarchical neural network models, etc.

I will review the link between all these models and the framework for learning them using the EM algorithm for maximum likelihood. I will then describe limitations of the maximum likelihood framework and how Bayesian methods overcome these limitations, allowing learning without overfitting, principled model selection, and the coherent handling of uncertainty.

Time permitting, I will describe the computational challenges of Bayesian learning and approximate methods for overcoming those challenges, such as variational methods.