A GENTLE TUTORIAL ON DEVELOPING GENERATIVE PROBABILISTIC MODELS AND DERIVING GIBBS SAMPLING A CASE STUDY ON LDA

We present a tutorial on the basics of Bayesian probabilistic modeling and Gibbs sampling algorithms for data analysis. Particular focus is put on explaining detailed steps to build a probabilistic model and to derive Gibbs sampling algorithm for the model. The tutorial begins with basic concepts that are necessary for understanding the underlying principles and notations often used in generative models. Latent Dirichlet Allocation (LDA) is then explained in details regarding both steps to build the model and to derive its collapsed Gibbs sampling algorithm. Following this LDA case study one can further develop either simple or more complex generative models for a variety of applications.