Paper Reviews on Probabilistic Graphical Models
In this paper, the author described an overview of how to formulate and build graphical models. The author proposed the generic method for model formalism using Box’s loop, An iterative method to estimate posteriors by updating model priors based on the model inference and data.
The author proposed the latent variable representation of the data and also the data generation distribution from the given data. The proposed data generation distribution is marginalized conditional distribution of $z$ (latent representation), $\theta$, and $\mu$.
The author also discussed the formulation, structure, and functioning of multiple graphical latent variable models, namely mixed memberships, linear factors, matrix factors, and hidden Markov and Kalman filter models. By providing a few examples for these models but the theory and mathematical rigor is still missing.
The main problem in graphical models is its inability to scale for large scale datasets, the author tried to demonstrate the method of variational inference to approximate posteriors, which in turn is the central problem in probabilistic modeling.
To address the criticism part defined in the box’s loop, the author describes several methods like predictive sample reuse(PSR) and posterior predictive checks(PPC), which involves likelihood estimation and discrepancy functions respectively to estimate the performance of the model. This information will be used as feedback and update the model as defined in the box’s loop.
To conclude the author describes Box’s loop for model formulation and describes all the modules involved in the box’s loop. This review provides a comprehensive view of latent variable probabilistic models but could be made more mathematical.
This paper briefs the idea of Bayesian networks and outlines the new framework of probabilistic programming. The paper is mainly a review of multiple methods of model-based machine learning. It also discusses the advantages of using model-based learning.
The author also reviews the effective use of probabilistic graphical models over bayesian networks, about how it can be used to reduce the number of parameters, estimate uncertainties, and encode information in prior distributions. various examples of graphical models and their extensions were discussed, which includes HMMs, and it’s extensions like autoregressive HMMs and factor HMMs.
The author discussed the methods of approximate inferencing, and few tricks to reduce the number of computation (factorization methods). The message parsing scheme using non-looping graphical models using the forward-backward algorithm was illustrated in the context of approximate inferencing.
A specific case study on bayesian skill rating was discussed to demonstrate the effect of faster convergence using true skills as compare to Elo.
Finally, probabilistic programming was introduced, which uses Csoft and extension of C#. In conclusion, the paper gives a brief idea about model-based learning and its advantages along with bayesian networks and graphical models.
The paper opens by sketching an age long difference in thoughts between two school of thoughts, namely, ‘Objectivist’ and ‘Subjectivist’. The comparison is painted in the backdrop of model validation and how both these school of thoughts approach this problem and the line of attack taken by them in coming up with a coherent model definition.
Firstly, the paper touches upon the rigorous definition of probability given by A. N. Kolmogorov though the definition is extremely cryptic and highly idealistic it provides a very robust set of rules upon which more complex ideas in probability theory can be built. The author gives a brief glimpse into the mindset of a classical ‘Objectivist’ who subscribe to the frequentist nature of probability. A frequentist tries to build models which imitate the empirical phenomenon, the problem arises when the relationship between the universe described by the model and empirical phenomenon do not show exact correspondence which often the case. Discussing the ‘Subjectivist’ view the contrast becomes evident, here probability is not something manifest in nature but rather your beliefs about the nature which the scholar with a Bayesian mindset tries to capture as ‘degree of belief’. An additional feature of the Bayesian frame of understanding is the heavy reliance on ‘priors’ which upon studying the empirical phenomenon are updated through the rules of probability. The then goes on to critique the Bayesian view by aptly capturing the irrational nature of the world and how the ‘Subjectivist’s in their pursuit of rational world end up getting stuck in circular logic - ‘If you are rational then you will believe in a rational theory’.
Towards the end the author gives examples of a few models (the most famous example chosen being the model for Hooke’s Law). The tries to reconcile the two school of thoughts by equating the Bayesian priors to limits in the Frequentist view. In conclusion the author tries to bring out some seemingly glaring and obvious shortcomings of both the school of thoughts and highlights the need to reconsider the present views be it either Frequentist or Bayesian.
Leave a Comment