“A probabilistic programming language is a high-level language that makes it easy for a developer to define probability models and then “solve” these models automatically.  These languages incorporate random events as primitives and their runtime environment handles inference. Now, it is a matter of programming that enables a clean separation between modeling and inference.”

writes Beau Cronin in this post about Probabilistic Programming (PP) (see e.g. [1] and [2]).  He goes on to informally describe a probabilistic graphical model (PGM) and how PP languages or extensions to existing languages like BLOG, BUGS, Church (Lisp), FACTORIE, Figaro (Scala), HANSEI (Ocaml), Hierarchical Bayesian Compiler, Infer.NET, ProbLog, and Stan make it much easier to set up and solve PGMs.  He provides links to a tutorial, a great easy-to-understand video, the NIPS workshop on PP, and several ongoing PP projects.

I also enjoyed the more detailed post “Why Probabilistic Programming Matters” by Rob Zinkov.  Rob shows how to represent the following machine learning techniques in a PP language:

• Bayesian Linear Regression
• Logistic Regression
• Naive Bayes
• K-Means Clustering
• Latent Dirichlet Allocation (LDA)
• Correlated Topic Models (CTM)
• Autoregressive Integrated Moving Average (ARIMA)
• Hidden Markov Models (HMM)
• Matrix Factorization
• Sparsity and Sparse Bayes
• Conditional Random Fields (CRF)

Related Posts via Categories

• “Multi-stage Markov Chain Monte Carlo Methods for Porous Media Flows”
• Review: “Basic Category Theory for Computer Scientists”
• “Deep Learning 101″
• Yann Esposito’s Category theory Slides with Haskell
• Searching a Game Tree with a GPU
• Simpson’s paradox and Judea Pearl’s Causal Calculus
• “Stochastic Superoptimization” and “Programming by Optimization”
• “Poor Man’s Agile: Scrum in 5 Simple Steps”
• “Recent Algorithms Development and Faster Belief Propagation algorithms”
• “The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo”