Carl, I, and the Wikipedia put together this list.

Wikipedia list

• Theory
• • Kolmogorov complexity theory
  • Minimum description length (MDL)
  • Compression
  • Bias-variance dilemma
  • On-line learning methods
  • Cross-validation
  • Ensemble learning
    • Bagging / bootstrapping
    • Boosting
    • Mixture of Experts
  • ROC curve
  • VC dimension
  • PAC learning

• Information theory
  • Noisy-channel coding
  • Entropy
  • Maximum Entropy Method
  • KL distance
  • Compressed Sensing
  • Mutual Information

• Learning
  • Reinforcement Learning
  • Supervised Learning
  • Semi-Supervised Learning
  • Online Learning
  • Multi-Armed Bandit
  • Active Learning
  • Manifold Learning

• Optimization Algorithms
  • Expectation-Maximization
  • Simulated annealing
  • Hill climbing
  • (Stochastic) Gradient Descent
  • Conjugate Gradient
  • Boosting algorithms
  • Branch and bound algorithms
  • Dynamic programming
  • Genetic algorithms
  • Particle swarms
  • A* Search
  • Bayesian Optimization

• Markov Methods & Probabilistic Graphical Models
  • Viterbi / Baum-Welch (algorithms)
  • Hidden Markov models
  • Message Passing, Loopy Belief Propagation
  • Forward-backward algorithm / sum-product algorithm
  • Conditional Random Fields
  • Markov Random Fields
  • Expectation propagation
  • Monte-Carlo Markov chain methods
    • Hastings-Metropolis
    • Gibbs Sampling

• Neural nets
  • Boltzmann machine, Restricted Boltzmann machines
  • Deep Belief Networks (Hinton1, Hinton2, Drop out)
  • Sparse Deep Belief Networks and Autoencoders

• Pattern Recognition
  • Naive Bayes
  • Support vector machines
  • Decision trees (CART)
  • Predictive Analytics
  • Kernel methods (kernelized support vector machine)
  • Kernal Density Estimation (Parzen Windows)
  • Random Forests
  • Structured Prediction
  • Dimension Reduction
    • LDA & QDA (Linear and Quadratic Discriminant Analysis)
    • PCA
    • ICA
    • Self-organizing maps. (Kohonen)
    • Latent Dirichlet Allocation
  • Curve fitting
    • Kriging
    • Linear, Logit, Probit, Non-linear, Lasso, Ridge Regression
    • Locally weighted Regression
    • Generalized Linear Model

• Games
  • Q-learning
  • Upper Confidence Trees
  • Regret Matching
  • Fictitious Play
  • TD-learning
  • Markov Decision Process

• Clustering
  • Mixture of Models (Mixture of Gaussians)
  • K-means
  • K-center
  • Hiearchial (Dendritic) clustering
  • Collaborative Filtering, Singular Value Decomposition

• Misc
  • Variational methods
  • Deconvolution
  • Relevance vector machine (RVM)
  • Regularization

Related Posts via Categories

• AlphaGo is changing how the Game is Played
• Counterfactual Regret Minimization
• Richardson Extrapolation
• University of Alberta Solves Poker
• 179 classifiers competing on 121 data sets and the winner is …
• A review of “Playing Atari with Deep Reinforcement Learning”
• A Review of “Knowledge Vault: A Web-Scale Approach to a Probabilistic Knowledge Fusion”
• Rules for CS Reseach and Seven Principles of Learning
• DeepLearning.net’s Reading List
• Gunnar Carlsson on the Shape of Data