Bengio and Lecun created this wonderful video on Deep Neural Networks.  Any logical function can be represented by a neural net with 3 layers (one hidden, see e.g. CNF), however simple 4 level logical functions with a small number of nodes may require a large number of nodes in a 3 layer representation.  They point to theorems that show that the number of nodes required to represent a k level logical function can require an exponential number of nodes in a k-1 level network. They go on to explain denoising auto encoders for the training of deep neural nets.

Related Posts via Categories

• AlphaGo is changing how the Game is Played
• Newly Published “Deep Learning” Book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
• Link: AlphaGo Wins against one of the best Go Players on the Planet
• Art and Artificial Intelligence by G. W. Smith
• List of Papers for Carnegie Mellon’s Deep Learning Course
• A short review of “Teaching Deep Convolutional Neural Networks to Play Go”
• DeepLearning.net’s Reading List
• The Best Deep Learning Blog Post Ever (Christopher Olah)
• Linear-Nonlinear-Poisson Neurons
• “Resurgence in Neural Networks”