Check out Markus Beissinger’s blog post “Deep Learning 101″. Markus reviews a lot of deep learning basics derived from the papers “Representation Learning: A Review and New Perspectives” (Bengio, Courville, Vincen 2012) and “Deep Learning of Representations: Looking Forward” (Bengio 2013). Beissinger covers the following topics:

- An easy intro to Deep Learing
- The Current State of Deep Learing
- Probabilistic Graphical Models
- Principal Component Analysis
- Restricted Boltzman Machines
- Auto-Encoders
- “Challenges Looking Ahead”

This is a great intro and I highly recommend it.

If you want more information, check out Ng’s lecture notes, Honglak Lee’s 2010 NIPS slides, and Hinton’s Videos ([2009] [2013]).