目前最全面的深度学习教程自学资源汇总

深度学习作为机器学习的一个分支，是近年来最热门同时也是发展最快的人工智能技术之一，相关学习资源包括免费公开教程和工具都极大丰富，同时这也为学习深度学习技术的IT人才带来选择上的困扰，Yerevann整理的这个深度学习完全指南，汇集了目前网络上最优秀的深度学习自学资源，而且会不定期更新，非常值得收藏关注，以下是IT经理网编译整理的指南内容：

自学基本要求（数学知识、编程知识）

数学知识：学员需要具备普通大学数学知识，例如《Deep Learning》一书中若干章节提到的数学概念：

• Deep Learning第二章：线性代数
• Deep Learning第三章：概率与信息理论
• Deep Learning第四章：数值计算

编程知识：你需要懂得编程才能开发和测试深度学习模型，我们建议在机器学习领域首选Python。同时也要用到面向科学计算的NumPy/SciPy代码库。资源链接如下（本文出现的星标代表难度等级）：

• Justin Johnson’s Python / NumPy / SciPy / Matplotlib tutorial for Stanford’s CS231n ★
• Scipy lecture notes – 涵盖了常用的各种库，介绍也比较详细，还涉及一些深入的技术话题 ★★

四大入门教程

如果你具备以上自学基本要求技能，我们建议从以下四大入门在线教程中任选一项或多项组合学习（星标为难度等级）：

• Hugo Larochelle’s video course 这是YouTube上很火的一个深度学习视频教程，录制于2013年，但今天看内容并不过时，很详细地阐释了神经网络背后的数学理论。 幻灯片和相关资料传送门 . ★★
• Stanford’s CS231n (应用于视觉识别的卷积神经网络) 由已经投奔Google的李飞飞教授和 Andrej Karpathy、Justin Johnson共同执教的课程，重点介绍了图像处理，同时也涵盖了深度学习领域的大多数重要概念。 视频 链接(2016) 、 讲义传送门  ★★
• Michael Nielsen的在线著作： Neural networks and deep learning 是目前学习神经网络最容易的教材，虽然该书并未涵盖所有重要议题，但是包含大量简明易懂的阐释，同时还为一些基础概念提供了实现代码。★
• Ian Goodfellow、Yoshua Bengio and Aaron Courville共同编著的 Deep learning是目前深度学习领域最全面的教程资源，比其他课程涵盖的范围都要广。 ★★★

机器学习基础

机器学习是通过数据教计算机做事的科学，同时也是一门艺术，机器学习是计算机科学和数学交汇的一个相对成熟的领域，深度学习只是其中新兴的一小部分，因此，了解机器学习的概念和工具对我们学好深度学习非常重要。以下是机器学习的一些重要学习资源（以下课程介绍部分内容不再翻译）：

• Visual introduction to machine learning – decision trees ★
• Andrew Ng’s course on machine learning, the most popular course on Coursera ★★
• Larochelle’s course doesn’t have separate introductory lectures for general machine learning, but all required concepts are defined and explained whenever needed.
• 1. Training and testing the models (kNN) ★★
• 2. Linear classification (SVM) ★★
• 3. Optimization (stochastic gradient descent) ★★
• 5. Machine learning basics ★★★
• Principal Component Analysis explained visually ★
• How to Use t-SNE Effectively ★★

机器学习的编程学习资料：大多数流行机器学习算法都部署在Scikit-learn 这个Python库中，从头部署算法能够帮我们更好地了解机器学习的工作原理，以下是相关学习资源：

• Practical Machine Learning Tutorial with Python covers linear regression, k-nearest-neighbors and support vector machines. First it shows how to use them from scikit-learn, then implements the algorithms from scratch. ★
• Andrew Ng’s course on Coursera has many assignments in Octave language. The same algorithms can be implemented in Python. ★★

神经网络基础

神经网络是强大的机器学习算法，同时也是深度学习的基础：

• A Visual and Interactive Guide to the Basics of Neural Networks – shows how simple neural networks can do linear regression ★
• 1. Feedforward neural network ★★
• 2. Training neural networks (up to 2.7) ★★
• 4. Backpropagation ★★
• 5. Architecture of neural networks ★★
• 1. Using neural nets to recognize handwritten digits ★
• 2. How the backpropagation algorithm works ★
• 4. A visual proof that neural nets can compute any function ★
• 6. Deep feedforward networks ★★★
• Yes you should understand backprop explains why it is important to implement backpropagation once from scratch ★★
• Calculus on computational graphs: backpropagation ★★
• Play with neural networks! ★

神经网络实操教程

• Implementing softmax classifier and a simple neural network in pure Python/NumPy – Jupyter notebook available ★
• Andrej Karpathy implements backpropagation in Javascript in his Hacker’s guide to Neural Networks. ★
• Implementing a neural network from scratch in Python ★

改进神经网络学习

神经网络的训练可不容易，很多时候机器压根不会学习（underfitting），有时候又“死学”，照本宣科你输入的知识，无法总结归纳出新的数据（overfitting）,解决上述问题的方法有很多，如下是

推荐教程：

• 2.8-2.11. Regularization, parameter initialization etc. ★★
• 7.5. Dropout ★★
• 6 (first half). Setting up the data and loss ★★
• 3. Improving the way neural networks learn ★
• 5. Why are deep neural networks hard to train? ★
• 7. Regularization for deep learning ★★★
• 8. Optimization for training deep models ★★★
• 11. Practical methodology ★★★
• ConvNetJS Trainer demo on MNIST – visualizes the performance of different optimization algorithms ★
• An overview of gradient descent optimization algorithms ★★★
• Neural Networks, Manifolds, and Topology ★★★

常用的主流框架

目前很多深度学习算法都对最新的计算机硬件进行了优化，大多数框架也提供Python接口（Torch除外，需要Lua）。当你了解基本的深度学习算法的部署后，是时候选择一个框架开工了（这部分还可CTOCIO文章：2016年人气最高的六款开源深度学习工具）：

• Theano provides low-level primitives for constructing all kinds of neural networks. It is maintained by a machine learning group at University of Montreal. See also: Speeding up your neural network with Theano and the GPU – Jupyter notebook available ★
• TensorFlow is another low-level framework. Its architecture is similar to Theano. It is maintained by the Google Brain team.
• Torch is a popular framework that uses Lua language. The main disadvantage is that Lua’s community is not as large as Python’s. Torch is mostly maintained by Facebook and Twitter.

There are also higher-level frameworks that run on top of these:

• Lasagne is a higher level framework built on top of Theano. It provides simple functions to create large networks with few lines of code.
• Keras is a higher level framework that works on top of either Theano or TensorFlow.

如果你有框架选择困难症，可以参考斯坦福课程Lecture 12 of Stanford’s CS231n. ★★

卷积神经网络

卷积神经网络Convolutional networks (CNNs)，是一种特定的神经网络，通过一些聪明的方法大大提高了学习速度和质量。卷积神经网络掀起了计算机视觉的革命，并广泛应用于语音识别和文本归类等领域，以下是

推荐教程：

• 9. Computer vision (up to 9.9) ★★
• 6 (second half). Intro to ConvNets ★★
• 7. Convolutional neural networks ★★
• 8. Localization and detection ★★
• 9. Visualization, Deep dream, Neural style, Adversarial examples ★★
• 13. Image segmentation (up to 38:00) includes upconvolutions ★★
• 6. Deep learning ★
• 9. Convolutional networks ★★★
• Image Kernels explained visually – shows how convolutional filters (also known as image kernels) transform the image ★
• ConvNetJS MNIST demo – live visualization of a convolutional network right in the browser ★
• Conv Nets: A Modular Perspective ★★
• Understanding Convolutions ★★★
• Understanding Convolutional neural networks for NLP ★★

卷积神经网络框架部署和应用

所有重要的框架都支持卷积神经网络的部署，通常使用高级函数库编写的代码的可读性要更好一些。

• Theano: Convolutional Neural Networks (LeNet) ★★
• Using Lasagne for training Deep Neural Networks ★
• Detecting diabetic retinopathy in eye images – a blog post by one of the best performers of Diabetic retinopathy detection contest in Kaggle. Includes a good example of data augmentation. ★★
• Face recognition for right whales using deep learning – the authors used different ConvNets for localization and classification. Code and models are available. ★★
• Tensorflow: Convolutional neural networks for image classification on CIFAR-10 dataset ★★
• Implementing a CNN for text classification in Tensorflow ★★
• DeepDream implementation in TensorFlow ★★★
• 92.45% on CIFAR-10 in Torch – implements famous VGGNet network with batch normalization layers in Torch ★
• Training and investigating Residual Nets – Residual networks perform very well on image classification tasks. Two researchers from Facebook and CornellTech implemented these networks in Torch ★★★
• ConvNets in practice – lots of practical tips on using convolutional networks including data augmentation, transfer learning, fast implementations of convolution operation ★★

递归神经网络

递归神经网络Recurrent entworks（RNNs）被设计用来处理序列数据（例如文本、股票、基因组、传感器等）相关问题，通常应用于语句分类（例如情感分析）和语音识别，也适用于文本生成甚至图像生成。

教程如下：

• The Unreasonable Effectiveness of Recurrent Neural Networks – describes how RNNs can generate text, math papers and C++ code ★
• Hugo Larochelle’s course doesn’t cover recurrent neural networks (although it covers many topics that RNNs are used for). We suggest watching Recurrent Neural Nets and LSTMs by Nando de Freitas to fill the gap ★★
• 10. Recurrent Neural Networks, Image Captioning, LSTM ★★
• 13. Soft attention (starting at 38:00) ★★
• Michael Nielsen’s book stops at convolutional networks. In the Other approaches to deep neural nets section there is just a brief review of simple recurrent networks and LSTMs. ★
• 10. Sequence Modeling: Recurrent and Recursive Nets ★★★
• Recurrent neural networks from Stanford’s CS224d (2016) by Richard Socher ★★
• Understanding LSTM Networks ★★

递归神经网络的框架部署与应用

• Theano: Recurrent Neural Networks with Word Embeddings★★★
• Theano: LSTM Networks for Sentiment Analysis ★★★
• Implementing a RNN with Python, Numpy and Theano ★★
• Lasagne implementation of Karpathy’s char-rnn ★
• Combining CNN and RNN for spoken language identification in Lasagne ★
• Automatic transliteration with LSTM using Lasagne ★
• Tensorflow: Recurrent Neural Networks for language modeling ★★
• Recurrent Neural Networks in Tensorflow ★★
• Understanding and Implementing Deepmind’s DRAW Model ★★★
• LSTM implementation explained ★★
• Torch implementation of Karpathy’s char-rnn ★★★

Autoencoders

Autoencoder是为非监督式学习设计的神经网络，例如当数据没有标记的情况。Autoencoder可以用来进行数据维度消减，以及为其他神经网络进行预训练，以及数据生成等。以下课程资源中，我们还收录了Autoencoder与概率图模型整合的一个autoencoders的变种，其背后的数学机理在下一章“概率图模型”中会介绍。

推荐教程：

• 6. Autoencoder ★★
• 7.6. Deep autoencoder ★★
• 14. Videos and unsupervised learning (from 32:29) – this video also touches an exciting topic of generative adversarial networks. ★★
• 14. Autoencoders ★★★
• ConvNetJS Denoising Autoencoder demo ★
• Karol Gregor on Variational Autoencoders and Image Generation ★★

Autoencoder的部署

大多数autoencoders都非常容易部署，但我们还是建议您从简单的开始尝试。课程资源如下：

• Theano: Denoising autoencoders ★★
• Diving Into TensorFlow With Stacked Autoencoders ★★
• Variational Autoencoder in TensorFlow ★★
• Training Autoencoders on ImageNet Using Torch 7 ★★
• Building autoencoders in Keras ★

概率图模型

概率图模型（Probabilistic Graphical model，PGM）是统计学和机器学习交叉分支领域，关于概率图模型的书籍和课程非常多，以下我们收录的资源重点关注概率图模型在深度学习场景中的应用。其中Hugo Larochelles的课程介绍了一些非常著名的模型，而Deep Learning一书有整整四个章节专门介绍，并在最后一章介绍了十几个模型。本领域的学习需要读者掌握大量数学知识：

• 3. Conditional Random Fields ★★★
• 4. Training CRFs ★★★
• 5. Restricted Boltzman machine ★★★
• 7.7-7.9. Deep Belief Networks ★★★
• 9.10. Convolutional RBM ★★★
• 13. Linear Factor Models – first steps towards probabilistic models ★★★
• 16. Structured Probabilistic Models for Deep Learning ★★★
• 17. Monte Carlo Methods ★★★
• 18. Confronting the Partition Function ★★★
• 19. Approximate Inference ★★★
• 20. Deep Generative Models – includes Boltzmann machines (RBM, DBN, …), variational autoencoders, generative adversarial networks, autoregressive models etc. ★★★
• Generative models – a blog post on variational autoencoders, generative adversarial networks and their improvements by OpenAI. ★★★
• The Neural Network Zoo attempts to organize lots of architectures using a single scheme. ★★

概率图模型的部署

高级框架（Lasagne、Keras）不支持概率图模型的部署，但是Theano、Tensorflow和Torch有很多可用的代码。

• Restricted Boltzmann Machines in Theano ★★★
• Deep Belief Networks in Theano ★★★
• Generating Large Images from Latent Vectors – uses a combination of variational autoencoders and generative adversarial networks. ★★★
• Image Completion with Deep Learning in TensorFlow – another application of generative adversarial networks. ★★★
• Generating Faces with Torch – Torch implementation of Generative Adversarial Networks ★★

精华论文、视频与论坛汇总

• Deep learning papers reading roadmap 深度学习重要论文的大清单。
• Arxiv Sanity Preserver 为浏览 arXiv上的论文提供了一个漂亮的界面.
• Videolectures.net 含有大量关于深度学习的高级议题视频
• /r/MachineLearning 一个非常活跃的Reddit分支. 几乎所有重要的新论文这里都有讨论。

本文最后更新于2016年12月26日

问题汇报地址：Github