We recommend the following tutorials:
We recommend the following books as reading material:
- T Rashid: Make Your Own Neural Network. A gentle hands-on introduction to neural networks (and IPython). Implement a functional neural network and apply it to an industry-standard benchmark for handwritten digits (MNIST). Does not implement deep neural networks.
- J Heaton: Artificial Intelligence for Humans, Volume 3: Deep Learning and Neural Networks. A more complete introduction to neural networks, including deep neural networks. Discusses implementation of multiple benchmarks. Uses pseudocode rather than any actual language. Uses MNIST, IRIS, and several other standard datasets.
- N Lewis: Deep Learning Step by Step with Python. Similar to Heaton, except it uses made-up examples with its own data. Includes snippets of Python code. Extensive footnotes with references to the research literature.
Other hands-on books:
- T Masters: Deep Belief Nets in C++ and CUDA C, Volume I. Has complete working code for DNN training, including GPU CUDA code. A good place to start if you want to write your own CUDA code rather than just use a package. For the course, you would only need to study the section on supervised feedforward networks (Chapter 2). The code is explained in detail. The book only discusses the code. There are no case studies or data. This book is not required for the course because of time constraints relative to the time required to learn CUDA coding and the Nvidia tools, such as DIGITS and CuDNN in addition to the algorithms. However, if you are learning CUDA anyway, this is a good reference. It is the first of a three volume series.
- J Brownlee: Deep Learning with Python. Hands-on lessons and projects. Based on Python, Theano, TensorFlow, and Keras. Nine projects based on benchmarks such as MNIST and CIFAR-10. Covers convolutional neural networks and LSTM recurrent neural networks.