Build Deeper

Welcome to the resource page of the book Build Deeper: The Path to Deep Learning.

Please find below the code samples, diagrams, and reference links for each chapter.

Chapter 1 - What is Deep Learning?

The History of Deep Learning

Chapter 2 - Milestones of Deep Learning

AlexNet

The AlexNet Architecture

Image source: Research Paper - ImageNet Classification with Deep Convolutional Neural Networks


ZF Net

The ZF Net Architecture

Image source: Research Paper - Visualizing and Understanding Convolutional Networks


VGG Net

The VGG Net Architecture

GoogLeNet

The GoogLeNet Architecture

Image source: Research Paper - Going Deeper with Convolutions


Microsoft ResNet

The ResNet Architecture

Image source: Research Paper - Deep Residual Learning for Image Recognition


DenseNet

DenseNet Architecture

A Dense Block

Image source: Research Paper - Densely Connected Convolutional Networks

Chapter 4 - How to Set Them Up

Links:

• Anaconda Downloads page - https://www.anaconda.com/download/
• Anaconda Package Lists - https://docs.anaconda.com/anaconda/packages/pkg-docs
• Anaconda Test Drive - https://conda.io/projects/conda/en/latest/user-guide/getting-started.html
• Installing OpenCV from source on Anaconda Python on Ubuntu - http://www.codesofinterest.com/2017/01/installing-opencv-source-ubuntu.html
• Dlib official website - http://dlib.net
• OpenBLAS pre-built binaries for Windows - https://sourceforge.net/projects/openblas/files/v0.2.15/
• Getting Theano working with OpenBLAS on Windows - http://www.codesofinterest.com/2016/10/getting-theano-working-with-openblas-on.html
• CUDA supported GPUs List - https://developer.nvidia.com/cuda-gpus
• NVIDIA CUDA Downloads page - https://developer.nvidia.com/cuda-downloads
• cuDNN Downloads Page - https://developer.nvidia.com/cudnn

Setting up your environment - Video

Chapter 5 - Build Your First Deep Learning Model

MNIST Official Website - http://yann.lecun.com/exdb/mnist/

The LeNet Architecture

The LeNet Architecture

The complete code for Chapter 5: https://github.com/Thimira/Build-Deeper/tree/master/Chapter%205

Chapter 6 - Looking Under the Hood

Graphviz downloads - https://graphviz.gitlab.io/download/

The Structure Visualization of the LeNet Model

Visualization with Both show_shapes and show_layer_names off

The complete code for Chapter 6: https://github.com/Thimira/Build-Deeper/tree/master/Chapter%206

Chapter 7 - What Next?

Code for VGG16 model using Keras Applications - https://gist.github.com/Thimira/c369aea98c4268042425649a6a687d8f

Code for ResNet50 model using Keras Applications - https://gist.github.com/Thimira/6dc1da782b0dca43485958dbee12a757

Deep Learning Models Repository - https://github.com/fchollet/deep-learning-models

• VGG16 - https://github.com/fchollet/deep-learning-models/blob/master/vgg16.py
• VGG19 - https://github.com/fchollet/deep-learning-models/blob/master/vgg19.py
• ResNet50 - https://github.com/fchollet/deep-learning-models/blob/master/resnet50.py
• InceptionV3 - https://github.com/fchollet/deep-learning-models/blob/master/inception_v3.py
• Xception - https://github.com/fchollet/deep-learning-models/blob/master/xception.py
• MobileNet - https://github.com/fchollet/deep-learning-models/blob/master/mobilenet.py

Deep Learning Releases page for the Weights files - https://github.com/fchollet/deep-learning-models/releases

Chapter 8 - Build Our Own Image Classifier with Transfer Learning

How Bottleneck Feature Extraction Works

Fine-Tuning Our Model

The complete code for Chapter 8: https://github.com/Thimira/Build-Deeper/tree/master/Chapter%208

Chapter 9 - Bonus – Getting Started with Computer Vision

The complete code for Chapter 9: https://github.com/Thimira/Build-Deeper/tree/master/Chapter%209

References and Useful Links

[1]. Deep Learning Installation Guides at Codes of Interest - http://www.codesofinterest.com/search/label/Installation
[2]. Troubleshooting Guides at Codes of Interest - http://www.codesofinterest.com/search/label/Troubleshooting
[3]. Deep Learning and Computer Vision Tutorials at Codes of Interest - http://www.codesofinterest.com/search/label/Tutorial
[4]. More Deep Learning Resources - http://www.codesofinterest.com/p/resources.html
[5]. AlexNet Research Paper - https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf
[6]. ZF Net Research Paper - https://arxiv.org/abs/1311.2901
[7]. GoogleNet Research Paper - https://arxiv.org/abs/1409.4842
[8]. ResNet Research paper - https://arxiv.org/abs/1512.03385
[9]. The 1000 Layer ResNet Model - https://github.com/KaimingHe/resnet-1k-layers
[10]. DenseNet GitHub Page - https://github.com/liuzhuang13/DenseNet
[11]. DenseNet Research Paper - https://arxiv.org/pdf/1608.06993.pdf
[12]. Large Scale Visual Recognition Challenge (ILSVRC) - http://www.image-net.org/challenges/LSVRC/
[13]. DeepMind AplphaGo - https://deepmind.com/research/alphago/
[14]. AlphaGo (Wikipedia) - https://en.wikipedia.org/wiki/AlphaGo
[15]. Go Game (Wikipedia) - https://en.wikipedia.org/wiki/Go_(game)
[16]. AlphaGo versus Lee Sedol - https://en.wikipedia.org/wiki/AlphaGo_versus_Lee_Sedol
[17]. AlphaGo Master Series - https://deepmind.com/research/alphago/match-archive/master/
[18]. AlphaGo versus Ke Jie- https://en.wikipedia.org/wiki/AlphaGo_versus_Ke_Jie
[19]. AlphaGo Zero: Learning from Scratch - https://deepmind.com/blog/alphago-zero-learning-scratch/
[20]. AlphaGo Zero (Wikipedia) - https://en.wikipedia.org/wiki/AlphaGo_Zero
[21]. AlphaZero (Wikipedia) - https://en.wikipedia.org/wiki/AlphaZero
[22]. AlphaZero Research Paper - https://arxiv.org/abs/1712.01815
[23]. AlphaZero vs. Stockfish - https://www.chess.com/news/view/updated-alphazero-crushes-stockfish-in-new-1-000-game-match
[24]. Stockfish Chess Engine - https://en.wikipedia.org/wiki/Stockfish_(chess)
[25]. AlphaZero vs. Stockfish Analysis - https://deepmind.com/blog/alphazero-shedding-new-light-grand-games-chess-shogi-and-go/
[26]. OpenAI Homepage - https://openai.com/
[27]. OpenAI (Wikipedia) - https://en.wikipedia.org/wiki/OpenAI
[28]. OpenAI Dota 2 Bot - https://blog.openai.com/dota-2/
[29]. More on Dota 2 Bot - https://blog.openai.com/more-on-dota-2/
[30]. Dota 2 Bot in Action - https://www.teslarati.com/openai-self-play-dota-2-musk/
[31]. OpenAI Five - https://openai.com/five/
[32]. OpenAI Five – OpenAI Blog - https://blog.openai.com/openai-five/
[33]. OpenAI Five Benchmark - https://blog.openai.com/openai-five-benchmark/
[34]. OpenAI Five Benchmark Results - https://blog.openai.com/openai-five-benchmark-results/
[35]. OpenAI 5v5 (YouTube) - https://www.youtube.com/watch?v=eaBYhLttETw
[36]. OpenAI Five wins 5v5 - https://www.theverge.com/2018/8/6/17655086/dota2-openai-bots-professional-gaming-ai
[37]. Difference Between Deep Learning Training and Inference - https://blogs.nvidia.com/blog/2016/08/22/difference-deep-learning-training-inference-ai/
[38]. AWS GPU-backed EC2 Instances - https://aws.amazon.com/ec2/instance-types/p3/
[39]. Anaconda Python Homepage - https://www.anaconda.com
[40]. Anaconda Getting Started Guide - https://conda.io/docs/user-guide/getting-started.html
[41]. OpenCV Homepage - http://opencv.org/
[42]. Dlib Homepage - http://dlib.net/
[43]. Theano Homepage - http://www.deeplearning.net/software/theano/
[44]. Keras Homepage - https://keras.io/
[45]. Switching between TensorFlow and Theano on Keras - http://www.codesofinterest.com/2016/11/switching-between-tensorflow-and-theano.html
[46]. What is the image_data_format parameter in Keras, and why is it important - https://www.codesofinterest.com/2017/09/keras-image-data-format.html
[47]. image_data_format vs. image_dim_ordering in Keras v2 - http://www.codesofinterest.com/2017/05/image-data-format-vs-image-dim-ordering-keras-v2.html
[48]. TensorFlow Homepage - https://www.tensorflow.org/
[49]. OpenBLAS Homepage - http://www.openblas.net/
[50]. Getting Theano working with OpenBLAS on Windows - http://www.codesofinterest.com/2016/10/getting-theano-working-with-openblas-on.html
[51]. NVIDIA CUDA Homepage - https://developer.nvidia.com/cuda-toolkit
[52]. cuDNN Homepage - https://developer.nvidia.com/cudnn
[53]. Fixing the Matplotlib PyPlot import errors - https://www.codesofinterest.com/2018/05/fixing-matplotlib-pyplot-import-errors.html
[54]. The Original NIST Database - https://www.nist.gov/sites/default/files/documents/srd/nistsd19.pdf
[55]. The MNIST Database - http://yann.lecun.com/exdb/mnist/
[56]. The LeNet Model - http://yann.lecun.com/exdb/lenet/
[57]. Graphviz Homepage - https://graphviz.gitlab.io/
[58]. Graphviz Dot Language - https://graphviz.gitlab.io/_pages/doc/info/lang.html
[59]. Pydot NG Package - https://pypi.python.org/pypi/pydot-ng
[60]. Keras Model Visualization API - https://keras.io/visualization/
[61]. An Intuitive Explanation of Convolutional Neural Networks - https://ujjwalkarn.me/2016/08/11/intuitive-explanation-convnets/
[62]. Convolution (Wikipedia) - https://en.wikipedia.org/wiki/Convolution
[63]. Keras Functional API - https://keras.io/getting-started/functional-api-guide/
[64]. Keras Sequential Model - https://keras.io/getting-started/sequential-model-guide/
[65]. Keras Applications - https://keras.io/applications/
[66]. Keras Image Pre-processing Options - https://keras.io/preprocessing/image/
[67]. Using Data Augmentations in Keras - https://www.codesofinterest.com/2018/02/using-data-augmentations-in-keras.html

Appendix II - Code

All code samples discussed in the book can be accessed at the dedicated code repository at GitHub: https://github.com/Thimira/Build-Deeper