1. Introduction
In the past I have mostly written about ‘classical’ Machine Learning, like Naive Bayes classification, Logistic Regression, and the Perceptron algorithm. In the past year I have also worked with Deep Learning techniques, and I would like to share with you how to make and train a Convolutional Neural Network from scratch, using tensorflow. Later on we can use this knowledge as a building block to make interesting Deep Learning applications.
For this you will need to have tensorflow installed (see installation instructions) and you should also have a basic understanding of Python programming and the theory behind Convolutional Neural Networks. After you have installed tensorflow, you can run the smaller Neural Networks without GPU, but for the deeper networks you will definitely need some GPU power.
The Internet is full with awesome websites and courses which explain how a convolutional neural network works. Some of them have good visualisations which make it easy to understand [click here for more info]. I don’t feel the need to explain the same things again, so before you continue, make sure you understand how a convolutional neural network works. For example,
-
What is a convolutional layer, and what is the filter of this convolutional layer?
-
What is an activation layer (ReLu layer (most widely used), sigmoid activation or tanh)?
-
What is a pooling layer (max pooling / average pooling), dropout?
-
How does Stochastic Gradient Descent work?
The contents of this blog-post is as follows:
-
Tensorflow basics:
-
1.1 Constants and Variables
-
1.2 Tensorflow Graphs and Sessions
-
1.3 Placeholders and feed_dicts
Neural Networks in Tensorflow
-
2.1 Introduction
-
2.2 Loading in the data
-
2.3 Creating a (simple) 1-layer Neural Network:
-
2.4 The many faces of Tensorflow
-
2.5 Creating the LeNet5 CNN
-
2.6 How the parameters affect the outputsize of an layer
-
2.7 Adjusting the LeNet5 architecture
-
2.8 Impact of Learning Rate and Optimizer
Deep Neural Networks in Tensorflow
-
3.1 AlexNet
-
3.2 VGG Net-16
-
3.3 AlexNet Performance
Final words
