Build Your Own Neural Network in Go

A beginner’s guide to building a simple neural network completely from scratch in Go language

Introduction

In this tutorial, we’ll build a simple neural network (single-layer perceptron ) in Golang, completely from scratch. We’ll also train it on sample data and perform predictions. Creating your own neural network from scratch will help you better understand what’s happening inside a neural network and the working of learning algorithms.

What’s a Perceptron?

Perceptrons — invented by Frank Rosenblatt in 1958, are the simplest neural network that consists of n number of inputs, only one neuron and one output , where n is the number of features of our dataset.

Hence, our single-layer perceptron consists of the following components

1. An input layer (x)
2. An output layer (ŷ)
3. A set of weights (w) and a bias (b) between these two layers
4. An activation function (σ) for the output layer. In this tutorial, we’ll be using the sigmoid activation function.

Our neural network is called a single-layer perceptron (SLP) as the neural network has only one layer of neurons. Neural networks with more than one layer of neurons are called multi-layer perceptron (MLP) .

(note epoch refers to one cycle through the full training dataset)

Before we start

We’ll build our own functions for the following math operations — vector addition , vector dot product & scalar matrix multiplication.

Initially, the weights of the neural network are set to random float values between 0 and 1 while the bias is set to zero.

Forward Propagation

The process of passing the data through the neural network is known as forward-propagation or forward pass. The output the perceptron is

In a nutshell, the dot product of the weight vector (w) and the input vector (x) is added with the bias ( b) and the sum is passed through an activation function. The output of the sigmoid activation function will be from 0 and 1.

The Learning Algorithm

The learning algorithm consists of two parts — Backpropagation and Optimization.

Backpropagation, short for backward propagation of errors , refers to the algorithm for computing the gradient of the loss function with respect to the weights. However, the term is often used to refer to the entire learning algorithm.

A loss function is used to get an estimation of how far are we from our desired solution. Generally, mean squared error is chosen as the loss function for regression problems and cross-entropy for classification problems. To keep it simple, we’ll use mean squared error as our loss function. Also, we will not be calculating the MSE but directly calculate its gradient.

The gradient of the loss function is calculated using the chain rule . The gradients of the loss function with respect to the weights and bias are calculated as follows.

(for the derivation of these expressions, check my article in which I have briefly explained the math concepts behind neural networks )

Optimizationis the selection of best weights and bias of the perceptron to get the desired results. Let’s choose gradient descent as our optimization algorithm. The weights and the bias are updated as follows till convergence .

Learning rate ( α ) is a hyperparameter which is used to control how much the weights and bias are changed. However, we will not be using the l earning rate in this tutorial.

Assembling the Pieces

Now let’s train and make predictions out of our neural network on the following data. The data has three inputs and only one output belonging to two classes(0 and 1). Hence the data can be trained on our single-layer perceptron.

As you can see, the output Y is only dependent on the input X1 . Now we will train our neural network on the above data and check how it performs after 1000 epochs . To make predictions, we have to just do a forward propagation with the test inputs.

As we compare the predicted values with the actual values, we can see that our trained single-layer perceptron has performed well. We’ve successfully created a neural network and trained it to produce desirable results.

What’s Next?

Now you’ve created your own neural network completely from scratch. Here are a few things you shall try next.

• Test on your own data
• Try other activation function besides the sigmoid function
• Calculate MSE after each epoch
• Try other error function besides the MSE
• Try creating a multi-layer perceptron

Final Thoughts

I hope that you’ve learned a lot from creating your own neural network in Golang. I’ll be writing more on topics related to machine learning & deep learning and I’ll be hopefully covering multi-layer perceptron in my next article.