Simple Neural Network in Pytorch

Amazing to see you here! If you have stumbled upon this blog then it means most probably you are starting your journey in deep learning. Yay! I am excited for you. If you are an advanced pytorch user then it will be a refresher for you. So this article is going to help you in coding up a simple neural network in pytorch and understanding some other important features of pytorch.

Screenshot 2019-05-02 at 7.58.10 PM.png


 

Firstly we need pytorch installed on our system. There are many ways to do that. We are going to use the simplest method `pip`!

pip3 install torch torchvision

Great! Now we have the tool installed. Now Let’s get our hands dirty with coding.

Firstly we would need to import necessary modules –

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim

torch.nn contains all the necessary tools we would need while coding up a neural network like Linear Layers, RNNs etc. I am going to write a blog soon explaining torch.nn in detail so please make sure that you follow the blog(shameless plug) via email. Meanwhile this awesome article by Jeremy Howard might help.

Next step is to get some data with which we can play or train the neural network.


We take advantage of `torch.tensor` for creating a simple data.

I am taking a few data points from iris dataset . Here X represents the petal length, petal width and sepal length. Y is the sepal width. Basically for given petal length, petal width and sepal length, can we predict sepal width? This is the problem.

xInput is the test data which we will pass to the neural network.

X = torch.tensor(([5.1, 3.5, 1.4], [6.7, 3.1, 4.4], [6.5, 3.2, 5.1]), dtype=torch.float) # 3 X 3 tensor
y = torch.tensor(([0.2], [1.4], [2]), dtype=torch.float) # 3 X 1 tensor
xInput = torch.tensor(([5.9, 3, 5.1]), dtype=torch.float)

Note – torch.size() is helpful for checking size of a tensor.

Now let us define a simple Neural Network. “nn.Module” is a base class for all the neural network models in pytorch. Our neural network will have two fully connected or Dense or Linear layers. It will have 3 input neuron, 10 neurons (in hidden layer) and 1 output neuron.

One thing to keep a note of is that always think neural network as composition of functions. The thing about pytorch is that it allows us express this idea beautifully.

class Neural_Network(nn.Module):

    def __init__(self):
        super(Neural_Network, self).__init__()
        self.fc1 = nn.Linear(3, 10)
        self.fc2 = nn.Linear(10, 1)

    def forward(self, x):
        x = F.sigmoid(self.fc1(x))
        x = F.sigmoid(self.fc2(x))
        return x

see how simple it was to define neural network in pytorch.

The following piece of code will help you to train and predict –

model = Neural_Network()

model.train()

optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

for i in range(1000):

    optimizer.zero_grad()
    output = model(X)
    loss = F.binary_cross_entropy_with_logits(output, Y)
    print loss
    loss.backward()
    optimizer.step()

print model(xInput)

We are using SGD(Stochastic Gradient Descent) for optimization of Binary Cross Entropy loss function. I won’t go into details of these things in blog.


That is all! Less than 30 lines of code to write your first neural network in pytorch. However, keep in mind that the above code and neural network architecture is not meant to perform optimally. It is just an example!

I hope this blog helps you to start with pytorch. Please comment your views.

Happy Coding!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s