Pytorch Code - Block by Block (AI/ML Guide for Biologists - (iiib))

Pytorch Code - Block by Block (AI/ML Guide for Biologists - (iiib))

This is a follow-up of my series on AI/ML for biologists. If you like to receive early information, please sign up here.

In this part of the lesson, we will go over the Pytorch code for linear regression. We started that process in part (a) of this lesson, and gave you a homework. I am sure none of you did it :)

We will postpone our understanding of how all three different looking codes from Numpy, Scikit-learn and Pytorch solve the same problem to the next lesson. That discussion will also help you better understand the role of calculus.

Scalars, Vectors, Matrices and Tensors

You may have come across scalars, vectors and matrices in your math classes. AI literature introduces another mathematical term - tensors. Actually it is just a generalization of what you already know.

A scalar is a single number (e.g. 1, 10, 97).

A vector is collection of numbers = [1, 2, 3, 4, 5, 6]

A matrix is a collection of vectors of same size = [[1, 2, 3,4 ,5 ,6], [10, 9, 8, 7, 6, 5]]

A tensor is a collection of matrices of same size. In fact, Pytorch describes all data categories like scalars, vectors and matrices as “tensors” of different dimensions. You can understand how that works from the following code.

import torch

# Single number (scalar)
torch.tensor([1])

# Vector
torch.tensor([1., 2., 3.])

# Matrix
torch.tensor([[1., 2.], [3., 4.]])

# Higher dimension
torch.tensor([[[1., 2.], [3, 4.]], [[5., 6.], [7., 8.]]])

# Here is a tensor of zeroes of dimension 4, 2, 3
torch.zeros([4, 2, 3], dtype=torch.float)

# Here is a tensor of ones of dimension 4, 2, 3
torch.ones([4, 2, 3], dtype=torch.float)

Pytorch Code for Linear Regression - Block by Block

The Pytorch code for linear regression (lesson 2) may appear large and complex at first, but once you recognize the blocks, you will start to identify them easily in all other places. That is why I am spending so much time on it.

1. Import block

Just like you import numpy or pandas, here you import “torch” (Pytorch). Also, you import “Variable” to describe the tensors.

import torch
from torch.autograd import Variable

2. Defining x and y

x_data = Variable(torch.Tensor([[1.0], [2.0], [3.0]]))
y_data = Variable(torch.Tensor([[2.0], [4.0], [6.0]]))

3. Creating the Neural Network Model

All neural network codes start with some sort of model. The “intelligence” of artificial intelligence is hidden within the parameters of this model, and the goal is to estimate the parameters from x and y. The papers show the models as nice drawings of blocks connected by wires, but understanding how it is implemented in code is way more powerful. In the bonus sections, we will look into the actual codes for the models of chatGPT (nanoGPT) and protein-folding programs.

class LinearRegressionModel(torch.nn.Module):

    def __init__(self):
        super(LinearRegressionModel, self).__init__()
        self.linear = torch.nn.Linear(1, 1)  # One in and one out

    def forward(self, x):
        y_pred = self.linear(x)
        return y_pred

# our model
our_model = LinearRegressionModel()

4. Definining Error and Optimizer

How do we estimate the parameters of the model? You need some kind of “error definition” to rank various incarnations of the model. That is the first line below.

The second line shows how to optimize to reach the best set of parameters. This is where calculus is hidden, and we will talk about it in the next lesson.

criterion = torch.nn.MSELoss(size_average = False)
optimizer = torch.optim.SGD(our_model.parameters(), lr = 0.01)

5. Fitting the parameters of the model

We have everything ready to start estimating the parameters. This is the real work, and it is called “training the model”.

In the following code, we start with some set of initial parameters and find what predicted y is from the given x.

Then we determine how bad the prediction is from the the actual numbers. That is the “loss” number.

Finally we use this loss to recalculate the parameters of the model. The goal is to make this loss as small as possible.

Those three steps above are repeated 500 times, and each iteration is called an “epoch”.

for epoch in range(500):

    # Forward pass: Compute predicted y by passing
    # x to the model
    pred_y = our_model(x_data)

    # Compute and print loss
    loss = criterion(pred_y, y_data)

    # Zero gradients, perform a backward pass,
    # and update the weights.
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print('epoch {}, loss {}'.format(epoch, loss.item()))

6. Predicting y for a new x

Now that the best model parameters are estimated, it is time to estimate y for a new x (x = 4). This is known as “inference”.

new_var = Variable(torch.Tensor([[4.0]]))
pred_y = our_model(new_var)
print("predict (after training)", 4, our_model(new_var).item())

Once you understand this entire process, you will find that all neural network programs go through similar steps. They have start with different forms of training data. For protein-folding, “x” is the protein sequences and “y” is its corresponding structure. The cleverness is in coming up with the appropriate model, but the rest of the process stays similar. Finally, after training the model (i.e. estimating its parameters), you give it a new protein sequence with unsolved structure and the model predicts its structure.

Pytorch Code from Statquest

Statquest explaines the Pytorch code of a different problem, and it received raving praise in the comment section.

Please continue to go through every single line of code including the parameters with excruciating detail like you do. None of my professors went over each line like that cuz they always “assumed we already knew” and everyone in the class who didnt already know was afraid to ask to avoid looking stupid. Thank you.”

For better understanding, please review that as well.


Written by M. //