unit 0.1 - What is machine learning?

The course teaches machine learning with neural networks, which are the revolutionary tool powering many product and solutions (internet search, message filtering, social-network algorithms, ChatGPT, Gemini, DALL-E, etc.). Neural networks is just one of many machine learning algorithms, but it has proven to be one of the most effective to date and possibly the only one to learn if you have to chose one!

Machine learning at the core is the set of scientific tools we use to power computer with comprehension of tasks, functions, applications based on a set of examples. Examples are a set of \((inputs, desired_outputs)\) pairs, which are used to train computer algorithms. These algorithms are what we call “machine learning”.

Functions and machine-learning

If all we want to is to learn a relationship between inputs and outputs we could just use equations! No need for anything fancier, or any machine-leanrning. After all functions can provide an output \(y\) for every input \(x\).

Have functions? …

Suppose you want to find an equation for a problem:

PROBLEM 1 - time for an object to fall from a table 1m tall

Here there is an equation and we can use it!

The interesting thing is - how was this equation discovered?

… or do not have functions?

Now suppose we want to find the equation for another problem:

PROBLEM 2 - is this a picture of a cat?

In this case, there is no equation…

But why?

a fancier dataset

CIFAR 10 dataset - I give you an image, you tell me what it is about (category).

This is another example of small images and the variety that they can have in the data / pixels. Is there a function for this?

Let us try it…

TRY TO USE Gemini:

“can you write code to train a lenet5 in PyTorch on the cifar10 dataset?”

(WAIT - what? why are we using this tool? Aren’t we supposed to learn to program in this course?)

[1]:

import torch
import torch.nn as nn
import torchvision.transforms as transforms
import torchvision.datasets as datasets
import torch.nn.functional as F

[2]:

class LeNet5(nn.Module):
    def __init__(self):
        super(LeNet5, self).__init__()

        # Convolutional layers
        self.conv1 = nn.Conv2d(3, 6, 5)
        self.conv2 = nn.Conv2d(6, 16, 5)

        # Pooling layers
        self.pool1 = nn.AvgPool2d(2)
        self.pool2 = nn.AvgPool2d(2)

        # Fully connected layers
        self.fc1 = nn.Linear(16 * 5 * 5, 120)
        self.fc2 = nn.Linear(120, 84)
        self.fc3 = nn.Linear(84, 10)

    def forward(self, x):
        # Convolutional layers
        x = F.relu(self.conv1(x))
        x = self.pool1(x)
        x = F.relu(self.conv2(x))
        x = self.pool2(x)

        # Fully connected layers
        x = x.view(-1, 16 * 5 * 5)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)

        return x

# Load the CIFAR-10 dataset
train_dataset = datasets.CIFAR10(root='./data', train=True, download=True, transform=transforms.ToTensor())
test_dataset = datasets.CIFAR10(root='./data', train=False, download=True, transform=transforms.ToTensor())

# Create the LeNet-5 model
model = LeNet5()

# Define the loss function and optimizer
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=0.001)

# Train the model
for epoch in range(10):
    # Train the model on the training dataset
    for i, (data, labels) in enumerate(train_dataset):
        # Forward pass
        outputs = model(data)

        # Compute the loss
        loss = criterion(outputs, labels)

        # Backward pass
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()

        # Print the progress
        if (i + 1) % 100 == 0:
            print('Epoch [{}/{}], Batch [{}/{}], Loss: {:.6f}'.format(epoch + 1, 10, i + 1, len(train_dataset), loss.item()))

# Evaluate the model on the test dataset
correct = 0
total = 0
with torch.no_grad():
    for data, labels in test_dataset:
        outputs = model(data)
        _, predicted = torch.max(outputs.data, 1)
        total += labels.size(0)
        correct += (predicted == labels).sum().item()

print('Accuracy of the network on the test images: {} %'.format(100 * correct / total))

Downloading https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz to ./data/cifar-10-python.tar.gz

100%|██████████| 170498071/170498071 [00:12<00:00, 14158887.88it/s]

Extracting ./data/cifar-10-python.tar.gz to ./data
Files already downloaded and verified

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[2], line 52
     49 outputs = model(data)
     51 # Compute the loss
---> 52 loss = criterion(outputs, labels)
     54 # Backward pass
     55 optimizer.zero_grad()

File /opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1518, in Module._wrapped_call_impl(self, *args, **kwargs)
   1516     return self._compiled_call_impl(*args, **kwargs)  # type: ignore[misc]
   1517 else:
-> 1518     return self._call_impl(*args, **kwargs)

File /opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1527, in Module._call_impl(self, *args, **kwargs)
   1522 # If we don't have any hooks, we want to skip the rest of the logic in
   1523 # this function, and just call forward.
   1524 if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   1525         or _global_backward_pre_hooks or _global_backward_hooks
   1526         or _global_forward_hooks or _global_forward_pre_hooks):
-> 1527     return forward_call(*args, **kwargs)
   1529 try:
   1530     result = None

File /opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/loss.py:1179, in CrossEntropyLoss.forward(self, input, target)
   1178 def forward(self, input: Tensor, target: Tensor) -> Tensor:
-> 1179     return F.cross_entropy(input, target, weight=self.weight,
   1180                            ignore_index=self.ignore_index, reduction=self.reduction,
   1181                            label_smoothing=self.label_smoothing)

File /opt/homebrew/lib/python3.11/site-packages/torch/nn/functional.py:3053, in cross_entropy(input, target, weight, size_average, ignore_index, reduce, reduction, label_smoothing)
   3051 if size_average is not None or reduce is not None:
   3052     reduction = _Reduction.legacy_get_string(size_average, reduce)
-> 3053 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)

TypeError: cross_entropy_loss(): argument 'target' (position 2) must be Tensor, not int

If you run the code above you will see:

It does not work and we do not know why, but it is quite close to being correct!

In this course we will learn how to code this properly, or fix this properly!