unit 1.1 - The simplest Neural Network

we will now implement the simplest neural network, just like on the right of this figure:

NEURAL NETWORKS

Here is a single neuron neural network. Imagine this is a network that can tell if there is a cat in a picture or not. Function = is cat in picture? Outputs = YES, NO (1,0). This function… well is too complicated for now. We will do this in a few lectures.

For now we will learn another function. A MUCH easier function: the AND function. The digital logic AND. Recall that function? Is is the one defined by the following truth table:

x1	x2	AND
0	0	0
0	1	0
1	0	0
1	1	1

Gemini: “write the AND function truth table in markdown”

(note the pun: neural nets write neural networks lectures)

x1 = 1
x2 = 1

w1 = 0.4
w2 = 0.4

bias = -0.5

# fNL = lambda x: 1 if x >= 0 else 0 # non-linear function

def fNL(x):
  return 1 if x >= 0 else 0

y_AND = fNL(x1*w1 + x2*w2)

print('AND:', y_AND)

AND: 1

Writing it like this is time consuming and inefficient, so we turn back to linear algebra and use pytorch

import torch
def forward(X):
  return fNL((X*W).sum()+bias)

X = torch.Tensor([0,1])
W = torch.Tensor([0.99, 0.99])
bias = torch.Tensor([-1])
y_AND = forward(X) # this is the non-linear function "f" - why is this needed?
print('AND:', y_AND)

AND: 0

\(x*w\) is now a vector-vector multiplication

‘.sum()’ –> operator sums up all the value in the vector

X = torch.Tensor([0,0])
y_AND = forward(X)
print('AND:', y_AND)

X = torch.Tensor([0,1])
y_AND = forward(X)
print('AND:', y_AND)

X = torch.Tensor([1,0])
y_AND = forward(X)
print('AND:', y_AND)

X = torch.Tensor([1,1])
y_AND = forward(X)
print('AND:', y_AND)

AND: 0
AND: 0
AND: 0
AND: 1

now we were able to run all the inputs in just a few lines

noticed that we did not change the wights \(w\) because we assume they are fixed for a specific function

HOMEWORK:

Try the OR function on your own.