{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sNg4U8cdP0i8"
      },
      "source": [
        "# unit 1.1 - The simplest Neural Network\n",
        "\n",
        "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/culurciello/deep-learning-course-source/blob/main/source/lectures/11-simplest-neural-net.ipynb)\n",
        "\n",
        "we will now implement the simplest neural network, just like on the right of this figure:\n",
        "![](https://github.com/culurciello/deep-learning-course-source/blob/main/source/lectures/images/real-ann.png?raw=1)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_Y_Yjy8aP0i9"
      },
      "source": [
        "## NEURAL NETWORKS\n",
        "\n",
        "Here is a single neuron neural network. Imagine this is a network that can tell if there is a cat in a picture or not. Function = is cat in picture? Outputs = YES, NO (1,0). This function... well is too complicated for now. We will do this in a few lectures.\n",
        "\n",
        "For now we will learn another function. A MUCH easier function: the AND function. The digital logic AND. Recall that function? Is is the one defined by the following truth table:\n",
        "\n",
        "| x1 | x2 | AND |\n",
        "| --- | --- | --- |\n",
        "| 0 | 0 | 0 |\n",
        "| 0 | 1 | 0 |\n",
        "| 1 | 0 | 0 |\n",
        "| 1 | 1 | 1 |\n",
        "\n",
        "[Gemini](https://gemini.google.com/app): \"write the AND function truth table in markdown\"\n",
        "\n",
        "(note the pun: neural nets write neural networks lectures)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "id": "GmSNwZ4XP0i-",
        "outputId": "bc388e37-c5df-49ee-8a92-dc2b0aaede93",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "AND: 1\n"
          ]
        }
      ],
      "source": [
        "x1 = 1\n",
        "x2 = 1\n",
        "\n",
        "w1 = 0.4\n",
        "w2 = 0.4\n",
        "\n",
        "bias = -0.5\n",
        "\n",
        "# fNL = lambda x: 1 if x >= 0 else 0 # non-linear function\n",
        "\n",
        "def fNL(x):\n",
        "  return 1 if x >= 0 else 0\n",
        "\n",
        "y_AND = fNL(x1*w1 + x2*w2)\n",
        "\n",
        "print('AND:', y_AND)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SS19olXrP0i-"
      },
      "source": [
        "Writing it like this is time consuming and inefficient, so we turn back to linear algebra and use pytorch"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "RilEROcRP0i-",
        "outputId": "14c4d5e3-d1dd-4090-a1ce-2acadb34c0b4",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "AND: 0\n"
          ]
        }
      ],
      "source": [
        "import torch\n",
        "def forward(X):\n",
        "  return fNL((X*W).sum()+bias)\n",
        "\n",
        "X = torch.Tensor([0,1])\n",
        "W = torch.Tensor([0.99, 0.99])\n",
        "bias = torch.Tensor([-1])\n",
        "y_AND = forward(X) # this is the non-linear function \"f\" - why is this needed?\n",
        "print('AND:', y_AND)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2jWFO9VJP0i_"
      },
      "source": [
        "$x*w$ is now a vector-vector multiplication\n",
        "\n",
        "'.sum()' --> operator sums up all the value in the vector"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "oe9QGBTNP0i_",
        "outputId": "05841721-4a08-44a2-d434-91f555670a8a",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "AND: 0\n",
            "AND: 0\n",
            "AND: 0\n",
            "AND: 1\n"
          ]
        }
      ],
      "source": [
        "X = torch.Tensor([0,0])\n",
        "y_AND = forward(X)\n",
        "print('AND:', y_AND)\n",
        "\n",
        "X = torch.Tensor([0,1])\n",
        "y_AND = forward(X)\n",
        "print('AND:', y_AND)\n",
        "\n",
        "X = torch.Tensor([1,0])\n",
        "y_AND = forward(X)\n",
        "print('AND:', y_AND)\n",
        "\n",
        "X = torch.Tensor([1,1])\n",
        "y_AND = forward(X)\n",
        "print('AND:', y_AND)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u3vTEpqSP0i_"
      },
      "source": [
        "now we were able to run all the inputs in just a few lines\n",
        "\n",
        "noticed that we did not change the wights $w$ because we assume they are fixed for a specific function\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HGFnJTo6P0i_"
      },
      "source": [
        "## HOMEWORK:\n",
        "\n",
        "Try the OR function on your own."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.6"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}