{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# unit 6.0 - Unsupervised and self-supervised learning\n",
        "\n",
        "[See also: unit  4 types of learning](https://githubtocolab.com/culurciello/deep-learning-course-source/blob/main/source/lectures/04-types-learning.ipynb)\n",
        "\n",
        "\n",
        "## Supervised Learning\n",
        "\n",
        "Supervised learning is the type of machine learning we have focused on this course so far. To summarize here is what we learned so far:\n",
        "\n",
        "- **Labeled Data**: In supervised learning, we have a dataset where each example is paired with a **label** (also known as the **ground truth**). These labels represent the correct output for a given input. For instance, in image classification, each image is associated with a specific class label (e.g., \"cat\" or \"dog\").\n",
        "\n",
        "- **Training the Model**: We train a model (such as a neural network) using this labeled data. The model learns to map inputs (features) to the corresponding labels by minimizing the difference between its predictions and the ground truth.\n",
        "\n",
        "- **Prediction**: Once trained, the model can make predictions on new, unseen data. It generalizes from the training examples to make accurate predictions for similar inputs.\n",
        "\n",
        "\n",
        "## Unsupervised Learning\n",
        "\n",
        "Unsupervised learning is a type of machine learning where we work with unlabeled data. Unlike supervised learning, there are no explicit labels (ground truth) associated with the examples. The primary goal of unsupervised learning is to discover patterns, structures, or relationships within the data itself.\n",
        "\n",
        "Examples:\n",
        "- Clustering: Algorithms like K-means group similar data points together based on their features.\n",
        "- Dimensionality Reduction: Techniques like PCA (Principal Component Analysis) and t-SNE reduce the dimensionality of data while preserving important information.\n",
        "- Density Estimation: Methods that estimate the underlying probability distribution of the data.\n",
        "\n",
        "Use Cases: Unsupervised learning is useful for tasks like anomaly detection, recommendation systems, and data visualization.\n",
        "\n",
        "## Self-Supervised Learning\n",
        "\n",
        "Now, let's explore self-supervised learning:\n",
        "\n",
        "- **Unlabeled Data**: Unlike supervised learning, self-supervised learning doesn't rely on labeled datasets. Instead, it operates on **unstructured data** (e.g., images, text, or audio) without explicit labels.\n",
        "\n",
        "- **Implicit Labels**: Self-supervised models generate their own **implicit labels** from the data itself. These labels are derived from the inherent structure or relationships within the input. For example:\n",
        "    - In text: one can predict the next word (or character, or token) from previous words (characters or tokens)\n",
        "    - In images one can reconstruct a missing portion of the image from the rest of the image \n",
        "    - In videos one can predict future frames from past ones\n",
        "\n",
        "- **Cost-Effectiveness**: Self-supervised learning is particularly useful when gathering labeled data is challenging or expensive. By avoiding manual annotation, it reduces the need for human experts to painstakingly label vast amounts of training data.\n",
        "\n",
        "- **Examples**: Self-supervised learning has powered various deep learning architectures:\n",
        "    - Large language models (e.g., BERT and GPT) for natural language understanding.\n",
        "    - Image synthesis models (like variational autoencoders and generative adversarial networks).\n",
        "    - Computer vision models (such as SimCLR and Momentum Contrast).\n",
        "\n",
        "In summary, while supervised learning relies on explicit labels, self-supervised learning creatively generates its own labels from unstructured data. Both approaches contribute to advancing artificial intelligence and solving complex problems."
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.9"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}