cifar_10.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 5,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "cifar-10.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IYGjWgvzzq0m"
      },
      "source": [
        "# Deep Learning Training using PCA with CIFAR - 10\n",
        "\n",
        "## Understanding the Data\n",
        "Before you go ahead and load the data, it's good to understand and look at the data that you will be working with!\n",
        "\n",
        "### CIFAR - 10\n",
        "The CIFAR-10 (Canadian Institute For Advanced Research) dataset consists of 60000 images each of 32x32x3 color images having ten classes, with 6000 images per category.\n",
        "The dataset consists of 50000 training images and 10000 test images.\n",
        "The classes in the dataset are airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck.\n",
        "You can download the CIFAR dataset from here, or you can also load it on the fly with the help of a deep learning library like Keras."
      ],
      "id": "IYGjWgvzzq0m"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "driven-vertical"
      },
      "source": [
        "from keras.datasets import cifar10\n",
        "import numpy as np\n",
        "import pandas as pd"
      ],
      "id": "driven-vertical",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "alpine-ticket",
        "outputId": "1e5403c1-ddfc-45fe-d189-4acab4cb9102"
      },
      "source": [
        "(x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
        "print('Traning data shape:', x_train.shape,y_train.shape)\n",
        "print('Testing data shape:', x_test.shape,y_test.shape)"
      ],
      "id": "alpine-ticket",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Downloading data from https://www.cs.toronto.edu/~kriz/cifar-10-python.tar.gz\n",
            "170500096/170498071 [==============================] - 2s 0us/step\n",
            "Traning data shape: (50000, 32, 32, 3) (50000, 1)\n",
            "Testing data shape: (10000, 32, 32, 3) (10000, 1)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "greatest-aerospace",
        "outputId": "9eb80d06-28a3-4997-c12a-9be3e00b546d"
      },
      "source": [
        "# Find the unique numbers from the train labels\n",
        "classes = np.unique(y_train)\n",
        "nClasses = len(classes)\n",
        "print('Total number of outputs : ', nClasses)\n",
        "print('Output classes : ', classes)"
      ],
      "id": "greatest-aerospace",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Total number of outputs :  10\n",
            "Output classes :  [0 1 2 3 4 5 6 7 8 9]\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 225
        },
        "id": "boolean-price",
        "outputId": "65ad8d38-3a16-44d0-dfa8-7d474de42b64"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "label_dict = {\n",
        " 0: 'airplane',\n",
        " 1: 'automobile',\n",
        " 2: 'bird',\n",
        " 3: 'cat',\n",
        " 4: 'deer',\n",
        " 5: 'dog',\n",
        " 6: 'frog',\n",
        " 7: 'horse',\n",
        " 8: 'ship',\n",
        " 9: 'truck',\n",
        "}\n",
        "\n",
        "plt.figure(figsize=[5,5])\n",
        "\n",
        "# Display the first image in training data\n",
        "plt.subplot(121)\n",
        "curr_img = np.reshape(x_train[0], (32,32,3))\n",
        "plt.imshow(curr_img)\n",
        "print(plt.title(\"(Label: \" + str(label_dict[y_train[0][0]]) + \")\"))\n",
        "\n",
        "# Display the first image in testing data\n",
        "plt.subplot(122)\n",
        "curr_img = np.reshape(x_test[0],(32,32,3))\n",
        "plt.imshow(curr_img)\n",
        "print(plt.title(\"(Label: \" + str(label_dict[y_test[0][0]]) + \")\"))"
      ],
      "id": "boolean-price",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Text(0.5, 1.0, '(Label: frog)')\n",
            "Text(0.5, 1.0, '(Label: cat)')\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAATkAAACuCAYAAABN9Xq+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO19aZBc53Xduf16nX0Gs2EHQYKruIqkJEqWqa2KksuSfrhiWUksJ3IplURluyqJTTuJI6cUW3YlXlKu2FaVaMkuWTJtyxYtM7FlipJKphbuFFcQxEIMMJjBYNae3ru//OgGu88BZjAAZnqAxj1VKMzt9/p79713++v3nT73XgshwOFwODoVsc12wOFwODYSPsk5HI6Ohk9yDoejo+GTnMPh6Gj4JOdwODoaPsk5HI6ORkdPcmb2G2b2Cxc5xh4zC2YW3+j3mtl1ZvaMmS2Z2c+dv7erjp0ys5fNbGQ9x3XUcbnF2nrBzMbM7CUzS7XzuOeDjp3kGh/mnwbwRw37XjOb2FyvzolfBPBoCKE3hPC/13PgEEIRwAMA7l/PcR2XbaxdEMzssJm997QdQpgC8CiAT2yeV6ujYyc5AD8D4OEQQn6zHTkP7AbwwkobzSy6yPH/DMDHLuVv3csUP4PLL9bWE18E8G8224mV0MmT3PsBfGstO5rZj5nZ02a2aGZHzexTZ9ntX5vZcTObNLP/2PLemJndb2avmdkpM3vQzIbO11kz+waAdwH4fTPLmtm1ZvZ5M/sDM3vYzJYBvMvMbjCzb5rZvJm9YGYfbBlji5n9beM8HjezT5vZd05vDyFMAJgD8Nbz9c+xKi6rWGuMtdPMvmJmJxtj/X7j9avN7BuN12bM7ItmNtDY9qcAdgH420aM/mJjuO8D2Gtmuy/Elw1HCKEj/wE4CeCuFvteABMr7HsvgJtRn/RvATAF4MONbXsABABfAtDd2O8kgPc2tv88gO8B2AEghfqS5Uvy3njDvh/A11bx+ZsAfrbF/jyABQBvb/jWC+AAgF8BkATwbgBLAK5r7P/lxr8uADcCOArgO3KMhwD83Gbfn076d7nFGoAIwLMAfqdxnDSAdzS2XQPgfY3xRwB8G8Dvtrz38Gl/ZMznAHxws+/FWc93sx3YwMArA7h+LYF3lvf+LoDfkeBpHeu3AHyu8fdLAN7Tsm1r49hxDbw1HPdsk9yftNg/AuAEgFjLa18C8KlG4JZPT3iNbZ8+yyT3RQC/utn3p5P+XW6xBuBtjclzLft+GMDTLfZKk9w/Afjpzb4XZ/vX1l9i2ow51J98zgkzewuAzwB4E+pPSCkAfyG7HW35+wjq37JAnUf7azOrtWyvAhi7AJ/PhtbjbgNwNITQeqwjALaj/q0bl/1b/z6NXgDz6+Sbo47LLdZ2AjgSQqicxb8xAL+H+hdqL+pPnHNrGPOSjatO5uSeA3DtGvf9M9SXcTtDCP0A/hCAyT47W/7eBeB44++jAN4fQhho+ZcOIRy7CN9b0Vom5jiAnWbWet92ATiG+jdzBfWlzNl8Po0bUF+qONYPl1usHQWwawW5ya+jHnM3hxD6APwL8e+MskWNca7BJRpXnTzJPQzgR/VFM0vLP0P9W2g2hFAws7sBfPQs4/1XM+sys5sA/CsAf954/Q8B/I/TpKuZjZjZhzbkjOoEbw7AL5pZwszuBfDjAL4cQqgC+AqATzX8vB51WcMbMLPtAIZQ53Uc64fLLdZ+AGASwGfMrLvh29sb23oBZAEsNOLlP8l7pwDsldfuBnA4hHDkAnzZcHTyJPcnAD5gZpmW17YDyMu/qwH8OwD/3cyWAPwqgAfPMt63UCf9HwHwP0MI/9B4/fdQ/2b+h8b7vwfgLWdzyMx+xcz+74WeUAihhPqk9n4AMwD+D+o8yMuNXT4JoB913u5PUefrii1DfBTAF0JdM+dYP1xWsdb4Qvxx1J++XgcwAeAnG5t/DcAdqP/g9Xeof3G24jcA/JfGr/unf/n956hPwJckrEEadiTM7NcBTIcQfnezfdkMmNlvAhgPIZzWxj0L4J0hhOlNdq3jcKXGmpmNoj4p3x5CKGy2P2dDR09yVxoaS9QkgB8CuAv1ZdTPhhD+ZlMdczg2EZ386+qViF7Ul6jbUOdO/heAr26qRw7HJsOf5BwOR0fjon54MLP7zOwVMztgZp747VhXeHw51gMX/CRn9WTx/aingEwAeBzAT4UQXlw/9xxXKjy+HOuFi+Hk7gZwIIRwEADM7MsAPgRgxSBMJBIhlU6/YVerVdoeE51hJBLJZJwfPBNixyMu0lGXJbXa8uAq41cq7I9O/5GOL18QNUpEAEKtud1iqvdk1Gp8bD3WGfvLsUxORu2YjBfF+FrotarJuYUz9KoM/bLUa3d0cmYmhHA+tezOO756M/GwpS/5hq0e6zkqzjwHts94f1BT9tcD6AtBF1J6PB1PBmjZrs8q53544bF07xDOca3Oca4aPzjHeOqB+lMLavMLU3OlFePrYia57eD0kwmsoNk5jVQ6jdvuePMb9vz8LG+P8Qd3KMknsmtLF9kjQ91kDw/0kJ2MEmTHUxmyEfHpz85xVkqpwscfHOgnO1Ytk10ssvysUGj+op7OpGlbFTyp5fJZsvsH+tjXwPuXiiWyI/C56iTZ28PXprubr10iwf7lZfygXxAxvnbqT0WC+pOf/qPzFYqed3xt6Uviv330+jdsky+CZIJ9NpnoSyW+fxW5v8lkkuxqbeUvtfr48iUu31uhzPfAJCYSSVZkRFD/m8er1jhDq1xh32o1nSB5rEqVtxdr+qXJOONLVr4ASiW+dtWq+C7vj8m5l+TaLksCWq7E+//Pvzi0YnxtuBjYzD5hZk+Y2ROVcvncb3A4zgOt8ZXNn5GK6XBc1CR3DJxjt6PxGiGE8NkQwp0hhDvjiYRudjhWwnnHV0/GFVGOM3ExUfE4gH1mdhXqwfcRnD0P7w0UCgW88GKz8O38zAxtH+IVE2wLvzBc5UIPlhkle7nGy99sVdb5xsuNXIGXWLk8L1fKVX5knhGSMB3n8SuyRIhalnSpFBfjzRWW+b019sUKW8jWpU5ZlsaZOF+rrCwfZ6v8lNPVJUulGH8BmSz1IUu7XIGfyvUpPYpfdPHh846vAEOp5Xv7jEK9sgRKga9BDHyR43FZbq5OocESvEOxJEv4mowvnJzSsHGlkGuyEqo0Y0CXezU5VsmELon4/pR0/6pwtsIZmyyP03LucaE3YvJZqeqqzni8AOXH+bMXRWt/PrvgSS6EUDGzTwL4e9RrmT0QQlixdLfDcT7w+HKsFy7q+T6E8DDqqUMOx7rD48uxHujkKiQOh8PR3tzVGIBMvGVtLbTNbuHg9oyxZGN0hHt2ZJRXkp+x80X+Cb5QZh4ryP7JjEhMREISavz+/iGWtFTKvH8y0RxPJIGIknzyxRL7Wq6wb12yf7ybfU3L9oox5xeTn+wrynGIRqCnm88tu5wT/5hTURng0uIC2o+A0MoVSUWpILykVZmHqpVFlpMRXgrCuQqHplrHpPzQVgls18pyfHm/6jZVlxlr4fQsYr45RCIJqnJ8nDjF92+5xGNns8KxioSpN82+J42vTV8Xx2cmxde+FuNrHTuDc+Px9SfLsgrnVoE/yTkcjo6GT3IOh6Oj4ZOcw+HoaLSVkzMLSLfoYXp7+fDXbh8ke0tG1uU15q2ys7yur9Z4zs7nmAeIMW2BPkkDiwuvNb+wxNvlag31Mm+1tMg8WKlFC5cXXZnqfnokzapcYo1XTNJiEqK7q0oKUlxItmJRUpQSfDFionsqZqVBk2gOU8JHVUSDtrDc/grrFgLi1ZbjRsJhic4sFUmGRFyIRRHGxVSbJbRQRXkiISoTSeapxvdw75vFedaNzpxiHjQRl3vWQmqXKhwf+cDHeukIjx1SzG+XI46/Uo/oLhdYg3psmlMge1J8/OoJ3r5rjH3f0svxm45r2hffm6TcmqpwhKvBn+QcDkdHwyc5h8PR0fBJzuFwdDTaysnFzTDYsnbPCK/UL9qvkT5Wx1RFR6Sr8iguRJFwKkXhZOLCA8RFS1YtMi8WhJOZFl6iWmaPlnJNTiVXZf6wJyOllIpST040WTGpLRalRAe1zHxlV4LHj4vGqiB5u/my6JiEcJrP8vjzOb6WWeE/C+XN+v5skjcWH+AtoousaLmfGJ9DqcLXKBkpDyr5lRKfUB2m5He+5b3vI/vJx75L9vH5U2QvV7Q8UpNHOzJxkrYdOsa1DFIDW8neMXYV2SHFeeElyT1O9HCptkqBS4Odmj5OdtcAc34T2SmyC8LhjvXyZ70rwZ/lapn5ydjaZXL+JOdwODobPsk5HI6Ohk9yDoejo9FeTi4yjAw0uaReWXenJR8uJjqnjOSWliW3r6Z16wNzKlrOvColmmtBtGzCowXRKS2VWBdXlVzIXEs9uorUplta5mMdm+WxElIKvi/L51Y+wbqn/AJzFruGryF7dHQH2dbLuaXFOeZ/sln2Z2GJObmZBeYrDx/l8apR+wtY1iyGYqzJLS3kWMdYrbB2b7CHObi+iONJecyacHSmbQxEa6i6ulyOtYff+Bq3xJ2aZ/+msvz+I8f4/Ucmm9XhozRrPqsRc7LdfcNkJ7pEI5rmz1ZK6sGlY6yjmxEd59Ydu8gu5Dl+Dh1iTm52QUq7G/uzZ4TthHx+rLr2KtD+JOdwODoaPsk5HI6Ohk9yDoejo9FW4iQRj7BtpLm270vyurqnizkvC9rdS1q+ic6pmBctjXB0W3q5Pl13N2vNFheY5+rvY15jSfJPjxzj/bNFqbHV4t72LtHkJYTTOsWau2KQvF0hgPr7WNd0z413kr04KRqunLx/mHVJxRz7lxU+KCW10XaO8/FHR8fInlpkzuXws69jo1GpGU7mm9dttsw6uW8/9i2yb9jHPNO7bmLealA44Zr2CZaaZzHpk1GV+NWujoeOHCJ7Ns/atNDFudyRtJWMDTZzqzPSLrNU4Otf0npvg3zufT1sT584QfbiHOeu9iY5XtLCl78+x5+NRC/3Yzl5guOhZ4rzxMf7pB6dtlDUfherwJ/kHA5HR8MnOYfD0dHwSc7hcHQ02q6TG+ptrrXjJeahUgl2pyvFOqdintfhZdElDQwwhxFE56S9JMtlyfcUzuP4SdYtvXaEtWAnl/j4kr6J3S318D78I7fRth1b+Vh/+eRBsr97gDkR7csal+S9pXnOXcxl2fdeyQ1EVXrIpnl7UjSLXSb9CkSntGvnNj7eLHMs32gDJ2dRCvH+Zk5m7pTc7yTnX87mRNdYYo62L8nXvCY1ziD146KI47VQYl5JwgkzS8zxab7n4Ahrz5Zri2QPozl+JDq3UoJ9Lyzz/ShkeazdY9znNyec27To4izB/OHCLPPhkDze/DLr5qIkX6vpRdYAToqObvew8J9MMa4Kf5JzOBwdDZ/kHA5HR8MnOYfD0dFoLycXj2N0qLn2z8/yujsmWpis1CzLl5gTiZtwKlLPTWfwvPTVHBhkHVxJ+hgcnOAaWbOLoj2TXNZIchX70s39R+PMiaRnmaDZ1zdO9uQQjzU1P012Mcfn8vT+/WTHKkxalLulfl0/69oQ42vf38+cSa/wTwXJ+w0l5nj2jLDuqh1IZ7px3S13v2FPfO8V2t7Tz5zc3W+7m+yu6AjZJeGxYnHmJS3BPFg1sC6vd3Qn2c88d4D9GWAebPvum8gOManpJjxbrdjMNy6VpDae+BrJZ+uFZ58juy8l9dyk50i35LoeP8G5qNrfIhLObrCXr9WC9CSZm2X70Anmv7eN8ecjLnzpavAnOYfD0dHwSc7hcHQ0fJJzOBwdjTZzcgkMDjd5kcEeXqdr7t+8aGfKy1xXPlbVenLMSwTR3fVIL8ky2H7pIPNay0XW9qTT0itStESZbuaxBlv6ej55QDiMEr+32M+cw8gg+2ZgTq1cYT4zJzqmZclVLVWYzzThJyXNFwnpGRpikksr/TEqReYYQ/U8ivCvE2JRHF39TZ5r917uayoyS+y6imvuDZelr8Uh5ujKopOrVvh+3/3OD/P4ezmf+KqbD5P95NPPkj3YwzFwfJrzP+OBOWDKJ5bLnRVd2oLkng5282dN71ZVOLbhEeYzi9ITZGaOOTQTfrpXcmPjUm+wVGCd3cGjE2SPDPBcsW8H506vBn+SczgcHY1zTnJm9oCZTZvZ8y2vDZnZ183s1cb/g6uN4XCsBo8xx0ZiLU9ynwdwn7x2P4BHQgj7ADzSsB2OC8Xn4THm2CCck5MLIXzbzPbIyx8CcG/j7y8A+CaAXzr34Qxo4d1MapQpUpJP2QVZ18scHZM+q2Xh6FIZrrk1c4J1ULkZ5gD3DjEvVmQaDGnh4K67ejv70/KGSsTnsih8YzxiTqM3yee6ZfBqsq/ex3mNh15/nOyX93PfzWRcOLPA/GZFenrGRAOYSLL/Nembqf01TIunrYL1ijGLxRClmnqu41Mv0fbb3nwX2d2iBYyW+JpVpSdIXDjYg0c5ft4xyL1M0cV9NXq7mXdKx1l7lpF8znSSOWDNB92+rdlL9cXXXqNtyaTUSlxiX/fs2Ef2tdffSPbsLMdnTx9rAI+fYN2mCWc7MMh5uAsS76opzXTx+PklvlYH5FpnkmuPrwvl5MZCCJONv08AGFttZ4fjAuAx5lgXXPQPD6Fe6mPFn9LM7BNm9oSZPbGUK6y0m8OxIlaLsdb4WlhYPNsujiscFzrJTZnZVgBo/D+90o4hhM+GEO4MIdzZ25VeaTeHQ7GmGGuNr/7+vrPt4rjCcaE6uYcAfAzAZxr/f3X13euohYB8S58EK+dlD9beLC/zN3OpzHNyJcaTZjbH6/ZFsbfv5NMNFd6+e5h5pau3MQ+VK/D27dfeSnYy8JPq3ELzXDOSp4hTzGHsHN9K9rzonPZezxxK32CX2DfwsU/yuc0tMOeXEM4vFpj/KQv/IxQcqqKTElndGbX8LgDnHWNmERLp5kRXKLAWsFhkoVxCOLAuye/t1l6kkfQkEZ7z85/9HNk//pOf5OMtc43AZEo5ZR7/qr3M8U7Pci51oaU37vgo96eYXWROq1jia7H3GtYIXn0NawoXnn6K7OUl5nAXl3l87Sucz/NnYUB6UFQDx2ffgNQrlDz1KMbXemJyxeeqM7AWCcmXAHwXwHVmNmFmH0c98N5nZq8CeG/DdjguCB5jjo3EWn5d/akVNr1nnX1xXKHwGHNsJDzjweFwdDTamrsaEFC1JtcTpE+A8jgZ4UR6eplDOX6SOb1DE9znIJ7g8ZJTwmlM8f77RpkXeM+9zIO9dkx6T27nfL7hLZx7OH2yma86MCAcWE16KojOaPoka7biae6HcXJ+kuxjk8yZJBJ8rQb6lDPhaxPi/H1nQrLVhKOLmejiRKO4CamrgBmsRY+YyzKvWchxvCSk5tnSKT5HRBx/CTCvuXWA79mrL3G9uOMTbCPH8Xdk4jDZt49zfbvtuzmetk2zimb5QDO3digltewGmKM7eJCPtXUb833zi8x/l4Vjmzp5iuxakPsvuag54eQsxtdWKFx0S24raqyzSxrfu9Ip5jdXgz/JORyOjoZPcg6Ho6Phk5zD4ehotJWTi6IYBgaa+XqVOHNy2Syv44P0bFhYYk7kyOtcoy2bZV4qk+Y5fPIQ8w5jac7P3L59N9kD2zgXMbEkYjHJrd1xK3Mq6RNNXi1TYf6vCj7X5WW2t3Yx31cSjsS6Oe9xR7f0PR1gPmdJOIzpKeZYytJXtVCSJqHS57U7xRrFUl44weTqeckbggDqhRoFvmZbh1mr2CX1Ab/xHOd/DkqfjH1DfE7pFMdnMs738OT0YbJrRc7f3HU1x1ck/nT1ceGV4THOhT0127zmC6KLk1KLGJF6cHHhIwuiSyuJDjJf4HioyAHULhRZl1ep8Gdxy/Ao2SbxlzS+limTWn6BOefV4E9yDoejo+GTnMPh6Gj4JOdwODoabeXkatUKluabXFC8xPlrCa1BxjIkxCPps5pljm6wl7U2A93MG+XnmJMb3SZ9L2/5UbKfn2BeYf8Btu/Zylqe+XnePnZ1M7c1BuZMSkXm6AaEP1qcZs4sI31Otw7JsavSo/MW5nPyoqv7p4cfInviKPsTncGpsbJJZHYoa22/sjRUaAPMgES8GSP90kNkQHp/Wo15nsXA8TMzx+c83Msfl265RtUYn/Ph44fJHhvk/M3d13ANt4Jcsh88yfXwjk0yp9fb07zHiQTH+gsHXufB5P7UxC4KJ5ddZl3agMRbRXRyk1OcS9rdy+cajzhgurqYU0tq7bwyx391mXWiY6Pe48HhcDgA+CTncDg6HD7JORyOjkZbOTkAiFqW8lXRVgXhfWJSX65qzMnNCYexuCj5mKLV2drPnMtd73oX2TuueyvZX/njB8geF21aJL1Ojx1kndX43ibnkt7C9bu6pZ5WbpY5jUyNObVSnjm9GamBPzDCmqst43vIzme5VlpM6ktWk5pryPeiLH1arSK5iEF0U5W2hxYAIGrJqR0fZa2g9gSpifZr6w6+hk8IpzZvHD8h4tzY/mG+Bv19zNkl0swj7RFOrqefOeI/fuBPyc6Jv4v5Zi51Ls++SMthjA+KDnKWe8oup9R3PteXX3mV7CnJ+16UenMDA+xAn352gtT2K7H/keT5jnTz/v1pzX5dGf4k53A4Oho+yTkcjo5GW9cUBsBaVpRVkRlouR6p/oOQl/0ly2poC/8sPd7Fy9077uQSzzfcw8vTuWl+5E5VWKKydwen1dTEgfFRTp2pFJrHz4m8pFRh38p5vhVV8OP9a8cmyP7h80+Qfc9befwt47z0WVzi5bBUYsLwHl6e1LR0UkmWo0IFLJzkn/iLS2tPu1kvxGIxkiL0DfJytVLla5yKs2zh2qu4zeMTT/LycjHBlEPNmHIY285Lwhdf+h7Z9/zoz5D93cd4u5b7L5dmyJ4+cRSM5j3KSmuAOPizMhhj+cn2DB9r4SQvRysR0yVjo2xXpUyaljsvCL2yLGlklRp/1soFLi02mmAqaFsPx1Oxoq0TVoY/yTkcjo6GT3IOh6Oj4ZOcw+HoaLS3/HkAai3Sg3yROa2k/MwcjzPHEcWYB7pmnHmCdIbn7D27d5J96ztYMrL1ulvIfua7f0z2rp08/vhNN7O/I1ezv12cypIrNHmH/CLzN1PHmV+Zm2LOrVpmTiPTy2k7w8N8bY4ef5rssa1c3rqSE7lOnuUItsycTTUw5xFMStOnpDTOONuLqbX/xL9eiMViVEZ7cJhLgFeMw70Q41Jb6R7W1WgbvdePcrmqd9x1E4+X5Xju6mWZxaTwqgf272f/qhzfUhEfy4vMEfduabaxXFjgeOnv4Xi57to3kf34sy+T/dTLh8l+x73vJ1vbNx48wKXdF0TSpGljBZGL7R5jvjPTzSl3Q0O8PUhZtkpp7fX1/UnO4XB0NHySczgcHQ2f5BwOR0ejvTo5MyRaWpfNyTq+WmAeJ9PF6/RISnCPii7u6CRrta6+4z6yd9zMNsCcW3lJ0nSkXMzItbeRvRzn8jMvPP042cWWVJvFRfZt5hiXwomEj0mn+dZsv4o5tluuZc1WJWKdWyLiFnWJJOum4gXWNeWOsE6pJmlbUr0aWSl71bWFjz8mZazagRBqqFWaMdU/xBzvcp7PKSd9E6OIT3LXTtZF7n+BtWQLOebgerpZZ7eTKVsc2c+pVMeOc/mrt73tLvZPeNReaSM41FKe//VZ5tjO5Ls5VvtGmK++vZfP9aS0IDx85Fmyl/Mcr/ML7KuWW+8PfK67e/j9o30cTwljHV+pzBxxt3lal8PhcADwSc7hcHQ4fJJzOBwdjfbq5Go1FFty3LpSfHhLy7o8xtqYIPlymR7e/4M/+UGy73n/e8juGx4je+ogl5eO5Hjz0gLx5OFXyD6+xBzPN//mb8juyTS1Y4UicxbjY8z39Unp9kMTrKMriW9D2/aQfe3NbyYbUg59dp41WjnhP+fyPL4F0ZTlmePJBilrJe0kb2BKsC2oVcpYOtXkfjKSL1ksSLmomsSfaAGHh5hX3B87SPb0LHO4pyK+Rv09nDt7/Zv4nh88wvdYOnBiXtoM7tu3j+2rmqTfkUmO1Rde+CH7NiPlxlPMVw72sC5t4gXm+E6cYo7MRGMYSRkpLVu1Wyi0XaL7TEt8Fwt8LWs11mGWJfd7NfiTnMPh6Gicc5Izs51m9qiZvWhmL5jZzzdeHzKzr5vZq43/B881lsOh8PhybDTW8iRXAfAfQgg3AngrgH9vZjcCuB/AIyGEfQAeadgOx/nC48uxoTgnJxdCmAQw2fh7ycxeArAdwIcA3NvY7QsAvgngl1YdCwG10MKL1KSEdoXX4RUpkaycSTrFuYa3vZl5qVRC6ns9w/mdc8e5XHmxyLzS0tws2UcPvEh2NrCOL1Hl9/e0tMfrSzPnNiLt6SanOC+yIrX2clJe+ughbTn3AvuW5VzZdJyvXSU1SvapCl/LTIY5ky5p55eRWmxLOeZsKrW1cSbrGV/FYhEHDzR5s137bqDtacl9rkn5+nhaeCKxe3uZx+rp42t2/fXXkf2P//Aw2bkFvsddQ3wPDkxwzb+dO1h3d9V1d5CdSjY/vnt38b7zs5yL/OJLrPGrSbn6Y1LvcFE0hQXheBfnmS8cHWed3eunePvQTo73UylpQVgT3Z3oNEOc70VR9l8N58XJmdkeALcD+D6AsUaAAsAJAGMrvM3hWBM8vhwbgTVPcmbWA+CvAPxCCIG+tkMIAcBZywKY2SfM7Akze0JV0g7HaaxHfC3J067DAaxxkjOzBOoB+MUQwlcaL0+Z2dbG9q0Aps/23hDCZ0MId4YQ7uzOJM+2i+MKx3rFly4nHQ5gDZycmRmAzwF4KYTw2y2bHgLwMQCfafz/1XMfLgBo8m61Cj/ZxaXxQFXW5SVpUTjWzz+4/f1DXyN7aIx5qtGtnK9XyrG2KCG6qp5u5lziUuCrWzi/8VHWVeWXmrxIJuKxT53k+v1l6aHQm2YOrJTlp5RXn+YeD5Mvc22yM2rgJ9j3qp7LDuYM0S21zVLMN6aFcxsE+3vDTayTAp7C2bCe8ZUrVvDMgeZcuOtNd9P2GljXZqq1qvHD4uIS85rz83zPtgxxLvMH7sv1ANUAAA8KSURBVON6hbfdej3ZD37lr/n40mKzX+J5+zbmuXr6WHwYVZrnMzTOH+WtVzGnuyAc69PPci7qZJaFbCHBsd8vPUOGr2aOLRLOrBp4vFcCx9eBExzvyYj3z2tutdyqSk2K7eFbWAlrEQO/HcC/BPBDM3um8dqvoB58D5rZxwEcAfDP1jCWw6Hw+HJsKNby6+p3AKyU8v+eFV53ONYEjy/HRsMzHhwOR0ejrbmrCIZarfmlnYzzujodl0aqMeEJpGZarcS8w8wM65CyJ9nOlFnLVQMff2iQeYeBbdJHtcp9EY4d5/GD/AAYizUvr/ZZjYz5vO4085EiGUSkL4hmsFpifjFW42u3mGPdVCnFnF3vNj635QzXv1sSXVJhmb8ft/TtJXt4tP315ApVw/6FJjc4U5U+AQnmeWJyzYLwPDHhLbdtZV3bj9zDurV0gnmmq3Zz/bcf+4mPkP2Xf/13ZM+cYH8mF/ieFwrcVyHZwlHPSu7xgSMcm5DPShhmTd/gKMdfTWLZJF5rEq814x8Vy1Krb6HK708npL9GnON12VhnVxb+O9T4fFaDP8k5HI6Ohk9yDoejo+GTnMPh6Gi0l5ODIWZNvVg6xdqqIDq47gyv+7t7uY9mrswcy5ZeXufHZbzSwhTZNamJlUswBzI2xlqvWol5qetuYR3TY48+wscLTV4hITXp81nmHPp6WZeUjPOtiUzquYmO6NAkc27z81Kfy1gjNnItf79tHxBdXuBrMzfD/iYLwiluF41gToqjtQHFqmH/fPO8vvodrql2226On/Ekc7xdCdGajXM9uK3DfI+u3sv3H4HjY1L6JDzwZebgnnqGc6G13t0ZJdMC37PQ0hekKnnc1Rjfn7joGCui0avEeHtaZwbRvRVK4ovw53HRzUU1jt9QkD6q4O2JGo8fGdulsvd4cDgcDgA+yTkcjg6HT3IOh6Oj0VZOLmZAMt6cV3NF1mZFUnOtJvmeOem9GCVYi5NKSn23BI+X7OJ8u/4+3n7iJHN2ue3MuYzu5F6nx6Y5l/Gmu95Odvbk8Tf+Prif82iXs6xDi0d8bv39zLGYcBaTx46T/foR0cml+Nz6xpjfHBmS8YXjs1l+/+Cc9IEd5T6eOwb4Wh14UXRabUAVhmwLz/rIU5zP++pr3KPhvjffSPbV2zg+Dh3kGmzvvOtNZKdFu7VUYp7rwf/HfXiffpHvWa4iNdWEx4ol+BmkJrm1MWvyWsqJVaVWY1E4rnJVajka686KEF2a9PSIx4Uzk561XV3M6SbBx6uK7LNqHF9V2aFSZg4v2bv2JiL+JOdwODoaPsk5HI6Ohk9yDoejo9FWTi4eN4yNNOfV8inWEeVlHb7M0i6EGK/r46Il6+tjrVZSOJP8MueuZkQXhRLbTzz2GNl7r2PObmKCeaeY8CJdqebxI+EXMxnmvJazzMnl82xXpPZeT4bHu+f2a8lOi+6uEjGnUS2z7i1/VPI6l5gfGu3iPNDbr72Jtw9wdfInJw+h3YjH49gy3Mw3np1jHmlyjnnQx57l3qLV8m4ZkXmlEeljYHJPf/DE82T/3Te+S3axxrwopE9GLLb6M0e1yDEQWji6mnBwyqFpfbeEfHYskvpskWhOZXsU8fu1YGkk5xKTfi1V0fzVhANU0m58nPnS3j62n8TK8Cc5h8PR0fBJzuFwdDR8knM4HB2NtnJyyaRh187mWr/fmPc5cJR5oqmTzCuUpPdjTw+7vyw9G6o17osQyZw+K7mFS1nmrQplHi8KbPf2cE3+qRPcp3Viuclz1YQTGRth/tCkPtbcPOeiprr53Af6mSNLik6pKD0jEGfOY7kouYBZyUUVXdU1OzmPc5vU/D86wXzlqZN8L9sBMyPuSHt2VArMMx2eYo62uPwS2e+8g3nOzMBWshcKzBt96/vcd6MQOJ7KFb7HqRTHf03yO3O51a9h1KItM03llN5mKeHQLCYffbEtxfxhJsMaVOXDy6JjWxJCvSoav6LUR+wf5Lzisa1s90gybV76b6wGf5JzOBwdDZ/kHA5HR8MnOYfD0dFoKycXxQ19g03uJy+8zeCoaHW6mReYmeJc14LUd4snWRsmm1ErM09Vlp4NC3nmwbpFi1bIsZYsX+Dc1ZKMX22xQ+Bzyy5KPbm+jNisA8rnef+ZU+xrTw/r7kx0SlZhTiQZ5+MJPYRkkv3dc80e9ifH433721wb7bn9Z+0FvbEIAbXWXr2qxYr4JEvS42M6y/Hw1Cuca/oBOeelwLzQsTm2Uz2sHavk+HgFyd3u6hLeS3Scur+19KCISX041cEF4dyCPN8kJACyEsulCnNsytGpLk85t2WpldczwJzbwAhzviXRhb7yMmsaE7W11yv0JzmHw9HR8EnO4XB0NHySczgcHY22cnJmhniL3iXdx7qloR6ec+N55iASGV7nL0qNM1T5/Zk098msSg+HapFzGZNdPF4izv5FEXOExcDjlcqSW9iijZM2qQgl5veqbCIhujYkmR+cn2NOLi99NfsHmJ+May6hnFtO+mFMzTC/NJdVHRRrBv/xm8yZTLVfJlfXhrXqseT+RJH0DhWeVPsiHJ7ma/DAgw+T/e577yT70PGTZOeqmp8pPFha4ivJdpdoH5MZ5s3yS02eTHVqQTixhOjMIul5rO+PJFdVa9nlc9lVt+v7Bwa5/uCWMdYczpxijem89FCef51r+11zFfdfWQ3+JOdwODoaPsk5HI6Ohk9yDoejo9FWTq5WM2RbcyQj1hH1dDMxlcjwOr9btDz9/dKLdDEvNudTZqUXaLnAdm+S8zG1hn9FdEpa5z4pXxmJVJOXMOkb2SV5t5pKWKlKTfuM1M4bYH5wdpb5oyXho/qG+NxyokN69TDn8b78w6Nkj0lPiLEdUhstxscbltzaQ6f43mwEoniEoYFm7f9Cga/Jcp7PORlJL1LhsWKS+/rtHzxH9qHjrKNbWGZedFZqBMolR3e36OgkdzWV4uPHhbNLZ5rxG8WYA4sneN+qPM9UhEMzsUNQzSefm/LPmTR/Noe3cLwNDjMHVxINYzEpuakp9r+mudeFtceTP8k5HI6OxjknOTNLm9kPzOxZM3vBzH6t8fpVZvZ9MztgZn9uZslzjeVwKDy+HBuNtTzJFQG8O4RwK4DbANxnZm8F8JsAfieEcA2AOQAf3zg3HR0Mjy/HhuKcnFyoJ6WdFsUkGv8CgHcD+Gjj9S8A+BSAP1htrFIJmDjStIvzvI7vHWEeKp0R7RdTGBgaYvezyyzOmp9ne+5UUmweL6qJNkjr5EuvSkj+nH5jWEvPh0hyCfOioZLSY0hIfblKjnVEVcllrQpnMZ/l7Vpeblb4y8MH+GLMn+JcxdIyDzDez7mGN+zeTrYMj8cPcp7vaaxnfIVaQLGFq0nJDSlW+ZompI9BRVKng2oLMxyAR0QXFxPtWaXM8aOcX0F63S5LDTbt+aAcXXeyec8zoqGLCUeaTEuPkS4+l1KJA3BmluOtJjrKuPSEHZQexmND3Bd1fJx1cvPLzG8vSf3E7AJrWAeG+P0zJ88eT2fDmjg5M4vM7BkA0wC+DuA1APMhvPHRnACwfaX3OxyrwePLsZFY0yQXQqiGEG4DsAPA3QCuX+sBzOwTZvaEmT2xkC2c+w2OKw7rFV/l3OK53+C44nBev66GEOYBPArgbQAGzN6ov7wDwLEV3vPZEMKdIYQ7+3vSZ9vF4QBw8fGV6Oo72y6OKxzn5OTMbARAOYQwb2YZAO9DnRR+FMBPAPgygI8B+Oq5xgoWRzXRrCNVTnLuX7HG6/RYhdfd6X4uZD8wwpPmYIx5g6Ec8xLzs6yLmp9hDiW/zJejWpEf9LQ+mXIseX5STbbomjRXcEn6A+TlKTcRWIfUG2PdWS3GTy3lMvue6mY+KC2ar4Ekj78XzKHcfCtzLNfdcivZe665huy738oc4MRxzm3E4wdxNqxnfNVqNRRb7kEqkj64Eu21MhOHUpINNfA9qon2sCb16Col0ZpV+fhac01t7fGgnNyc5CvPtvjfJ/UE+yVXtE/yYNPgz05VPntxYw42SvG5Fgu8fyrO56rvr0j/lUqO35+dZ064Jjq8dIo554L2iV0FaxEDbwXwBTOLUH/yezCE8DUzexHAl83s0wCeBvC5NR/V4WjC48uxoVjLr6vPAbj9LK8fRJ0/cTguGB5fjo2GZzw4HI6OhikvsKEHMzsJ4AiAYQBrF7q0F5eyb8Dl69/uEMLIRh74Mokv4NL271L2DbiA+GrrJPfGQc2eCCHcee49249L2TfA/btcfFgNl7J/l7JvwIX558tVh8PR0fBJzuFwdDQ2a5L77CYddy24lH0D3L+14FLwYTVcyv5dyr4BF+DfpnByDofD0S74ctXhcHQ02jrJmdl9ZvZKoxDi/e089gr+PGBm02b2fMtrQ2b2dTN7tfH/4Cb6t9PMHjWzFxsFJX/+UvHxUix26fF13v5dGfEVQmjLPwAR6iV09gJIAngWwI3tOv4KPr0TwB0Anm957bcA3N/4+34Av7mJ/m0FcEfj714A+wHceCn4CMAA9DT+TgD4PoC3AngQwEcar/8hgH/r8eXxtZnx1U6n3wbg71vsXwbwy5t1g1v82CNB+AqArS1B8Mpm+9ji21dRT2C/pHwE0AXgKQBvQV2oGT/bPd9gHzy+Lt7Xjoyvdi5XtwNobQF1qRZCHAshTDb+PgFgbDOdOQ0z24N6juf3cYn4eIkVu/T4ugh0cnz5Dw+rINS/Ljb952cz6wHwVwB+IYRANZY208dwEcUuHR5f58J6xVc7J7ljAHa22CsWQtxkTJnZVgBo/D+9mc6YWQL1APxiCOErjZcvKR/DBRS73AB4fF0AroT4auck9ziAfY1fR5IAPgLgoTYef614CPUijcAaizVuFMzMUK+j9lII4bdbNm26j2Y2YmYDjb9PF7t8Cc1il+32zePrPHHFxFebCcQPoP4LzmsA/vMlQLR+CcAkgDLq6/uPA9gC4BEArwL4RwBDm+jfO1BfKjwH4JnGvw9cCj4CuAX1YpbPAXgewK82Xt8L4AcADgD4CwApjy+Pr82ML894cDgcHQ3/4cHhcHQ0fJJzOBwdDZ/kHA5HR8MnOYfD0dHwSc7hcHQ0fJJzOBwdDZ/kHA5HR8MnOYfD0dH4/29HwcFtkPCaAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 360x360 with 2 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "occasional-population"
      },
      "source": [
        "First, let's normalize the training and testing images. If you remember the training images were normalized in the PCA visualization part, so you only need to normalize the testing images. So, let's quickly do that!"
      ],
      "id": "occasional-population"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "driven-trinity",
        "outputId": "efa2963d-8c8b-4398-f253-9135b418a3dc"
      },
      "source": [
        "np.min(x_train),np.max(x_train)"
      ],
      "id": "driven-trinity",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0, 255)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "elder-blogger"
      },
      "source": [
        "x_train = x_train.astype('float16')\n",
        "x_train = x_train / 255.0\n",
        "\n",
        "x_test = x_test.astype('float16')\n",
        "x_test = x_test/255.0"
      ],
      "id": "elder-blogger",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hollow-cancellation",
        "outputId": "95b1fc85-af7b-4df4-fe76-b19f49696fcb"
      },
      "source": [
        "np.min(x_train),np.max(x_train)"
      ],
      "id": "hollow-cancellation",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(0.0, 1.0)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "informal-release"
      },
      "source": [
        "x_train_flat = x_train.reshape(-1,3072)\n",
        "x_test_flat = x_test.reshape(-1,3072)"
      ],
      "id": "informal-release",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "transsexual-antarctica"
      },
      "source": [
        "## Speed Up Deep Learning Training using PCA with CIFAR - 10 Dataset\n"
      ],
      "id": "transsexual-antarctica"
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "saving-pressure"
      },
      "source": [
        "Here, you can also pass how much variance you want PCA to capture. Let's pass 0.95 as a parameter to the PCA model, which means that PCA will hold 90% of the variance and the number of components required to capture 95% variance will be used."
      ],
      "id": "saving-pressure"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "operating-munich"
      },
      "source": [
        "from sklearn.decomposition import PCA\n",
        "pca = PCA(0.95)\n",
        "principalComponents_cifar = pca.fit(x_train_flat)"
      ],
      "id": "operating-munich",
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "alike-greeting"
      },
      "source": [
        "Now let's find out how many n_components PCA used to capture 0.95 variance."
      ],
      "id": "alike-greeting"
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "background_save": true
        },
        "id": "ignored-cincinnati",
        "outputId": "0f925a2e-ddfd-4e8f-a662-3a10d2b4e8f4"
      },
      "source": [
        "pca.n_components_"
      ],
      "id": "ignored-cincinnati",
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "217"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 0
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "smooth-rainbow"
      },
      "source": [
        "From the above output, you can observe that to achieve 90% variance, the dimension was reduced to 217 principal components from the actual 3072 dimensions.\n",
        "\n",
        "Finally, you will apply transform on both the training and test set to generate a transformed dataset from the parameters generated from the fit method."
      ],
      "id": "smooth-rainbow"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "irish-distribution"
      },
      "source": [
        "train_img_pca = pca.transform(x_train_flat)\n",
        "test_img_pca = pca.transform(x_test_flat)"
      ],
      "id": "irish-distribution",
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "internal-screening"
      },
      "source": [
        "Next, let's quickly import the necessary libraries to run the deep learning model."
      ],
      "id": "internal-screening"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "constant-category"
      },
      "source": [
        "from keras.models import Sequential\n",
        "from keras.layers import Dense\n",
        "from keras.utils import np_utils\n",
        "from keras.optimizers import RMSprop"
      ],
      "id": "constant-category",
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "coated-compromise"
      },
      "source": [
        "Now, you will convert your training and testing labels to one-hot encoding vector.\n",
        "\n"
      ],
      "id": "coated-compromise"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "previous-february"
      },
      "source": [
        "y_train = np_utils.to_categorical(y_train)\n",
        "y_test = np_utils.to_categorical(y_test)"
      ],
      "id": "previous-february",
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "subtle-reggae"
      },
      "source": [
        "# Let's define the number of epochs, number of classes, and the batch size for your model.\n",
        "batch_size = 128\n",
        "num_classes = 10\n",
        "epochs = 20"
      ],
      "id": "subtle-reggae",
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hairy-permission",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "414532d9-ff01-4a28-f5ef-c285347af6d4"
      },
      "source": [
        "model = Sequential()\n",
        "model.add(Dense(1024, activation='relu', input_shape=(217,)))\n",
        "model.add(Dense(1024, activation='relu'))\n",
        "model.add(Dense(512, activation='relu'))\n",
        "model.add(Dense(256, activation='relu'))\n",
        "model.add(Dense(num_classes, activation='softmax'))\n",
        "\n",
        "model.summary()"
      ],
      "id": "hairy-permission",
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Model: \"sequential_1\"\n",
            "_________________________________________________________________\n",
            "Layer (type)                 Output Shape              Param #   \n",
            "=================================================================\n",
            "dense_5 (Dense)              (None, 1024)              223232    \n",
            "_________________________________________________________________\n",
            "dense_6 (Dense)              (None, 1024)              1049600   \n",
            "_________________________________________________________________\n",
            "dense_7 (Dense)              (None, 512)               524800    \n",
            "_________________________________________________________________\n",
            "dense_8 (Dense)              (None, 256)               131328    \n",
            "_________________________________________________________________\n",
            "dense_9 (Dense)              (None, 10)                2570      \n",
            "=================================================================\n",
            "Total params: 1,931,530\n",
            "Trainable params: 1,931,530\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "outstanding-event",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3b2d3919-30e4-4507-d09c-6614dd208e72"
      },
      "source": [
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=RMSprop(),\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "history = model.fit(train_img_pca, y_train,batch_size=batch_size,epochs=epochs,verbose=1,\n",
        "                    validation_data=(test_img_pca, y_test))"
      ],
      "id": "outstanding-event",
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "391/391 [==============================] - 2s 4ms/step - loss: 1.7912 - accuracy: 0.3602 - val_loss: 1.4418 - val_accuracy: 0.4825\n",
            "Epoch 2/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 1.3010 - accuracy: 0.5379 - val_loss: 1.3588 - val_accuracy: 0.5245\n",
            "Epoch 3/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 1.0602 - accuracy: 0.6216 - val_loss: 1.3254 - val_accuracy: 0.5449\n",
            "Epoch 4/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.8462 - accuracy: 0.6970 - val_loss: 1.3331 - val_accuracy: 0.5596\n",
            "Epoch 5/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.6551 - accuracy: 0.7640 - val_loss: 1.4740 - val_accuracy: 0.5636\n",
            "Epoch 6/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.4992 - accuracy: 0.8236 - val_loss: 1.6968 - val_accuracy: 0.5627\n",
            "Epoch 7/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.3819 - accuracy: 0.8653 - val_loss: 1.8473 - val_accuracy: 0.5372\n",
            "Epoch 8/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.3045 - accuracy: 0.8935 - val_loss: 2.3053 - val_accuracy: 0.5567\n",
            "Epoch 9/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.2566 - accuracy: 0.9132 - val_loss: 2.3476 - val_accuracy: 0.5554\n",
            "Epoch 10/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.2201 - accuracy: 0.9272 - val_loss: 2.4639 - val_accuracy: 0.5654\n",
            "Epoch 11/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1903 - accuracy: 0.9372 - val_loss: 2.7148 - val_accuracy: 0.5560\n",
            "Epoch 12/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1811 - accuracy: 0.9413 - val_loss: 2.9707 - val_accuracy: 0.5630\n",
            "Epoch 13/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1659 - accuracy: 0.9476 - val_loss: 3.1105 - val_accuracy: 0.5596\n",
            "Epoch 14/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1530 - accuracy: 0.9538 - val_loss: 3.4211 - val_accuracy: 0.5579\n",
            "Epoch 15/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1480 - accuracy: 0.9559 - val_loss: 3.4050 - val_accuracy: 0.5572\n",
            "Epoch 16/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1454 - accuracy: 0.9565 - val_loss: 3.1114 - val_accuracy: 0.5567\n",
            "Epoch 17/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1346 - accuracy: 0.9595 - val_loss: 3.4983 - val_accuracy: 0.5476\n",
            "Epoch 18/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1316 - accuracy: 0.9624 - val_loss: 3.9248 - val_accuracy: 0.5431\n",
            "Epoch 19/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1314 - accuracy: 0.9627 - val_loss: 3.3444 - val_accuracy: 0.5500\n",
            "Epoch 20/20\n",
            "391/391 [==============================] - 1s 3ms/step - loss: 0.1268 - accuracy: 0.9633 - val_loss: 3.8732 - val_accuracy: 0.5633\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "attached-lecture"
      },
      "source": [
        "From the above output, you can observe that the time taken for training each epoch was just 1 seconds on a GPU. The model did a decent job on the training data, achieving 96% accuracy while it achieved only 56% accuracy on the test dat. This means that it overfitted the training data. However, remember that the data was projected to 99 dimensions from 3072 dimensions and despite that it did a great job!\n",
        "\n",
        "Finally, let's see how much time the model takes to train on the original dataset and how much accuracy it can achieve using the same deep learning model."
      ],
      "id": "attached-lecture"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "alternative-february",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "944dc048-ed4f-48ad-ea6f-1511bbb89fe0"
      },
      "source": [
        "model = Sequential()\n",
        "model.add(Dense(1024, activation='relu', input_shape=(3072,)))\n",
        "model.add(Dense(1024, activation='relu'))\n",
        "model.add(Dense(512, activation='relu'))\n",
        "model.add(Dense(256, activation='relu'))\n",
        "model.add(Dense(num_classes, activation='softmax'))\n",
        "model.compile(loss='categorical_crossentropy',\n",
        "              optimizer=RMSprop(),\n",
        "              metrics=['accuracy'])\n",
        "\n",
        "history = model.fit(x_train_flat, y_train,batch_size=batch_size,epochs=epochs,verbose=1,\n",
        "                    validation_data=(x_test_flat, y_test))"
      ],
      "id": "alternative-february",
      "execution_count": 22,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 1/20\n",
            "391/391 [==============================] - 3s 6ms/step - loss: 2.9355 - accuracy: 0.1808 - val_loss: 1.8509 - val_accuracy: 0.3329\n",
            "Epoch 2/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.8855 - accuracy: 0.3223 - val_loss: 1.7238 - val_accuracy: 0.3797\n",
            "Epoch 3/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.7585 - accuracy: 0.3701 - val_loss: 1.7551 - val_accuracy: 0.3790\n",
            "Epoch 4/20\n",
            "391/391 [==============================] - 2s 6ms/step - loss: 1.6749 - accuracy: 0.4037 - val_loss: 1.6172 - val_accuracy: 0.4302\n",
            "Epoch 5/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.6169 - accuracy: 0.4198 - val_loss: 1.6115 - val_accuracy: 0.4292\n",
            "Epoch 6/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.5639 - accuracy: 0.4431 - val_loss: 1.6247 - val_accuracy: 0.4211\n",
            "Epoch 7/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.5237 - accuracy: 0.4578 - val_loss: 1.5168 - val_accuracy: 0.4617\n",
            "Epoch 8/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.5035 - accuracy: 0.4631 - val_loss: 1.5467 - val_accuracy: 0.4522\n",
            "Epoch 9/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.4610 - accuracy: 0.4803 - val_loss: 1.5559 - val_accuracy: 0.4545\n",
            "Epoch 10/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.4358 - accuracy: 0.4908 - val_loss: 1.5702 - val_accuracy: 0.4488\n",
            "Epoch 11/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.4241 - accuracy: 0.4932 - val_loss: 1.6496 - val_accuracy: 0.4495\n",
            "Epoch 12/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3940 - accuracy: 0.5058 - val_loss: 1.5528 - val_accuracy: 0.4553\n",
            "Epoch 13/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3828 - accuracy: 0.5067 - val_loss: 1.5175 - val_accuracy: 0.4754\n",
            "Epoch 14/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3652 - accuracy: 0.5146 - val_loss: 1.8389 - val_accuracy: 0.4067\n",
            "Epoch 15/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3388 - accuracy: 0.5236 - val_loss: 1.5507 - val_accuracy: 0.4703\n",
            "Epoch 16/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3335 - accuracy: 0.5298 - val_loss: 1.5612 - val_accuracy: 0.4765\n",
            "Epoch 17/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3089 - accuracy: 0.5360 - val_loss: 1.5887 - val_accuracy: 0.4641\n",
            "Epoch 18/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.3040 - accuracy: 0.5414 - val_loss: 1.5527 - val_accuracy: 0.4790\n",
            "Epoch 19/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.2760 - accuracy: 0.5457 - val_loss: 1.6176 - val_accuracy: 0.4465\n",
            "Epoch 20/20\n",
            "391/391 [==============================] - 2s 5ms/step - loss: 1.2764 - accuracy: 0.5475 - val_loss: 1.7202 - val_accuracy: 0.4430\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "timely-passenger"
      },
      "source": [
        "From the above output, it is quite evident that the time taken for training each epoch was around 2 seconds on a CPU which was almost half times more than the model trained on the PCA output.\n",
        "\n",
        "Moreover, both the training and testing accuracy is less than the accuracy you achieved with the 99 principal components as an input to the model.\n",
        "\n",
        "So, by applying PCA on the training data you were able to train your deep learning algorithm not only fast, but it also achieved better accuracy on the testing data when compared with the deep learning algorithm trained with original training data."
      ],
      "id": "timely-passenger"
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "experienced-china"
      },
      "source": [
        ""
      ],
      "id": "experienced-china",
      "execution_count": null,
      "outputs": []
    }
  ]
}