ML_Tuning.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ecI9eXFD2atD"
      },
      "source": [
        "# Hyperparameter Tuning of Machine Learning Models\n",
        "\n",
        "Link to notebook: https://colab.research.google.com/drive/1Qz3C2gdh5CypkH3LAlZZDh3CnmoD0neN?usp=sharing\n",
        "\n",
        "In this notebook, we attempt to improve on the limitation of this paper: Rahman A & Hossen MdS (2019) Sentiment analysis on movie review data using Machine Learning Approach. 2019 International Conference on Bangla Speech and Language Processing (ICBSLP). doi: https://ieeexplore.ieee.org/document/9084046\n",
        "\n",
        "The authors did not conduct hyperparameter tuning of the Machine Learning Models. For this notebook, we attempt to explore with the available hyperparameters for the following ML models: Logistic Regression, Multinomial Naive Bayes, Random Forest Classifier."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {
        "id": "rJYT-tQtsCCQ"
      },
      "outputs": [],
      "source": [
        "from sklearn.ensemble import RandomForestClassifier\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.naive_bayes import MultinomialNB, GaussianNB\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
        "from sklearn.metrics import classification_report, confusion_matrix\n",
        "from sklearn.model_selection import GridSearchCV, train_test_split, RandomizedSearchCV\n",
        "from sklearn.pipeline import Pipeline\n",
        "from sklearn.metrics import accuracy_score\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NnTMIYa9-xV1"
      },
      "source": [
        "# Reading the data"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "19TpkU5ysWH1",
        "outputId": "4ac54937-d83a-4b6d-add4-2a8e68792c4a"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ],
      "source": [
        "# This cell reads files from Google Colab. If not using Colab, change the file directories accordingly\n",
        "from google.colab import drive\n",
        "drive.mount('/content/drive')\n",
        "\n",
        "train = pd.read_csv('/content/drive/MyDrive/Datasets/ML_train.csv', index_col = 0)\n",
        "test = pd.read_csv('/content/drive/MyDrive/Datasets/ML_test.csv', index_col = 0)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {
        "id": "-ZxpU6V8wG28"
      },
      "outputs": [],
      "source": [
        "def good_bad(row):\n",
        "  if row > 4:\n",
        "    return 1\n",
        "  else:\n",
        "    return 0"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "OEUU06OywHbu"
      },
      "outputs": [],
      "source": [
        "train['Sentiment'] = train['Sentiment'].apply(good_bad)\n",
        "test['Sentiment'] = test['Sentiment'].apply(good_bad)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "nDTwI6visXXt",
        "outputId": "e53e72e2-09f2-4835-8ea7-f021adc3dbc8"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"train\",\n  \"rows\": 63848,\n  \"fields\": [\n    {\n      \"column\": \"Text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 39653,\n        \"samples\": [\n          \"show accurately described tv bubblegum is chewing good taste last long time like bubblegum leave bad taste mouth many chewsbr show feature simple question take form simply game like hangman guessing simple phrase guessing item short list caller are contestant anybody playbr question though are terribly easy is techno music pumped background is fashioned continuous loop never seems get old never add much show anyway hostess however regardless one is extremely personable pleasant enough watch make amusing facial expression rather enjoy british womanbr is nothing show is simplistic uncompelling although entertaining absolutely nothing else is\",\n          \"unless is huge ironic conspiracy going jaw dropped read positive review film cannot believe film wa even released is badbr admit is kind movie tried watch objectively anyway know bad is funny wa still offended sheer awfulnessbr acting is atrocious cannot have watched rush guessing wa one take per scene really is terrible is worst film have seen many year fact would even call film is tragedy gay black friend noone actually call gay is implied is crazy homophobic is good fact is downright vomit inducing joke die pathetic arses music is bad defies belief person compiled soundtrack essentially chose earmutilatingly bad song ever heard put waste film stock oh good christ cannot believe 80 produced utter garbage like grew cannot find one thing worth note must have dark time cinemagoerbr even contemplate watching film go see psychiatrist will accordingly slap sick sick person\",\n          \"is movie basic element genre make want cry make laugh disgust make angry etc topic story is fortunately disease drug is common trend gay themed movie day focus social interaction character could considered high school elite play direction could little bit sophisticated hand is somehow better really show distress character are experiencing wa intended is remarkable job assuredly achievement specially young director is actually good story give little inside is fat girl acknowledge\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Sentiment\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "train"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-b4f53fc8-9d3f-45ba-92cc-fbf05290a686\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Text</th>\n",
              "      <th>Sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>saw premiered rewatched ifc is great telling m...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>movie is one alltime favorite think sean penn ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>describing stalingrad war film may bit inaccur...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>tale two sister one creepiest film have seen r...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>well notice imdb offered plot infothat is is p...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b4f53fc8-9d3f-45ba-92cc-fbf05290a686')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-b4f53fc8-9d3f-45ba-92cc-fbf05290a686 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-b4f53fc8-9d3f-45ba-92cc-fbf05290a686');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-8f8278b6-9422-4b2b-bb7b-664768b01336\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-8f8278b6-9422-4b2b-bb7b-664768b01336')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-8f8278b6-9422-4b2b-bb7b-664768b01336 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                Text  Sentiment\n",
              "0  saw premiered rewatched ifc is great telling m...          1\n",
              "1  movie is one alltime favorite think sean penn ...          1\n",
              "2  describing stalingrad war film may bit inaccur...          1\n",
              "3  tale two sister one creepiest film have seen r...          1\n",
              "4  well notice imdb offered plot infothat is is p...          0"
            ]
          },
          "execution_count": 5,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "train.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "RAkivcI0sgu9",
        "outputId": "2c15d789-08f0-45ac-9e14-fb8959557645"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"trial_train\",\n  \"rows\": 63848,\n  \"fields\": [\n    {\n      \"column\": \"Text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 39653,\n        \"samples\": [\n          \"show accurately described tv bubblegum is chewing good taste last long time like bubblegum leave bad taste mouth many chewsbr show feature simple question take form simply game like hangman guessing simple phrase guessing item short list caller are contestant anybody playbr question though are terribly easy is techno music pumped background is fashioned continuous loop never seems get old never add much show anyway hostess however regardless one is extremely personable pleasant enough watch make amusing facial expression rather enjoy british womanbr is nothing show is simplistic uncompelling although entertaining absolutely nothing else is\",\n          \"unless is huge ironic conspiracy going jaw dropped read positive review film cannot believe film wa even released is badbr admit is kind movie tried watch objectively anyway know bad is funny wa still offended sheer awfulnessbr acting is atrocious cannot have watched rush guessing wa one take per scene really is terrible is worst film have seen many year fact would even call film is tragedy gay black friend noone actually call gay is implied is crazy homophobic is good fact is downright vomit inducing joke die pathetic arses music is bad defies belief person compiled soundtrack essentially chose earmutilatingly bad song ever heard put waste film stock oh good christ cannot believe 80 produced utter garbage like grew cannot find one thing worth note must have dark time cinemagoerbr even contemplate watching film go see psychiatrist will accordingly slap sick sick person\",\n          \"is movie basic element genre make want cry make laugh disgust make angry etc topic story is fortunately disease drug is common trend gay themed movie day focus social interaction character could considered high school elite play direction could little bit sophisticated hand is somehow better really show distress character are experiencing wa intended is remarkable job assuredly achievement specially young director is actually good story give little inside is fat girl acknowledge\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Sentiment\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "trial_train"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-e2bdb543-b171-4f8e-9284-a87c30519e58\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Text</th>\n",
              "      <th>Sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>saw premiered rewatched ifc is great telling m...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>movie is one alltime favorite think sean penn ...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>describing stalingrad war film may bit inaccur...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>tale two sister one creepiest film have seen r...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>well notice imdb offered plot infothat is is p...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e2bdb543-b171-4f8e-9284-a87c30519e58')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-e2bdb543-b171-4f8e-9284-a87c30519e58 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-e2bdb543-b171-4f8e-9284-a87c30519e58');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-5fa32d3b-e0c8-4425-9ec7-dc134c9ae630\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-5fa32d3b-e0c8-4425-9ec7-dc134c9ae630')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-5fa32d3b-e0c8-4425-9ec7-dc134c9ae630 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                Text  Sentiment\n",
              "0  saw premiered rewatched ifc is great telling m...          1\n",
              "1  movie is one alltime favorite think sean penn ...          1\n",
              "2  describing stalingrad war film may bit inaccur...          1\n",
              "3  tale two sister one creepiest film have seen r...          1\n",
              "4  well notice imdb offered plot infothat is is p...          0"
            ]
          },
          "execution_count": 6,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trial_train = train.copy()\n",
        "trial_train.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "PSwJtDopslYF"
      },
      "outputs": [],
      "source": [
        "class2_X_train = trial_train['Text']\n",
        "class2_y_train = trial_train['Sentiment']"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "iKG9KW3RsmtE",
        "outputId": "50a2afbf-21e9-4ab9-fe65-f8b44cb4b3f4"
      },
      "outputs": [
        {
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "summary": "{\n  \"name\": \"trial_test\",\n  \"rows\": 9915,\n  \"fields\": [\n    {\n      \"column\": \"Text\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 9915,\n        \"samples\": [\n          \"movie is one awful have ever seen is dialogue awful never end will think is ending is long is 140 160 minute even know know will never watch is like someone took romantic comedy took comedy decided downplay romance leaving u pile crap managed make way screen take word find terrible film is\",\n          \"have start saying long time since have seen have seen 5 time wonderful little romp wa clearly inspired musicalcomedy pairing new fading star musical group prominence kay kysers mystery would good examplebr spike jones unleashed is best part show band play many tune are part action fine job support hugh obrien play face buddy hackett part rumor wa offered lou costello thus abbott costello replacing lead know wa truebr pleasant movie important have much spike jones band film history wish wa released have seen heard two decade hope is lost\",\n          \"excellent film cried cried loved loved wa frustrated film touched heart wa reality check since is reality 19 year old soldier\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"Sentiment\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 0,\n        \"max\": 1,\n        \"num_unique_values\": 2,\n        \"samples\": [\n          0,\n          1\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}",
              "type": "dataframe",
              "variable_name": "trial_test"
            },
            "text/html": [
              "\n",
              "  <div id=\"df-6d5130c6-9ac4-4636-b384-1685f353b489\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Text</th>\n",
              "      <th>Sentiment</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>frank horrigan clint eastwood is harassed mitc...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>carly jones elisha curtberth bad boy brother n...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>dig would say anyone even like metallica see k...</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>is great premise movie overall plot is origina...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>underground comedy movie is possibly worst tra...</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-6d5130c6-9ac4-4636-b384-1685f353b489')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-6d5130c6-9ac4-4636-b384-1685f353b489 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-6d5130c6-9ac4-4636-b384-1685f353b489');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-a7098391-4a8b-4277-99e1-ccc78b94c09d\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a7098391-4a8b-4277-99e1-ccc78b94c09d')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-a7098391-4a8b-4277-99e1-ccc78b94c09d button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "text/plain": [
              "                                                Text  Sentiment\n",
              "0  frank horrigan clint eastwood is harassed mitc...          1\n",
              "1  carly jones elisha curtberth bad boy brother n...          1\n",
              "2  dig would say anyone even like metallica see k...          1\n",
              "3  is great premise movie overall plot is origina...          0\n",
              "4  underground comedy movie is possibly worst tra...          0"
            ]
          },
          "execution_count": 8,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "trial_test = test.copy()\n",
        "trial_test.head()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "id": "zJ37L122sqgN"
      },
      "outputs": [],
      "source": [
        "class2_X_test = trial_test['Text']\n",
        "class2_y_test = trial_test['Sentiment']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3updXoh6srgc"
      },
      "source": [
        "# Vectorization"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gAsZGa2FsvjM"
      },
      "source": [
        "BOW Vectorizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "id": "lzk1Z5yustaF"
      },
      "outputs": [],
      "source": [
        "bow_vectorizer = CountVectorizer()\n",
        "bow_vectorizer.fit(class2_X_train)\n",
        "\n",
        "bow_X_train = bow_vectorizer.transform(class2_X_train)\n",
        "bow_X_test = bow_vectorizer.transform(class2_X_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "tfc2F-dpsxEV"
      },
      "source": [
        "TF-IDF Vectorizer"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "id": "K6LOD-a7QXqk"
      },
      "outputs": [],
      "source": [
        "# ngram_range=(1, 3): This tells the vectorizer to consider unigrams, bigrams, and trigrams\n",
        "# min_df=2: This means an n-gram must appear in at least two documents to be considered. This helps in removing very rare n-grams that might not be useful for modeling.\n",
        "# max_df=0.85: This means an n-gram appearing in more than 85% of the documents will be ignored, helping in filtering out too common n-grams.\n",
        "\n",
        "tfidf_vectorizer = TfidfVectorizer(ngram_range=(1, 3), min_df=2, max_df=0.85)\n",
        "tfidf_vectorizer.fit(class2_X_train)\n",
        "\n",
        "tfidf_X_train = tfidf_vectorizer.transform(class2_X_train)\n",
        "tfidf_X_test = tfidf_vectorizer.transform(class2_X_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VMeW9HAmvpAF"
      },
      "source": [
        "# Function to help us evaluate model performance"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "fyhDavQpvjVd"
      },
      "outputs": [],
      "source": [
        "def train_and_eval(model, trainX, trainY, testX, testY):\n",
        "\n",
        "    # training the model\n",
        "    fitted_model = model.fit(trainX, trainY)\n",
        "\n",
        "    # getting predictions\n",
        "    y_preds_train = fitted_model.predict(trainX)\n",
        "    y_preds_test = fitted_model.predict(testX)\n",
        "\n",
        "    # evaluating the model\n",
        "    print()\n",
        "    print(model)\n",
        "    print(f\"Train accuracy score : {accuracy_score(trainY, y_preds_train)}\")\n",
        "    print(f\"Test accuracy score : {accuracy_score(testY, y_preds_test)}\")\n",
        "    print(classification_report(testY, y_preds_test))\n",
        "    print('\\n',40*'-')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "o4vodA-gJGSM"
      },
      "source": [
        "## Logistic Regression Hyperparameter Tuning\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "epSeYn_PTb_L"
      },
      "source": [
        "Logistic Regression with BOW\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "scZWap9o3sbh"
      },
      "outputs": [],
      "source": [
        "#Selections of param grid are based on previous mini hyperparameter tuning to find better C values\n",
        "param_grid = {\n",
        "    'C': [0.1, 1, 10],\n",
        "    'penalty': ['l1', 'l2'],\n",
        "    'solver': ['liblinear'],\n",
        "    'max_iter': [400]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xfFn5gFi7Wjk"
      },
      "outputs": [],
      "source": [
        "log_model = LogisticRegression(random_state = 42, n_jobs = -1)\n",
        "clf = GridSearchCV(log_model, param_grid=param_grid, cv=3, verbose=4, scoring='accuracy')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "udDBhhbr7W4E",
        "outputId": "bdf0e671-c392-44e8-d127-d11309ce4905"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 6 candidates, totalling 18 fits\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.879 total time=   2.2s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.875 total time=   1.5s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.878 total time=   1.3s\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.888 total time=  15.6s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.885 total time=   9.5s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.890 total time=   7.0s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.874 total time=   1.6s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.876 total time=   2.1s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.876 total time=   1.9s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.883 total time=  10.9s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.881 total time=  14.4s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.883 total time=  18.7s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.865 total time=   2.0s\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.864 total time=   2.0s\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.868 total time=   2.0s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l2, solver=liblinear;, score=0.875 total time=  25.1s\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l2, solver=liblinear;, score=0.875 total time=  23.6s\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l2, solver=liblinear;, score=0.876 total time=  18.2s\n"
          ]
        }
      ],
      "source": [
        "best_clf = clf.fit(bow_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "SJuDJCgD7Y6x",
        "outputId": "29ce231d-df0b-4359-8f39-11b0a31a301f"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'C': 0.1, 'max_iter': 400, 'penalty': 'l2', 'solver': 'liblinear'}"
            ]
          },
          "execution_count": 52,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "best_clf.best_params_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gPcP7eiu7aXN",
        "outputId": "c1e19a0b-b8a4-4900-e2f1-37884153719a"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.8873499923593785"
            ]
          },
          "execution_count": 53,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "best_clf.best_score_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OTsDE-IS7cC0"
      },
      "outputs": [],
      "source": [
        "best_log = LogisticRegression(**best_clf.best_params_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2CUAzxIu7dfD",
        "outputId": "2552668c-0080-417b-89ab-a51e90043d10"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "LogisticRegression(C=0.1, max_iter=400, solver='liblinear')\n",
            "Train accuracy score : 0.96795\n",
            "Test accuracy score : 0.8924892489248925\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.90      0.88      0.89      5000\n",
            "           1       0.89      0.90      0.89      4999\n",
            "\n",
            "    accuracy                           0.89      9999\n",
            "   macro avg       0.89      0.89      0.89      9999\n",
            "weighted avg       0.89      0.89      0.89      9999\n",
            "\n",
            "\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "train_and_eval(best_log, bow_X_train, class2_y_train, bow_X_test, class2_y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "H0hzS2IeuT64"
      },
      "source": [
        "Best Logistic Regression Model with BOW gives us 89.2% accuracy on test data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AElZgE7pTadc"
      },
      "source": [
        "Logistic Regression with TF-IDF\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "QaDJ82BN0J7N"
      },
      "outputs": [],
      "source": [
        "param_grid = {\n",
        "    'C': [0.1, 1, 10],\n",
        "    'penalty': ['l1', 'l2'],\n",
        "    'solver': ['liblinear', 'saga'],\n",
        "    'max_iter': [400]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "STpiKGFA0o9s"
      },
      "outputs": [],
      "source": [
        "log_model = LogisticRegression(random_state = 42, n_jobs = -1)\n",
        "clf = RandomizedSearchCV(log_model, param_distributions=param_grid, cv=3, verbose=4, scoring='accuracy', random_state=42, n_iter=20)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "37690A4h04EQ",
        "outputId": "077fca93-1c50-4225-cbf3-906f10bb2db7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 12 candidates, totalling 36 fits\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.879 total time=   1.3s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.875 total time=   1.3s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.878 total time=   1.4s\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l1, solver=saga;, score=0.878 total time= 1.9min\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l1, solver=saga;, score=0.873 total time= 2.0min\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l1, solver=saga;, score=0.878 total time= 1.9min\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.888 total time=   8.6s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.884 total time=  10.4s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.890 total time=  10.4s\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l2, solver=saga;, score=0.888 total time=  49.6s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l2, solver=saga;, score=0.885 total time=  48.0s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l2, solver=saga;, score=0.889 total time=  50.1s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.874 total time=   1.7s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.876 total time=   1.7s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.876 total time=   2.0s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l1, solver=saga;, score=0.884 total time=11.6min\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l1, solver=saga;, score=0.883 total time=12.7min\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l1, solver=saga;, score=0.886 total time=12.7min\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.882 total time=  18.5s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.881 total time=  17.3s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.883 total time=  22.8s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l2, solver=saga;, score=0.886 total time=  50.0s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l2, solver=saga;, score=0.883 total time=  47.8s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l2, solver=saga;, score=0.888 total time=  47.0s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.865 total time=   2.5s\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.864 total time=   3.1s\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.868 total time=   2.8s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l1, solver=saga;, score=0.886 total time=57.8min\n"
          ]
        }
      ],
      "source": [
        "best_clf = clf.fit(bow_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "m9KfmPZe1qNz"
      },
      "outputs": [],
      "source": [
        "log_model = LogisticRegression(random_state = 42, n_jobs = -1)\n",
        "clf = RandomizedSearchCV(log_model, param_distributions=param_grid, cv=3, verbose=4, scoring='accuracy', random_state=42, n_iter=20)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "t0qlN8-x1sLY",
        "outputId": "dcb7d211-88e6-45ff-9d20-df20e8328b90"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 12 candidates, totalling 36 fits\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.766 total time=  18.2s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.765 total time=  15.9s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l1, solver=liblinear;, score=0.772 total time=  13.9s\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l1, solver=saga;, score=0.765 total time=  20.3s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l1, solver=saga;, score=0.765 total time=  17.4s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l1, solver=saga;, score=0.772 total time=  16.3s\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.854 total time=   3.7s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.854 total time=   4.7s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l2, solver=liblinear;, score=0.859 total time=   4.8s\n",
            "[CV 1/3] END C=0.1, max_iter=400, penalty=l2, solver=saga;, score=0.854 total time=  15.0s\n",
            "[CV 2/3] END C=0.1, max_iter=400, penalty=l2, solver=saga;, score=0.854 total time=  15.1s\n",
            "[CV 3/3] END C=0.1, max_iter=400, penalty=l2, solver=saga;, score=0.859 total time=  15.6s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.872 total time=  12.0s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.867 total time=   9.1s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l1, solver=liblinear;, score=0.871 total time=  10.3s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l1, solver=saga;, score=0.872 total time= 4.5min\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l1, solver=saga;, score=0.867 total time= 3.7min\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l1, solver=saga;, score=0.871 total time= 2.1min\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.895 total time=   8.0s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.888 total time=   8.7s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l2, solver=liblinear;, score=0.894 total time=   6.9s\n",
            "[CV 1/3] END C=1, max_iter=400, penalty=l2, solver=saga;, score=0.895 total time=  13.4s\n",
            "[CV 2/3] END C=1, max_iter=400, penalty=l2, solver=saga;, score=0.888 total time=  13.5s\n",
            "[CV 3/3] END C=1, max_iter=400, penalty=l2, solver=saga;, score=0.894 total time=  13.3s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.891 total time=  12.2s\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.888 total time=   9.8s\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l1, solver=liblinear;, score=0.890 total time=   8.9s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l1, solver=saga;, score=0.892 total time=12.2min\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l1, solver=saga;, score=0.890 total time= 9.2min\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l1, solver=saga;, score=0.890 total time= 9.5min\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l2, solver=liblinear;, score=0.906 total time=  11.9s\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l2, solver=liblinear;, score=0.900 total time=  12.4s\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l2, solver=liblinear;, score=0.905 total time=  12.1s\n",
            "[CV 1/3] END C=10, max_iter=400, penalty=l2, solver=saga;, score=0.906 total time=  18.7s\n",
            "[CV 2/3] END C=10, max_iter=400, penalty=l2, solver=saga;, score=0.900 total time=  20.2s\n",
            "[CV 3/3] END C=10, max_iter=400, penalty=l2, solver=saga;, score=0.905 total time=  18.7s\n",
            "CPU times: user 1h 5min 52s, sys: 41.7 s, total: 1h 6min 33s\n",
            "Wall time: 47min 51s\n"
          ]
        }
      ],
      "source": [
        "%%time\n",
        "best_clf = clf.fit(tfidf_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "r47nU__71xe6",
        "outputId": "a6c1fb19-0821-4bee-8920-632ee0a229f6"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'solver': 'liblinear', 'penalty': 'l2', 'max_iter': 400, 'C': 10}"
            ]
          },
          "execution_count": 56,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "best_clf.best_params_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "aKLFdeap1xc2",
        "outputId": "d462d8dd-0945-4564-fecd-b006fcbd5ae3"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.9037249386311904"
            ]
          },
          "execution_count": 57,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "best_clf.best_score_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "xK84tE_P10uy"
      },
      "outputs": [],
      "source": [
        "best_log = LogisticRegression(**best_clf.best_params_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "p7sVhhjlfitN",
        "outputId": "d2fbd0ce-99f3-4253-e442-145f9d262305"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "LogisticRegression(C=10, max_iter=400, solver='liblinear')\n",
            "Train accuracy score : 0.999175\n",
            "Test accuracy score : 0.9062906290629063\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.91      0.90      0.91      5000\n",
            "           1       0.90      0.91      0.91      4999\n",
            "\n",
            "    accuracy                           0.91      9999\n",
            "   macro avg       0.91      0.91      0.91      9999\n",
            "weighted avg       0.91      0.91      0.91      9999\n",
            "\n",
            "\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "train_and_eval(best_log, tfidf_X_train, class2_y_train, tfidf_X_test, class2_y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "IaYhV1HEub30"
      },
      "source": [
        "Best Logistic Regression Model with TFIDF gives us 90.6% accuracy on test data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ln_nVbrKw9AM"
      },
      "source": [
        "## Random Forest Hyperparameter Tuning\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F5zPQ09NEGJs"
      },
      "source": [
        "Random Forest Classifier with BOW"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "4LjVbp_Zxkpd"
      },
      "outputs": [],
      "source": [
        "param_grid = {\n",
        "    'n_estimators': [50, 100, 150, 200, 250],\n",
        "    'criterion': ['gini', 'entropy', 'log_loss'],\n",
        "    'max_depth': [20, 30, 50, 75, 100],\n",
        "    'min_samples_split': [10, 25, 50, 100, 150, 200],\n",
        "    'min_samples_leaf': [25, 50, 100]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "6RdR--bazG0I"
      },
      "outputs": [],
      "source": [
        "rf_model = RandomForestClassifier(random_state = 42, n_jobs = -1)\n",
        "clf = RandomizedSearchCV(rf_model, param_distributions=param_grid, cv=3, verbose=4, scoring='accuracy', random_state=42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1_Dyf51XzJaN",
        "outputId": "232fbd11-e8e8-4195-c611-08260d2fdfe0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 10 candidates, totalling 30 fits\n",
            "[CV 1/3] END criterion=log_loss, max_depth=50, min_samples_leaf=50, min_samples_split=100, n_estimators=100;, score=0.847 total time=  23.6s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=50, min_samples_leaf=50, min_samples_split=100, n_estimators=100;, score=0.835 total time=  19.5s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=50, min_samples_leaf=50, min_samples_split=100, n_estimators=100;, score=0.843 total time=  17.0s\n",
            "[CV 1/3] END criterion=entropy, max_depth=100, min_samples_leaf=50, min_samples_split=150, n_estimators=50;, score=0.834 total time=  11.0s\n",
            "[CV 2/3] END criterion=entropy, max_depth=100, min_samples_leaf=50, min_samples_split=150, n_estimators=50;, score=0.821 total time=   9.4s\n",
            "[CV 3/3] END criterion=entropy, max_depth=100, min_samples_leaf=50, min_samples_split=150, n_estimators=50;, score=0.838 total time=   7.9s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=100, min_samples_leaf=50, min_samples_split=10, n_estimators=250;, score=0.849 total time=  48.7s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=100, min_samples_leaf=50, min_samples_split=10, n_estimators=250;, score=0.846 total time=  44.9s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=100, min_samples_leaf=50, min_samples_split=10, n_estimators=250;, score=0.853 total time=  44.8s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=50, min_samples_leaf=50, min_samples_split=150, n_estimators=50;, score=0.835 total time=   9.7s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=50, min_samples_leaf=50, min_samples_split=150, n_estimators=50;, score=0.822 total time=   9.7s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=50, min_samples_leaf=50, min_samples_split=150, n_estimators=50;, score=0.838 total time=   8.8s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=50, min_samples_leaf=25, min_samples_split=100, n_estimators=50;, score=0.844 total time=  12.1s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=50, min_samples_leaf=25, min_samples_split=100, n_estimators=50;, score=0.842 total time=  12.1s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=50, min_samples_leaf=25, min_samples_split=100, n_estimators=50;, score=0.838 total time=  11.6s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=30, min_samples_leaf=50, min_samples_split=150, n_estimators=250;, score=0.848 total time=  41.7s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=30, min_samples_leaf=50, min_samples_split=150, n_estimators=250;, score=0.845 total time=  40.5s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=30, min_samples_leaf=50, min_samples_split=150, n_estimators=250;, score=0.852 total time=  40.3s\n",
            "[CV 1/3] END criterion=gini, max_depth=30, min_samples_leaf=50, min_samples_split=10, n_estimators=100;, score=0.844 total time=  18.1s\n",
            "[CV 2/3] END criterion=gini, max_depth=30, min_samples_leaf=50, min_samples_split=10, n_estimators=100;, score=0.833 total time=  15.9s\n",
            "[CV 3/3] END criterion=gini, max_depth=30, min_samples_leaf=50, min_samples_split=10, n_estimators=100;, score=0.844 total time=  15.6s\n",
            "[CV 1/3] END criterion=entropy, max_depth=20, min_samples_leaf=25, min_samples_split=100, n_estimators=100;, score=0.848 total time=  14.9s\n",
            "[CV 2/3] END criterion=entropy, max_depth=20, min_samples_leaf=25, min_samples_split=100, n_estimators=100;, score=0.838 total time=  14.4s\n",
            "[CV 3/3] END criterion=entropy, max_depth=20, min_samples_leaf=25, min_samples_split=100, n_estimators=100;, score=0.843 total time=  14.5s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=75, min_samples_leaf=100, min_samples_split=25, n_estimators=200;, score=0.841 total time=  25.6s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=75, min_samples_leaf=100, min_samples_split=25, n_estimators=200;, score=0.834 total time=  24.4s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=75, min_samples_leaf=100, min_samples_split=25, n_estimators=200;, score=0.843 total time=  25.4s\n",
            "[CV 1/3] END criterion=gini, max_depth=75, min_samples_leaf=100, min_samples_split=10, n_estimators=50;, score=0.822 total time=   5.8s\n",
            "[CV 2/3] END criterion=gini, max_depth=75, min_samples_leaf=100, min_samples_split=10, n_estimators=50;, score=0.804 total time=   6.1s\n",
            "[CV 3/3] END criterion=gini, max_depth=75, min_samples_leaf=100, min_samples_split=10, n_estimators=50;, score=0.827 total time=   6.2s\n"
          ]
        }
      ],
      "source": [
        "best_clf = clf.fit(bow_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "8ecTYX7SzKB0",
        "outputId": "c96ea575-c081-40c9-c120-668d2fe988e2"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'n_estimators': 250,\n",
              " 'min_samples_split': 10,\n",
              " 'min_samples_leaf': 50,\n",
              " 'max_depth': 100,\n",
              " 'criterion': 'log_loss'}"
            ]
          },
          "execution_count": 61,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "best_clf.best_params_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "h22ii8uLzJ_h",
        "outputId": "3dbdfdeb-9d1f-44da-ffab-2dbc2f5bf47b"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.8492000173110673"
            ]
          },
          "execution_count": 62,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "best_clf.best_score_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "oAGoKaCdzJ-M"
      },
      "outputs": [],
      "source": [
        "best_rf = RandomForestClassifier(**best_clf.best_params_)\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P5KteeEjzJ6d",
        "outputId": "777fde9e-f988-4410-df31-9cd9f69dc743"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "RandomForestClassifier(criterion='log_loss', max_depth=100, min_samples_leaf=50,\n",
            "                       min_samples_split=10, n_estimators=250)\n",
            "Train accuracy score : 0.86275\n",
            "Test accuracy score : 0.8525852585258525\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.87      0.83      0.85      5000\n",
            "           1       0.83      0.88      0.86      4999\n",
            "\n",
            "    accuracy                           0.85      9999\n",
            "   macro avg       0.85      0.85      0.85      9999\n",
            "weighted avg       0.85      0.85      0.85      9999\n",
            "\n",
            "\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "train_and_eval(best_rf, bow_X_train, class2_y_train, bow_X_test, class2_y_test)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DKB8y4v4wHQO"
      },
      "source": [
        "Best Random Forest Classifier with BOW gives us 85.3% accuracy on test data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NCHvXbystQTt"
      },
      "source": [
        "Random Forest with TF-IDF"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ya__XiQ6u_2N"
      },
      "outputs": [],
      "source": [
        "param_grid = {\n",
        "    'n_estimators': [50, 100, 150, 200, 500],\n",
        "    'criterion': ['gini', 'entropy', 'log_loss'],\n",
        "    'max_depth': [20, 30, 50, 75, 100],\n",
        "    'min_samples_split': [10, 25, 50, 100, 150, 200, 250],\n",
        "    'min_samples_leaf': [25, 50, 100]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gPpNCzG-u_2N"
      },
      "outputs": [],
      "source": [
        "rf_model = RandomForestClassifier(random_state = 42, n_jobs = -1)\n",
        "clf = RandomizedSearchCV(rf_model, param_distributions=param_grid, cv=3, verbose=4, scoring='accuracy', random_state=42)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "nYw27ZQPu_2O",
        "outputId": "169fb3bf-675e-4abf-8b91-801bb8edb277"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 3 folds for each of 10 candidates, totalling 30 fits\n",
            "[CV 1/3] END criterion=log_loss, max_depth=20, min_samples_leaf=100, min_samples_split=25, n_estimators=100;, score=0.811 total time=  28.1s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=20, min_samples_leaf=100, min_samples_split=25, n_estimators=100;, score=0.820 total time=  25.4s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=20, min_samples_leaf=100, min_samples_split=25, n_estimators=100;, score=0.817 total time=  20.1s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=75, min_samples_leaf=100, min_samples_split=150, n_estimators=500;, score=0.842 total time= 1.4min\n",
            "[CV 2/3] END criterion=log_loss, max_depth=75, min_samples_leaf=100, min_samples_split=150, n_estimators=500;, score=0.844 total time= 1.5min\n",
            "[CV 3/3] END criterion=log_loss, max_depth=75, min_samples_leaf=100, min_samples_split=150, n_estimators=500;, score=0.842 total time= 1.4min\n",
            "[CV 1/3] END criterion=entropy, max_depth=75, min_samples_leaf=25, min_samples_split=150, n_estimators=50;, score=0.844 total time=  22.8s\n",
            "[CV 2/3] END criterion=entropy, max_depth=75, min_samples_leaf=25, min_samples_split=150, n_estimators=50;, score=0.840 total time=  22.3s\n",
            "[CV 3/3] END criterion=entropy, max_depth=75, min_samples_leaf=25, min_samples_split=150, n_estimators=50;, score=0.845 total time=  25.5s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=50, min_samples_leaf=25, min_samples_split=250, n_estimators=500;, score=0.865 total time= 3.7min\n",
            "[CV 2/3] END criterion=log_loss, max_depth=50, min_samples_leaf=25, min_samples_split=250, n_estimators=500;, score=0.862 total time= 3.5min\n",
            "[CV 3/3] END criterion=log_loss, max_depth=50, min_samples_leaf=25, min_samples_split=250, n_estimators=500;, score=0.866 total time= 3.6min\n",
            "[CV 1/3] END criterion=log_loss, max_depth=20, min_samples_leaf=100, min_samples_split=50, n_estimators=50;, score=0.778 total time=   9.2s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=20, min_samples_leaf=100, min_samples_split=50, n_estimators=50;, score=0.790 total time=  10.7s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=20, min_samples_leaf=100, min_samples_split=50, n_estimators=50;, score=0.789 total time=   8.7s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=20, min_samples_leaf=50, min_samples_split=50, n_estimators=50;, score=0.819 total time=  11.8s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=20, min_samples_leaf=50, min_samples_split=50, n_estimators=50;, score=0.815 total time=  11.7s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=20, min_samples_leaf=50, min_samples_split=50, n_estimators=50;, score=0.805 total time=  12.8s\n",
            "[CV 1/3] END criterion=entropy, max_depth=100, min_samples_leaf=100, min_samples_split=200, n_estimators=500;, score=0.842 total time= 1.4min\n",
            "[CV 2/3] END criterion=entropy, max_depth=100, min_samples_leaf=100, min_samples_split=200, n_estimators=500;, score=0.844 total time= 1.4min\n",
            "[CV 3/3] END criterion=entropy, max_depth=100, min_samples_leaf=100, min_samples_split=200, n_estimators=500;, score=0.842 total time= 1.5min\n",
            "[CV 1/3] END criterion=gini, max_depth=30, min_samples_leaf=25, min_samples_split=100, n_estimators=100;, score=0.844 total time=  31.8s\n",
            "[CV 2/3] END criterion=gini, max_depth=30, min_samples_leaf=25, min_samples_split=100, n_estimators=100;, score=0.842 total time=  30.4s\n",
            "[CV 3/3] END criterion=gini, max_depth=30, min_samples_leaf=25, min_samples_split=100, n_estimators=100;, score=0.850 total time=  33.4s\n",
            "[CV 1/3] END criterion=gini, max_depth=100, min_samples_leaf=50, min_samples_split=50, n_estimators=100;, score=0.842 total time=  31.6s\n",
            "[CV 2/3] END criterion=gini, max_depth=100, min_samples_leaf=50, min_samples_split=50, n_estimators=100;, score=0.844 total time=  31.1s\n",
            "[CV 3/3] END criterion=gini, max_depth=100, min_samples_leaf=50, min_samples_split=50, n_estimators=100;, score=0.840 total time=  33.7s\n",
            "[CV 1/3] END criterion=log_loss, max_depth=30, min_samples_leaf=100, min_samples_split=50, n_estimators=200;, score=0.830 total time=  33.7s\n",
            "[CV 2/3] END criterion=log_loss, max_depth=30, min_samples_leaf=100, min_samples_split=50, n_estimators=200;, score=0.833 total time=  37.3s\n",
            "[CV 3/3] END criterion=log_loss, max_depth=30, min_samples_leaf=100, min_samples_split=50, n_estimators=200;, score=0.832 total time=  36.2s\n"
          ]
        }
      ],
      "source": [
        "best_clf = clf.fit(tfidf_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z-zQhETGwaA4",
        "outputId": "c4ab78c5-e383-4ed9-9e9b-fa0e90654fac"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'n_estimators': 500,\n",
              " 'min_samples_split': 250,\n",
              " 'min_samples_leaf': 25,\n",
              " 'max_depth': 50,\n",
              " 'criterion': 'log_loss'}"
            ]
          },
          "execution_count": 72,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# best parameters\n",
        "best_clf.best_params_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "HSTSgzTxwaA4",
        "outputId": "2db4748e-600b-4255-e88f-5129cda71834"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.8643249773309729"
            ]
          },
          "execution_count": 73,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# highest accuracy score\n",
        "best_clf.best_score_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "CNfl_43jwmb3"
      },
      "outputs": [],
      "source": [
        "best_rf = RandomForestClassifier(**best_clf.best_params_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "--ctDXx7wuOm",
        "outputId": "98070e9f-3c75-441f-93de-7d4671ed9dd1"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "RandomForestClassifier(criterion='log_loss', max_depth=50, min_samples_leaf=25,\n",
            "                       min_samples_split=250, n_estimators=500)\n",
            "Train accuracy score : 0.884975\n",
            "Test accuracy score : 0.863986398639864\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.89      0.84      0.86      5000\n",
            "           1       0.85      0.89      0.87      4999\n",
            "\n",
            "    accuracy                           0.86      9999\n",
            "   macro avg       0.87      0.86      0.86      9999\n",
            "weighted avg       0.87      0.86      0.86      9999\n",
            "\n",
            "\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "train_and_eval(best_rf, tfidf_X_train, class2_y_train, tfidf_X_test, class2_y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bu_5xpCzxT10"
      },
      "source": [
        "Best Random forest Classifier with TF-IDF gives us 86.3% accuracy on test data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "fpwU-46U-MWQ"
      },
      "source": [
        "# Multinomial Naive Bayes Hyperparameter Tuning"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZHfG38vZD9v3"
      },
      "source": [
        "Multinomial Naive Bayes with BOW"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {
        "id": "N0bDF3V6tKB9"
      },
      "outputs": [],
      "source": [
        "param_grid = {\n",
        "    'alpha': [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],\n",
        "    'force_alpha': [True, False],\n",
        "    'fit_prior': [True, False]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {
        "id": "O66B6KmOuVkc"
      },
      "outputs": [],
      "source": [
        "nb_model = MultinomialNB()\n",
        "clf = GridSearchCV(nb_model,\n",
        "                   param_grid=param_grid,\n",
        "                   cv=5,\n",
        "                   verbose=4,\n",
        "                   scoring='accuracy')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OkYsQ5E9upCF",
        "outputId": "b96225e2-5d10-4e16-f17f-feb021d52323"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 44 candidates, totalling 220 fits\n",
            "[CV 1/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.738 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 2/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.739 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.735 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 4/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.917 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.970 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 1/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.860 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.861 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 3/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.870 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 5/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.970 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.738 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 2/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.739 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.735 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 4/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.917 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.970 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 1/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.860 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.861 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 3/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.870 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n",
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 5/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.970 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.899 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.919 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.929 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.899 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.919 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.929 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.899 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.919 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.929 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.899 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.919 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.929 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.913 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.920 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.913 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.920 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.913 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.920 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.913 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.920 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.895 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.896 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.908 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.913 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.895 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.896 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.908 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.913 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.895 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.896 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.908 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.913 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.895 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.896 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.908 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.913 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.893 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.890 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.895 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.903 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.908 total time=   0.2s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.893 total time=   0.2s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.890 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.895 total time=   0.2s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.903 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.908 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.893 total time=   0.2s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.890 total time=   0.2s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.895 total time=   0.2s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.903 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.908 total time=   0.2s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.893 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.890 total time=   0.2s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.895 total time=   0.2s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.903 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.908 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.892 total time=   0.2s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.888 total time=   0.2s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.894 total time=   0.2s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.901 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.903 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.892 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.888 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.901 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.903 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.892 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.888 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.901 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.903 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.892 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.888 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.901 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.903 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.887 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.893 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.900 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.887 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.893 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.900 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.887 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.893 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.897 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.900 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.887 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.893 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.897 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.900 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.890 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.886 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.898 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.890 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.886 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.898 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.890 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.886 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.898 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.890 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.886 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.898 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.889 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.893 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.889 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.893 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.889 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.893 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.894 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.889 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.893 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.894 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.889 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.892 total time=   0.1s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.889 total time=   0.1s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.891 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.892 total time=   0.2s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.889 total time=   0.2s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.885 total time=   0.1s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.891 total time=   0.1s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.891 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.892 total time=   0.2s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.889 total time=   0.2s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.885 total time=   0.2s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.891 total time=   0.2s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.891 total time=   0.2s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.892 total time=   0.3s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.889 total time=   0.3s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.885 total time=   0.3s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.890 total time=   0.3s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.890 total time=   0.3s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.890 total time=   0.4s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.889 total time=   0.3s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.885 total time=   0.4s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.890 total time=   0.3s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.890 total time=   0.4s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.890 total time=   0.4s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.889 total time=   0.4s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.885 total time=   0.3s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.890 total time=   0.3s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.890 total time=   0.4s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.890 total time=   0.3s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.889 total time=   0.2s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.885 total time=   0.2s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.890 total time=   0.2s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.890 total time=   0.2s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.890 total time=   0.2s\n"
          ]
        }
      ],
      "source": [
        "best_clf = clf.fit(bow_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "WAnnhni5vUpF",
        "outputId": "d66c2795-7b53-4fae-bcc5-542562b995e2"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'alpha': 0.1, 'fit_prior': True, 'force_alpha': True}"
            ]
          },
          "execution_count": 18,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# best parameters\n",
        "best_clf.best_params_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "U6iYyU7ivbT-",
        "outputId": "c528d24f-fd84-4924-9a5e-0a41e7759c29"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.9082043574968326"
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# highest accuracy score\n",
        "best_clf.best_score_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {
        "id": "XFJcoFeGvrTu"
      },
      "outputs": [],
      "source": [
        "best_nb = MultinomialNB(**best_clf.best_params_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "VxTmOhVDv0SF",
        "outputId": "a95b5d9e-101f-4010-b81a-5e73430af6f8"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "MultinomialNB(alpha=0.1, force_alpha=True)\n",
            "Train accuracy score : 0.9512122541034959\n",
            "Test accuracy score : 0.8533535047907211\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.83      0.88      0.86      4940\n",
            "           1       0.87      0.83      0.85      4975\n",
            "\n",
            "    accuracy                           0.85      9915\n",
            "   macro avg       0.85      0.85      0.85      9915\n",
            "weighted avg       0.85      0.85      0.85      9915\n",
            "\n",
            "\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "train_and_eval(best_nb, bow_X_train, class2_y_train, bow_X_test, class2_y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0QEF0W-nD5wT"
      },
      "source": [
        "Best Naive Bayes model with BOW gives us 85.3% accuracy on test data"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DB2M9sccDvWv"
      },
      "source": [
        "Multinomial Naive Bayes with TF-IDF"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {
        "id": "PQ38qmDtQ4gT"
      },
      "outputs": [],
      "source": [
        "param_grid = {\n",
        "    'alpha': [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],\n",
        "    'force_alpha': [True, False],\n",
        "    'fit_prior': [True, False]\n",
        "}"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 23,
      "metadata": {
        "id": "609VKV4VQ_HM"
      },
      "outputs": [],
      "source": [
        "nb_model = MultinomialNB()\n",
        "clf = GridSearchCV(nb_model, param_grid=param_grid, cv=5, verbose=4, scoring='accuracy')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 24,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7QSqn2_XQ_HM",
        "outputId": "7e38137b-bfe6-4a78-e2e5-029b0a6f31e7"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Fitting 5 folds for each of 44 candidates, totalling 220 fits\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 1/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.699 total time=   1.0s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 2/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.696 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 3/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.688 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 4/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=0.939 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 5/5] END alpha=0.0, fit_prior=True, force_alpha=True;, score=1.000 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 1/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.910 total time=   2.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 2/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.915 total time=   3.2s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 3/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.914 total time=   1.7s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 4/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=0.981 total time=   1.7s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 5/5] END alpha=0.0, fit_prior=True, force_alpha=False;, score=1.000 total time=   1.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 1/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.699 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 2/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.696 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 3/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.688 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 4/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=0.939 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:907: RuntimeWarning: divide by zero encountered in log\n",
            "  self.feature_log_prob_ = np.log(smoothed_fc) - np.log(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 5/5] END alpha=0.0, fit_prior=False, force_alpha=True;, score=1.000 total time=   1.1s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 1/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.910 total time=   1.4s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 2/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.915 total time=   1.4s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 3/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.914 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 4/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=0.981 total time=   0.9s\n"
          ]
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/sklearn/naive_bayes.py:635: UserWarning: alpha too small will result in numeric errors, setting alpha = 1.0e-10. Use `force_alpha=True` to keep alpha unchanged.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[CV 5/5] END alpha=0.0, fit_prior=False, force_alpha=False;, score=1.000 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=0.988 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=True, force_alpha=True;, score=1.000 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=0.988 total time=   1.2s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=True, force_alpha=False;, score=1.000 total time=   1.3s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.945 total time=   1.2s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=0.988 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=False, force_alpha=True;, score=1.000 total time=   0.8s\n",
            "[CV 1/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=0.988 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.1, fit_prior=False, force_alpha=False;, score=1.000 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.946 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.946 total time=   0.8s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.946 total time=   1.3s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=0.988 total time=   1.3s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=True, force_alpha=True;, score=1.000 total time=   1.1s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.946 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.946 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.946 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=0.988 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=True, force_alpha=False;, score=1.000 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.946 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.946 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.946 total time=   0.8s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=0.988 total time=   0.8s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=False, force_alpha=True;, score=1.000 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.946 total time=   0.8s\n",
            "[CV 2/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.946 total time=   1.4s\n",
            "[CV 3/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.946 total time=   1.4s\n",
            "[CV 4/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=0.988 total time=   1.5s\n",
            "[CV 5/5] END alpha=0.2, fit_prior=False, force_alpha=False;, score=1.000 total time=   2.1s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.947 total time=   1.7s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.946 total time=   2.1s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.945 total time=   2.2s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.988 total time=   1.6s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=True, force_alpha=True;, score=0.999 total time=   1.6s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.947 total time=   3.7s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.946 total time=   2.4s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.945 total time=   1.4s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.988 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=True, force_alpha=False;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.947 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.946 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.988 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=False, force_alpha=True;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.947 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.946 total time=   1.1s\n",
            "[CV 3/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.945 total time=   1.4s\n",
            "[CV 4/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.988 total time=   1.4s\n",
            "[CV 5/5] END alpha=0.3, fit_prior=False, force_alpha=False;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.946 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.987 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=True, force_alpha=True;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.946 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.987 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=True, force_alpha=False;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.946 total time=   1.9s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.945 total time=   1.5s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.987 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=False, force_alpha=True;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.946 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.987 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.4, fit_prior=False, force_alpha=False;, score=0.999 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.987 total time=   1.3s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=True, force_alpha=True;, score=0.997 total time=   1.4s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.945 total time=   1.2s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.945 total time=   1.3s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.987 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=True, force_alpha=False;, score=0.997 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.987 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=False, force_alpha=True;, score=0.997 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.945 total time=   1.3s\n",
            "[CV 3/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.945 total time=   1.4s\n",
            "[CV 4/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.987 total time=   1.2s\n",
            "[CV 5/5] END alpha=0.5, fit_prior=False, force_alpha=False;, score=0.997 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.986 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=True, force_alpha=True;, score=0.996 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.986 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=True, force_alpha=False;, score=0.996 total time=   1.1s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.945 total time=   1.4s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.944 total time=   1.3s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.986 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=False, force_alpha=True;, score=0.996 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.986 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.6, fit_prior=False, force_alpha=False;, score=0.996 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.944 total time=   1.0s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.985 total time=   1.4s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=True, force_alpha=True;, score=0.995 total time=   1.4s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.985 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=True, force_alpha=False;, score=0.995 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.985 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=False, force_alpha=True;, score=0.995 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.945 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.944 total time=   1.3s\n",
            "[CV 3/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.944 total time=   1.4s\n",
            "[CV 4/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.985 total time=   1.2s\n",
            "[CV 5/5] END alpha=0.7, fit_prior=False, force_alpha=False;, score=0.995 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.982 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=True, force_alpha=True;, score=0.994 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.982 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=True, force_alpha=False;, score=0.994 total time=   1.3s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.944 total time=   1.5s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.943 total time=   1.6s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.944 total time=   1.6s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.982 total time=   1.0s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=False, force_alpha=True;, score=0.994 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.944 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.944 total time=   1.0s\n",
            "[CV 4/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.982 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.8, fit_prior=False, force_alpha=False;, score=0.994 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.981 total time=   1.3s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=True, force_alpha=True;, score=0.993 total time=   1.5s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.944 total time=   1.2s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.981 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=True, force_alpha=False;, score=0.993 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.944 total time=   0.9s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.981 total time=   0.9s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=False, force_alpha=True;, score=0.993 total time=   0.9s\n",
            "[CV 1/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.944 total time=   1.0s\n",
            "[CV 2/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.943 total time=   1.3s\n",
            "[CV 3/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.943 total time=   1.4s\n",
            "[CV 4/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.981 total time=   1.2s\n",
            "[CV 5/5] END alpha=0.9, fit_prior=False, force_alpha=False;, score=0.993 total time=   0.9s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.942 total time=   0.9s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.978 total time=   0.9s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=True, force_alpha=True;, score=0.992 total time=   0.9s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.942 total time=   0.9s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.978 total time=   0.9s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=True, force_alpha=False;, score=0.992 total time=   1.4s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.943 total time=   1.5s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.943 total time=   1.5s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.942 total time=   0.9s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.978 total time=   0.9s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=False, force_alpha=True;, score=0.992 total time=   0.9s\n",
            "[CV 1/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 2/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.943 total time=   0.9s\n",
            "[CV 3/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.942 total time=   0.9s\n",
            "[CV 4/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.978 total time=   0.9s\n",
            "[CV 5/5] END alpha=1.0, fit_prior=False, force_alpha=False;, score=0.992 total time=   0.9s\n"
          ]
        }
      ],
      "source": [
        "best_clf = clf.fit(tfidf_X_train, class2_y_train)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 25,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "OR-6UptLRl9j",
        "outputId": "ecf92e4d-e8ee-4646-d3eb-f273c821ce82"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "{'alpha': 0.2, 'fit_prior': True, 'force_alpha': True}"
            ]
          },
          "execution_count": 25,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# best parameters\n",
        "best_clf.best_params_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 26,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "PApJtr4tRl9k",
        "outputId": "0f119eeb-91e1-4917-8d7d-89ca56ed53f1"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "0.9650428759010555"
            ]
          },
          "execution_count": 26,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# highest accuracy score\n",
        "best_clf.best_score_"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 27,
      "metadata": {
        "id": "9LN8FvtxRl9k"
      },
      "outputs": [],
      "source": [
        "best_nb = MultinomialNB(**best_clf.best_params_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 28,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "9Vojc59-Rvcr",
        "outputId": "1fec0b75-8fb7-4c82-9d4f-47c76f76f022"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "\n",
            "MultinomialNB(alpha=0.2, force_alpha=True)\n",
            "Train accuracy score : 0.9951290565092094\n",
            "Test accuracy score : 0.891981845688351\n",
            "              precision    recall  f1-score   support\n",
            "\n",
            "           0       0.89      0.90      0.89      4940\n",
            "           1       0.90      0.89      0.89      4975\n",
            "\n",
            "    accuracy                           0.89      9915\n",
            "   macro avg       0.89      0.89      0.89      9915\n",
            "weighted avg       0.89      0.89      0.89      9915\n",
            "\n",
            "\n",
            " ----------------------------------------\n"
          ]
        }
      ],
      "source": [
        "train_and_eval(best_nb, tfidf_X_train, class2_y_train, tfidf_X_test, class2_y_test)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vw16QtYmRyIz"
      },
      "source": [
        "Best Multinomial Naive Bayes gives us an accuracy of 89.2% on test data"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}