teller/demo/thierrymoudiki_081119_boston_housing.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "teller_boston2.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VKJGj4IzMMBB",
        "colab_type": "text"
      },
      "source": [
        "# Illustrating the `teller` (v0.2.0)\n",
        "\n",
        "This notebook illustrates the use of the [`teller`](https://github.com/thierrymoudiki/teller), a model-agnostic tool for Machine Learning explainability. Version `0.2.0` improves the interface, and introduces tests on the significance of marginal effects. \n",
        "\n",
        "Currently, the `teller` can be installed from Github as: "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "O0x4rfI9MByJ",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "pip install git+https://github.com/thierrymoudiki/teller.git"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dUZwnL7NM8ri",
        "colab_type": "text"
      },
      "source": [
        "Data for the demo is Boston Housing dataset. The response is MEDV, Median value of owner-occupied homes in $1000’s (the __reponse__):\n",
        "\n",
        "\n",
        "\n",
        "- CRIM per capita crime rate by town\n",
        "- ZN proportion of residential land zoned for lots over 25,000 sq.ft.\n",
        "- INDUS proportion of non-retail business acres per town\n",
        "- CHAS Charles River dummy variable (= 1 if tract bounds river; 0 otherwise)\n",
        "- NOX nitric oxides concentration (parts per 10 million)\n",
        "- RM average number of rooms per dwelling\n",
        "- AGE proportion of owner-occupied units built prior to 1940\n",
        "- DIS weighted distances to five Boston employment centres\n",
        "- RAD index of accessibility to radial highways\n",
        "- TAX full-value property-tax rate per $10,000\n",
        "\n",
        "- PTRATIO pupil-teacher ratio by town\n",
        "- LSTAT % lower status of the population\n",
        "- MEDV Median value of owner-occupied homes in $1000’s (the __reponse__)\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "C4X01QkpNDWc",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import teller as tr\n",
        "import pandas as pd\n",
        "import numpy as np      \n",
        "\n",
        "from sklearn import datasets, linear_model\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "from sklearn import datasets\n",
        "from sklearn.model_selection import train_test_split\n",
        "\n",
        "\n",
        "# import data\n",
        "boston = datasets.load_boston()\n",
        "X = np.delete(boston.data, 11, 1)\n",
        "y = boston.target\n",
        "col_names = np.append(np.delete(boston.feature_names, 11), 'MEDV')\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ChTdmrmHNIUS",
        "colab_type": "text"
      },
      "source": [
        "Split data into a training and a testing set:"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "fGFLkh4FNLnD",
        "colab_type": "code",
        "outputId": "914da018-f5d9-4d90-c455-07cfce9a9b45",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 52
        }
      },
      "source": [
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, \n",
        "                                                    random_state=123)\n",
        "print(X_train.shape)\n",
        "print(X_test.shape)\n"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "(404, 12)\n",
            "(102, 12)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AXdQVSDMRAp8",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# fit a random forest model \n",
        "regr2 = RandomForestRegressor(n_estimators=1000, random_state=123)\n",
        "regr2.fit(X_train, y_train)\n",
        "\n",
        "\n",
        "# creating the explainer\n",
        "expr = tr.Explainer(obj=regr2)\n",
        "\n",
        "\n",
        "# fitting the explainer (for heterogeneity of effects only)\n",
        "expr.fit(X_test, y_test, X_names=col_names[:-1], method=\"avg\")\n",
        "\n",
        "\n",
        "# heterogeneity of effects\n",
        "print(expr.summary())"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PrNLlUtiRzc1",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 781
        },
        "outputId": "fc6b4899-8f4f-4c4e-e6c7-928645b8c428"
      },
      "source": [
        "# confidence int. and tests on effects (Jackkife)\n",
        "expr.fit(X_test, y_test, X_names=col_names[:-1], method=\"ci\")\n",
        "\n",
        "print(expr.summary())\n"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "\n",
            "\n",
            "Residuals: \n",
            "     Min      1Q  Median      3Q     Max\n",
            "-20.7672 -1.1802 -0.2857  1.0939  8.8958\n",
            "\n",
            "\n",
            "Tests on marginal effects (Jackknife): \n",
            "          Estimate   Std. Error   95% lbound   95% ubound     Pr(>|t|)     \n",
            "LSTAT     -11.6629      4.21614     -20.0266     -3.29925   0.00674382   **\n",
            "PTRATIO   -5.83981      1.01549     -7.85428     -3.82534  9.51649e-08  ***\n",
            "INDUS     -3.45108     0.865402     -5.16781     -1.73436  0.000126284  ***\n",
            "TAX     -0.0527896   9.7629e-16   -0.0527896   -0.0527896            0  ***\n",
            "CHAS             0  2.22045e-16 -4.40477e-16  4.40477e-16            1     \n",
            "AGE       0.983501    0.0202223     0.943385      1.02362  6.71929e-72  ***\n",
            "ZN         1.05412   0.00505945      1.04409      1.06416  7.3924e-135  ***\n",
            "NOX        1.42195      11.3752     -21.1434      23.9873     0.900769     \n",
            "DIS        2.03424  1.78522e-14      2.03424      2.03424            0  ***\n",
            "RAD          18.61     0.929718      16.7657      20.4543   6.2148e-37  ***\n",
            "RM         28.8366      1.15918      26.5371      31.1361  7.21593e-45  ***\n",
            "CRIM       72.9152  9.99721e-13      72.9152      72.9152            0  ***\n",
            "\n",
            "\n",
            "Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1\n",
            "\n",
            "\n",
            "Multiple R-squared:  0.774,\tAdjusted R-squared:  0.744\n",
            "\n",
            "\n",
            "Heterogeneity of marginal effects: \n",
            "              mean         std          min          max\n",
            "LSTAT   -11.541770  104.111356  -680.369720   335.990384\n",
            "PTRATIO  -5.795078   26.975073  -155.914653    56.827716\n",
            "INDUS    -3.425733   26.951393  -258.382895     0.000000\n",
            "TAX      -0.052272    0.824834    -6.479723     4.839278\n",
            "CHAS      0.000000    0.000000     0.000000     0.000000\n",
            "AGE       0.970438    5.204533    -7.242999    39.647849\n",
            "ZN        1.043840   11.672871   -28.280289    83.808739\n",
            "NOX       1.286747  325.585815 -1258.347012  1937.006074\n",
            "DIS       2.014293   20.343364     0.000000   205.457901\n",
            "RAD      18.420244  192.075879  -247.710558  1791.773035\n",
            "RM       28.570050  146.327113  -123.772764  1126.812921\n",
            "CRIM     72.200382  585.402432     0.000000  5685.533164\n",
            "None\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}