SEN_Transport_flood-disruption_2021-11-26.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Network analysis in Senegal\n",
    "\n",
    "### Objectives\n",
    "    1)\tUse measures of road-based accessibility to identify road segments that, if rehabilitated, would improve agricultural market activities in Senegal, including during flood conditions.\n",
    "    2)\tGain a better understanding of the accessibility, connectivity, and criticality of roads in Senegal in relationship to agricultural origins, processing & transfer sites, and markets.\n",
    "\n",
    "To this end, the team will develop an accessibility model which measures the travel time from sites of agricultural production to their nearest populated areas, processing centers, and markets. \n",
    "\n",
    "### Datasets for analysis\n",
    "#### ORIGIN\n",
    "    1) agriculture: MapSPAM 2017. Measuring value in international dollars.\n",
    "    2) agriculture: UMD Land Cover 2019 30m. Assign MapSPAM value onto land cover cropland class for more precise origin information.\n",
    "    3) population: WorldPop 2020, UN-adjusted.\n",
    "    4) settlement extent: GRID3 2020.\n",
    "#### DESTINATION\n",
    "    4) markets: derived from WorldPop 2020 and GRID3 2020 urban clusters.\n",
    "    5) agricultural processing hubs: to be acquired.\n",
    "#### TRAVEL ROUTE\n",
    "    6) roads: OpenStreetMap, July 2021.\n",
    "    7) elevation: \n",
    "#### OBSTACLE\n",
    "    8) flood: FATHOM. 1-in-10, 20, and 50 year flood return periods. \n",
    "#### INTERVENTION\n",
    "    9) upcoming road projects: AGEROUTE interventions separate from the World Bank-financed project\n",
    "    10) targeted road projects: critical road segments identified by this accessibility model's baseline outputs\n",
    "\n",
    "\n",
    "### Model design\n",
    "#### Basic formula: \n",
    "    (a) Off-road driving time from origin to closest road node\n",
    "    +\n",
    "    (b) Driving time from road node in (a) to a destination (closeness measured by road segments speeds)\n",
    "\n",
    "#### Model origin & destination (OD) sets:\n",
    "    A)\tTravel time from an area that has agricultural value/potential to the nearest processing hub (if provided).\n",
    "    B)\tTravel time from an area that has agricultural value/potential to the nearest larger settlement, (“larger” settlement identified using a case-appropriate population metric to be determined).\n",
    "    C)\tTravel time from an area that has agricultural value/potential to the nearest market.\n",
    "    D)\tTravel time from all settlements to the nearest market.\n",
    "    E)\tTravel time from larger settlements to the nearest market.\n",
    "\n",
    "#### Before/after scenarios for each OD set:\n",
    "    1)\tPre-project, baseline weather: No inclement weather. Road network status as of November 2021.\n",
    "    2)\tPre-project, flood: 1-in-10, 1-in-20 and 1-in-50 year flood return period. Road network status as of November 2021.\n",
    "    3)\tPost-project, baseline weather: No inclement weather. Road network status if X number of critical road segments to high-value areas are protected (i.e., their travel times reduced).\n",
    "    4)\tPost-project, flood: 1-in-10 year flood return period. Road network status if X number of critical road segments to high-value areas are protected (i.e., their travel times reduced).\n",
    "\n",
    "#### Notes:\n",
    "    --Destinations are expected to be proximal to the road network, so no measure is taken between road and destination.\n",
    "    --All travel times will be assigned to each model variation’s point of origin; the aggregation up to admin areas is possible if desired.\n",
    "    --Obstacles & interventions modify the road segment speeds. Basic formula is then applied to the modified road network.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prep workspace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\wb527163\\\\GEO-Cdrive-Grace'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os, sys\n",
    "GISFolder = os.getcwd()\n",
    "GISFolder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: needed to reinstall rtree due to geopandas import error. Did so in the console. \n",
    "# conda install -c conda-forge rtree=0.9.3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load and filter osm network (step 1)\n",
    "import geopandas as gpd\n",
    "from geopandas import GeoDataFrame\n",
    "import pandas as pd\n",
    "import time\n",
    "sys.path.append(r\"C:\\Users\\wb527163\\.conda\\envs\\geo\\GOSTnets-master\")\n",
    "import GOSTnets as gn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import networkx as nx\n",
    "import osmnx as ox\n",
    "import numpy as np\n",
    "import rasterio as rt\n",
    "import shapely\n",
    "from shapely.geometry import Point, box\n",
    "from shapely.ops import unary_union\n",
    "from shapely.wkt import loads\n",
    "from shapely import wkt\n",
    "from shapely.geometry import LineString, MultiLineString, Point\n",
    "import peartree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### Might not use these\n",
    "import fiona\n",
    "from osgeo import gdal\n",
    "import importlib\n",
    "import matplotlib.pyplot as plt\n",
    "import subprocess, glob"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\wb527163\\\\GEO-Cdrive-Grace\\\\SEN-Cdrive'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pth = os.path.join(GISFolder, \"SEN-Cdrive\") # Personal folder system for running model.\n",
    "pth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'C:\\\\Users\\\\wb527163\\\\GEO-Cdrive-Grace\\\\SEN-Cdrive\\\\outputs'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "out_pth = os.path.join(GISFolder, \"SEN-Cdrive\\outputs\") # For storing intermediate outputs from the model.\n",
    "out_pth"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'R:\\\\SEN\\\\GEO'"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "team_pth = 'R:\\\\SEN\\\\GEO' # This is where the unmodified input data is stored. Finalized outputs also housed here.\n",
    "team_pth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Prepare and clean the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Return periods from FATHOM: 1-in-10, 20, and 50 year floods.\n",
    "\n",
    "We are starting with just the 1-in-10 year return period. Joining 1-in-20 and 1-in-50 to the road dataframe was causing file size errors. 20 and 50 will be run in a replicated script."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'geopandas.geodataframe.GeoDataFrame'>\n",
      "RangeIndex: 2459356 entries, 0 to 2459355\n",
      "Data columns (total 4 columns):\n",
      " #   Column        Dtype   \n",
      "---  ------        -----   \n",
      " 0   PFU_1in50     float64 \n",
      " 1   Shape_Length  float64 \n",
      " 2   Shape_Area    float64 \n",
      " 3   geometry      geometry\n",
      "dtypes: float64(3), geometry(1)\n",
      "memory usage: 75.1 MB\n"
     ]
    }
   ],
   "source": [
    "flood50 = gpd.read_file(\"C:/Users/wb527163/GEO-Cdrive-Grace/SEN-Cdrive/scratch.gdb\", layer=\"PFU_1in50\")\n",
    "flood50.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>stnode</th>\n",
       "      <th>endnode</th>\n",
       "      <th>osmid</th>\n",
       "      <th>mode</th>\n",
       "      <th>highway</th>\n",
       "      <th>time</th>\n",
       "      <th>ref</th>\n",
       "      <th>length</th>\n",
       "      <th>access</th>\n",
       "      <th>oneway</th>\n",
       "      <th>bridge</th>\n",
       "      <th>lanes</th>\n",
       "      <th>junction</th>\n",
       "      <th>tunnel</th>\n",
       "      <th>width</th>\n",
       "      <th>maxspeed</th>\n",
       "      <th>name</th>\n",
       "      <th>area</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>358284990</td>\n",
       "      <td>5217543379</td>\n",
       "      <td>59618174</td>\n",
       "      <td>drive</td>\n",
       "      <td>unclassified</td>\n",
       "      <td>2.385144</td>\n",
       "      <td>D 523</td>\n",
       "      <td>33.127</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D 523</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32368 12.3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>358284990</td>\n",
       "      <td>1888282175</td>\n",
       "      <td>178482063</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>0.769920</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.832</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32351 12.3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>358284990</td>\n",
       "      <td>5329792467</td>\n",
       "      <td>178482063</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>2.926860</td>\n",
       "      <td>NaN</td>\n",
       "      <td>48.781</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32317 12.3...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>358284993</td>\n",
       "      <td>1888282575</td>\n",
       "      <td>178470940</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>7.594620</td>\n",
       "      <td>NaN</td>\n",
       "      <td>126.577</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28205 12.4...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>358284993</td>\n",
       "      <td>1888198886</td>\n",
       "      <td>178470940</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>7.234680</td>\n",
       "      <td>NaN</td>\n",
       "      <td>120.578</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28061 12.4...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012560</th>\n",
       "      <td>9246539941</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>366052716</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.742</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-17.48521 14.72445, -17.48451 14.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012561</th>\n",
       "      <td>9246539942</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>366052716</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.742</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48521 14.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012562</th>\n",
       "      <td>9246539942</td>\n",
       "      <td>3700438702</td>\n",
       "      <td>366052716</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>0.076140</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.846</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48451 14.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012563</th>\n",
       "      <td>9276108905</td>\n",
       "      <td>6048975958</td>\n",
       "      <td>177950649</td>\n",
       "      <td>drive</td>\n",
       "      <td>secondary</td>\n",
       "      <td>8.840674</td>\n",
       "      <td>NaN</td>\n",
       "      <td>171.902</td>\n",
       "      <td>NaN</td>\n",
       "      <td>True</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Route de l'Aeroport</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50357 14.7...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012564</th>\n",
       "      <td>9276108905</td>\n",
       "      <td>7493947541</td>\n",
       "      <td>801279967</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>1.501740</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.686</td>\n",
       "      <td>NaN</td>\n",
       "      <td>False</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50490 14.7...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4012565 rows × 19 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             stnode     endnode      osmid   mode       highway      time  \\\n",
       "0         358284990  5217543379   59618174  drive  unclassified  2.385144   \n",
       "1         358284990  1888282175  178482063  drive      tertiary  0.769920   \n",
       "2         358284990  5329792467  178482063  drive      tertiary  2.926860   \n",
       "3         358284993  1888282575  178470940  drive      tertiary  7.594620   \n",
       "4         358284993  1888198886  178470940  drive      tertiary  7.234680   \n",
       "...             ...         ...        ...    ...           ...       ...   \n",
       "4012560  9246539941  9246539942  366052716  drive   residential  6.906780   \n",
       "4012561  9246539942  9246539941  366052716  drive   residential  6.906780   \n",
       "4012562  9246539942  3700438702  366052716  drive   residential  0.076140   \n",
       "4012563  9276108905  6048975958  177950649  drive     secondary  8.840674   \n",
       "4012564  9276108905  7493947541  801279967  drive   residential  1.501740   \n",
       "\n",
       "           ref   length access  oneway bridge lanes junction tunnel width  \\\n",
       "0        D 523   33.127    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "1          NaN   12.832    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "2          NaN   48.781    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "3          NaN  126.577    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "4          NaN  120.578    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "...        ...      ...    ...     ...    ...   ...      ...    ...   ...   \n",
       "4012560    NaN   76.742    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "4012561    NaN   76.742    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "4012562    NaN    0.846    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "4012563    NaN  171.902    NaN    True    NaN     2      NaN    NaN   NaN   \n",
       "4012564    NaN   16.686    NaN   False    NaN   NaN      NaN    NaN   NaN   \n",
       "\n",
       "        maxspeed                 name area  \\\n",
       "0            NaN                D 523  NaN   \n",
       "1            NaN                  NaN  NaN   \n",
       "2            NaN                  NaN  NaN   \n",
       "3            NaN                  NaN  NaN   \n",
       "4            NaN                  NaN  NaN   \n",
       "...          ...                  ...  ...   \n",
       "4012560      NaN                  NaN  NaN   \n",
       "4012561      NaN                  NaN  NaN   \n",
       "4012562      NaN                  NaN  NaN   \n",
       "4012563      NaN  Route de l'Aeroport  NaN   \n",
       "4012564      NaN                  NaN  NaN   \n",
       "\n",
       "                                                  geometry  \n",
       "0        LINESTRING (-12.32347 12.38119, -12.32368 12.3...  \n",
       "1        LINESTRING (-12.32347 12.38119, -12.32351 12.3...  \n",
       "2        LINESTRING (-12.32347 12.38119, -12.32317 12.3...  \n",
       "3        LINESTRING (-12.28135 12.41380, -12.28205 12.4...  \n",
       "4        LINESTRING (-12.28135 12.41380, -12.28061 12.4...  \n",
       "...                                                    ...  \n",
       "4012560  LINESTRING (-17.48521 14.72445, -17.48451 14.7...  \n",
       "4012561  LINESTRING (-17.48451 14.72456, -17.48521 14.7...  \n",
       "4012562  LINESTRING (-17.48451 14.72456, -17.48451 14.7...  \n",
       "4012563  LINESTRING (-17.50499 14.74971, -17.50357 14.7...  \n",
       "4012564  LINESTRING (-17.50499 14.74971, -17.50490 14.7...  \n",
       "\n",
       "[4012565 rows x 19 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "gTime = nx.read_gpickle(\"SEN-Cdrive/gTime.pickle\")\n",
    "gTime_edge = gn.edge_gdf_from_graph(gTime)\n",
    "gTime_edge"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1829568 entries, 0 to 1829567\n",
      "Data columns (total 7 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   node_ID     int64  \n",
      " 2   y           float64\n",
      " 3   highway     object \n",
      " 4   x           float64\n",
      " 5   ref         object \n",
      " 6   geometry    object \n",
      "dtypes: float64(2), int64(2), object(3)\n",
      "memory usage: 97.7+ MB\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wb527163\\.conda\\envs\\geo\\lib\\site-packages\\IPython\\core\\interactiveshell.py:3444: DtypeWarning: Columns (5) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  exec(code_obj, self.user_global_ns, self.user_ns)\n"
     ]
    }
   ],
   "source": [
    "gTime_node = os.path.join(pth, \"gTime_nodes.csv\")\n",
    "gTime_node = pd.read_csv(gTime_node)\n",
    "gTime_node.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Update driving times based on flood intersection."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Join road network and flood raster into single table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'geopandas.geodataframe.GeoDataFrame'>\n",
      "RangeIndex: 4012565 entries, 0 to 4012564\n",
      "Data columns (total 20 columns):\n",
      " #   Column    Dtype   \n",
      "---  ------    -----   \n",
      " 0   ID_graph  int64   \n",
      " 1   stnode    int64   \n",
      " 2   endnode   int64   \n",
      " 3   osmid     int64   \n",
      " 4   mode      object  \n",
      " 5   highway   object  \n",
      " 6   time      float64 \n",
      " 7   ref       object  \n",
      " 8   length    float64 \n",
      " 9   access    object  \n",
      " 10  oneway    bool    \n",
      " 11  bridge    object  \n",
      " 12  lanes     object  \n",
      " 13  junction  object  \n",
      " 14  tunnel    object  \n",
      " 15  width     object  \n",
      " 16  maxspeed  object  \n",
      " 17  name      object  \n",
      " 18  area      object  \n",
      " 19  geometry  geometry\n",
      "dtypes: bool(1), float64(2), geometry(1), int64(4), object(12)\n",
      "memory usage: 585.5+ MB\n"
     ]
    }
   ],
   "source": [
    "gTime_edge.reset_index(inplace=True) # To create unique ID and to avoid:  ValueError: cannot reindex from a duplicate axis.\n",
    "gTime_edge.rename(columns={'index': 'ID_graph'}, inplace=True)\n",
    "gTime_edge.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Spatial join should be on projected GDFs.\n",
    "gTime_edge = gTime_edge.to_crs(\"EPSG:31028\")\n",
    "flood50 = flood50.to_crs(\"EPSG:31028\")\n",
    "gTime_edge.crs == flood50.crs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'geopandas.geodataframe.GeoDataFrame'>\n",
      "Int64Index: 4195759 entries, 0 to 4012564\n",
      "Data columns (total 24 columns):\n",
      " #   Column        Dtype   \n",
      "---  ------        -----   \n",
      " 0   ID_graph      int64   \n",
      " 1   stnode        int64   \n",
      " 2   endnode       int64   \n",
      " 3   osmid         int64   \n",
      " 4   mode          object  \n",
      " 5   highway       object  \n",
      " 6   time          float64 \n",
      " 7   ref           object  \n",
      " 8   length        float64 \n",
      " 9   access        object  \n",
      " 10  oneway        bool    \n",
      " 11  bridge        object  \n",
      " 12  lanes         object  \n",
      " 13  junction      object  \n",
      " 14  tunnel        object  \n",
      " 15  width         object  \n",
      " 16  maxspeed      object  \n",
      " 17  name          object  \n",
      " 18  area          object  \n",
      " 19  geometry      geometry\n",
      " 20  index_right   float64 \n",
      " 21  PFU_1in50     float64 \n",
      " 22  Shape_Length  float64 \n",
      " 23  Shape_Area    float64 \n",
      "dtypes: bool(1), float64(6), geometry(1), int64(4), object(12)\n",
      "memory usage: 772.3+ MB\n"
     ]
    }
   ],
   "source": [
    "join50 = gpd.sjoin_nearest(gTime_edge, flood50, how=\"left\", max_distance=3) \n",
    "join50.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_graph</th>\n",
       "      <th>stnode</th>\n",
       "      <th>endnode</th>\n",
       "      <th>osmid</th>\n",
       "      <th>mode</th>\n",
       "      <th>highway</th>\n",
       "      <th>time</th>\n",
       "      <th>ref</th>\n",
       "      <th>length</th>\n",
       "      <th>access</th>\n",
       "      <th>...</th>\n",
       "      <th>tunnel</th>\n",
       "      <th>width</th>\n",
       "      <th>maxspeed</th>\n",
       "      <th>name</th>\n",
       "      <th>area</th>\n",
       "      <th>geometry</th>\n",
       "      <th>index_right</th>\n",
       "      <th>PFU_1in50</th>\n",
       "      <th>Shape_Length</th>\n",
       "      <th>Shape_Area</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5217543379</td>\n",
       "      <td>59618174</td>\n",
       "      <td>drive</td>\n",
       "      <td>unclassified</td>\n",
       "      <td>2.385144</td>\n",
       "      <td>D 523</td>\n",
       "      <td>33.127</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>D 523</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (790871.607 1370083.257, 790848.290...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>358284990</td>\n",
       "      <td>1888282175</td>\n",
       "      <td>178482063</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>0.769920</td>\n",
       "      <td>NaN</td>\n",
       "      <td>12.832</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (790871.607 1370083.257, 790867.287...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5329792467</td>\n",
       "      <td>178482063</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>2.926860</td>\n",
       "      <td>NaN</td>\n",
       "      <td>48.781</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (790871.607 1370083.257, 790904.086...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888282575</td>\n",
       "      <td>178470940</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>7.594620</td>\n",
       "      <td>NaN</td>\n",
       "      <td>126.577</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (795418.079 1373739.218, 795343.307...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888198886</td>\n",
       "      <td>178470940</td>\n",
       "      <td>drive</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>7.234680</td>\n",
       "      <td>NaN</td>\n",
       "      <td>120.578</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (795418.079 1373739.218, 795497.547...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012560</th>\n",
       "      <td>4012560</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>366052716</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.742</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (232231.532 1629246.092, 232307.524...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012561</th>\n",
       "      <td>4012561</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>366052716</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>NaN</td>\n",
       "      <td>76.742</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (232307.524 1629257.675, 232231.532...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012562</th>\n",
       "      <td>4012562</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>3700438702</td>\n",
       "      <td>366052716</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>0.076140</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.846</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (232307.524 1629257.675, 232307.393...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012563</th>\n",
       "      <td>4012563</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>6048975958</td>\n",
       "      <td>177950649</td>\n",
       "      <td>drive</td>\n",
       "      <td>secondary</td>\n",
       "      <td>8.840674</td>\n",
       "      <td>NaN</td>\n",
       "      <td>171.902</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>Route de l'Aeroport</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (230131.706 1632066.089, 230285.874...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012564</th>\n",
       "      <td>4012564</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>7493947541</td>\n",
       "      <td>801279967</td>\n",
       "      <td>drive</td>\n",
       "      <td>residential</td>\n",
       "      <td>1.501740</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16.686</td>\n",
       "      <td>NaN</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>LINESTRING (230131.706 1632066.089, 230141.099...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4195759 rows × 24 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID_graph      stnode     endnode      osmid   mode       highway  \\\n",
       "0               0   358284990  5217543379   59618174  drive  unclassified   \n",
       "1               1   358284990  1888282175  178482063  drive      tertiary   \n",
       "2               2   358284990  5329792467  178482063  drive      tertiary   \n",
       "3               3   358284993  1888282575  178470940  drive      tertiary   \n",
       "4               4   358284993  1888198886  178470940  drive      tertiary   \n",
       "...           ...         ...         ...        ...    ...           ...   \n",
       "4012560   4012560  9246539941  9246539942  366052716  drive   residential   \n",
       "4012561   4012561  9246539942  9246539941  366052716  drive   residential   \n",
       "4012562   4012562  9246539942  3700438702  366052716  drive   residential   \n",
       "4012563   4012563  9276108905  6048975958  177950649  drive     secondary   \n",
       "4012564   4012564  9276108905  7493947541  801279967  drive   residential   \n",
       "\n",
       "             time    ref   length access  ...  tunnel width maxspeed  \\\n",
       "0        2.385144  D 523   33.127    NaN  ...     NaN   NaN      NaN   \n",
       "1        0.769920    NaN   12.832    NaN  ...     NaN   NaN      NaN   \n",
       "2        2.926860    NaN   48.781    NaN  ...     NaN   NaN      NaN   \n",
       "3        7.594620    NaN  126.577    NaN  ...     NaN   NaN      NaN   \n",
       "4        7.234680    NaN  120.578    NaN  ...     NaN   NaN      NaN   \n",
       "...           ...    ...      ...    ...  ...     ...   ...      ...   \n",
       "4012560  6.906780    NaN   76.742    NaN  ...     NaN   NaN      NaN   \n",
       "4012561  6.906780    NaN   76.742    NaN  ...     NaN   NaN      NaN   \n",
       "4012562  0.076140    NaN    0.846    NaN  ...     NaN   NaN      NaN   \n",
       "4012563  8.840674    NaN  171.902    NaN  ...     NaN   NaN      NaN   \n",
       "4012564  1.501740    NaN   16.686    NaN  ...     NaN   NaN      NaN   \n",
       "\n",
       "                        name area  \\\n",
       "0                      D 523  NaN   \n",
       "1                        NaN  NaN   \n",
       "2                        NaN  NaN   \n",
       "3                        NaN  NaN   \n",
       "4                        NaN  NaN   \n",
       "...                      ...  ...   \n",
       "4012560                  NaN  NaN   \n",
       "4012561                  NaN  NaN   \n",
       "4012562                  NaN  NaN   \n",
       "4012563  Route de l'Aeroport  NaN   \n",
       "4012564                  NaN  NaN   \n",
       "\n",
       "                                                  geometry index_right  \\\n",
       "0        LINESTRING (790871.607 1370083.257, 790848.290...         NaN   \n",
       "1        LINESTRING (790871.607 1370083.257, 790867.287...         NaN   \n",
       "2        LINESTRING (790871.607 1370083.257, 790904.086...         NaN   \n",
       "3        LINESTRING (795418.079 1373739.218, 795343.307...         NaN   \n",
       "4        LINESTRING (795418.079 1373739.218, 795497.547...         NaN   \n",
       "...                                                    ...         ...   \n",
       "4012560  LINESTRING (232231.532 1629246.092, 232307.524...         NaN   \n",
       "4012561  LINESTRING (232307.524 1629257.675, 232231.532...         NaN   \n",
       "4012562  LINESTRING (232307.524 1629257.675, 232307.393...         NaN   \n",
       "4012563  LINESTRING (230131.706 1632066.089, 230285.874...         NaN   \n",
       "4012564  LINESTRING (230131.706 1632066.089, 230141.099...         NaN   \n",
       "\n",
       "        PFU_1in50 Shape_Length Shape_Area  \n",
       "0             NaN          NaN        NaN  \n",
       "1             NaN          NaN        NaN  \n",
       "2             NaN          NaN        NaN  \n",
       "3             NaN          NaN        NaN  \n",
       "4             NaN          NaN        NaN  \n",
       "...           ...          ...        ...  \n",
       "4012560       NaN          NaN        NaN  \n",
       "4012561       NaN          NaN        NaN  \n",
       "4012562       NaN          NaN        NaN  \n",
       "4012563       NaN          NaN        NaN  \n",
       "4012564       NaN          NaN        NaN  \n",
       "\n",
       "[4195759 rows x 24 columns]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "join50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "No flood crossing at node: 3748839 locations\n",
      "Flood crossing at node: 446920 locations\n",
      "\n",
      "Percent flooded: 10.651708069982094 percent out of 4195759 possible locations\n"
     ]
    }
   ],
   "source": [
    "# How many nodes experienced flooding?\n",
    "pc_flooded = join50[\"PFU_1in50\"].count() / len(join50) * 100\n",
    "\n",
    "print(\"No flood crossing at node:\", join50[\"PFU_1in50\"].isnull().sum(), \"locations\", end=\"\\n\")\n",
    "print(\"Flood crossing at node:\", join50[\"PFU_1in50\"].count(), \"locations\", end=\"\\n\")\n",
    "print(\"\\nPercent flooded:\", pc_flooded, \"percent\", \"out of\", len(join50), \"possible locations\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'geopandas.geodataframe.GeoDataFrame'>\n",
      "Int64Index: 4195759 entries, 0 to 4012564\n",
      "Data columns (total 9 columns):\n",
      " #   Column     Dtype   \n",
      "---  ------     -----   \n",
      " 0   ID_graph   int64   \n",
      " 1   stnode     int64   \n",
      " 2   endnode    int64   \n",
      " 3   time       float64 \n",
      " 4   length     float64 \n",
      " 5   highway    object  \n",
      " 6   osmid      int64   \n",
      " 7   geometry   geometry\n",
      " 8   PFU_1in50  float64 \n",
      "dtypes: float64(3), geometry(1), int64(4), object(1)\n",
      "memory usage: 320.1+ MB\n"
     ]
    }
   ],
   "source": [
    "join50 = join50[['ID_graph', 'stnode', 'endnode', 'time', 'length', 'highway', 'osmid', 'geometry', 'PFU_1in50']]\n",
    "join50.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Geographic 2D CRS: EPSG:4326>\n",
       "Name: WGS 84\n",
       "Axis Info [ellipsoidal]:\n",
       "- Lat[north]: Geodetic latitude (degree)\n",
       "- Lon[east]: Geodetic longitude (degree)\n",
       "Area of Use:\n",
       "- name: World.\n",
       "- bounds: (-180.0, -90.0, 180.0, 90.0)\n",
       "Datum: World Geodetic System 1984 ensemble\n",
       "- Ellipsoid: WGS 84\n",
       "- Prime Meridian: Greenwich"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "join50 = join50.to_crs(\"EPSG:4326\")\n",
    "join50.crs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_graph</th>\n",
       "      <th>stnode</th>\n",
       "      <th>endnode</th>\n",
       "      <th>time</th>\n",
       "      <th>length</th>\n",
       "      <th>highway</th>\n",
       "      <th>osmid</th>\n",
       "      <th>geometry</th>\n",
       "      <th>PFU_1in50</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5217543379</td>\n",
       "      <td>2.385144</td>\n",
       "      <td>33.127</td>\n",
       "      <td>unclassified</td>\n",
       "      <td>59618174</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32368 12.3...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>358284990</td>\n",
       "      <td>1888282175</td>\n",
       "      <td>0.769920</td>\n",
       "      <td>12.832</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32351 12.3...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5329792467</td>\n",
       "      <td>2.926860</td>\n",
       "      <td>48.781</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32317 12.3...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888282575</td>\n",
       "      <td>7.594620</td>\n",
       "      <td>126.577</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28205 12.4...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888198886</td>\n",
       "      <td>7.234680</td>\n",
       "      <td>120.578</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28061 12.4...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012560</th>\n",
       "      <td>4012560</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48521 14.72445, -17.48451 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012561</th>\n",
       "      <td>4012561</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48521 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012562</th>\n",
       "      <td>4012562</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>3700438702</td>\n",
       "      <td>0.076140</td>\n",
       "      <td>0.846</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48451 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012563</th>\n",
       "      <td>4012563</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>6048975958</td>\n",
       "      <td>8.840674</td>\n",
       "      <td>171.902</td>\n",
       "      <td>secondary</td>\n",
       "      <td>177950649</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50357 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012564</th>\n",
       "      <td>4012564</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>7493947541</td>\n",
       "      <td>1.501740</td>\n",
       "      <td>16.686</td>\n",
       "      <td>residential</td>\n",
       "      <td>801279967</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50490 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4195759 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID_graph      stnode     endnode      time   length       highway  \\\n",
       "0               0   358284990  5217543379  2.385144   33.127  unclassified   \n",
       "1               1   358284990  1888282175  0.769920   12.832      tertiary   \n",
       "2               2   358284990  5329792467  2.926860   48.781      tertiary   \n",
       "3               3   358284993  1888282575  7.594620  126.577      tertiary   \n",
       "4               4   358284993  1888198886  7.234680  120.578      tertiary   \n",
       "...           ...         ...         ...       ...      ...           ...   \n",
       "4012560   4012560  9246539941  9246539942  6.906780   76.742   residential   \n",
       "4012561   4012561  9246539942  9246539941  6.906780   76.742   residential   \n",
       "4012562   4012562  9246539942  3700438702  0.076140    0.846   residential   \n",
       "4012563   4012563  9276108905  6048975958  8.840674  171.902     secondary   \n",
       "4012564   4012564  9276108905  7493947541  1.501740   16.686   residential   \n",
       "\n",
       "             osmid                                           geometry  \\\n",
       "0         59618174  LINESTRING (-12.32347 12.38119, -12.32368 12.3...   \n",
       "1        178482063  LINESTRING (-12.32347 12.38119, -12.32351 12.3...   \n",
       "2        178482063  LINESTRING (-12.32347 12.38119, -12.32317 12.3...   \n",
       "3        178470940  LINESTRING (-12.28135 12.41380, -12.28205 12.4...   \n",
       "4        178470940  LINESTRING (-12.28135 12.41380, -12.28061 12.4...   \n",
       "...            ...                                                ...   \n",
       "4012560  366052716  LINESTRING (-17.48521 14.72445, -17.48451 14.7...   \n",
       "4012561  366052716  LINESTRING (-17.48451 14.72456, -17.48521 14.7...   \n",
       "4012562  366052716  LINESTRING (-17.48451 14.72456, -17.48451 14.7...   \n",
       "4012563  177950649  LINESTRING (-17.50499 14.74971, -17.50357 14.7...   \n",
       "4012564  801279967  LINESTRING (-17.50499 14.74971, -17.50490 14.7...   \n",
       "\n",
       "         PFU_1in50  \n",
       "0              NaN  \n",
       "1              NaN  \n",
       "2              NaN  \n",
       "3              NaN  \n",
       "4              NaN  \n",
       "...            ...  \n",
       "4012560        NaN  \n",
       "4012561        NaN  \n",
       "4012562        NaN  \n",
       "4012563        NaN  \n",
       "4012564        NaN  \n",
       "\n",
       "[4195759 rows x 9 columns]"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "join50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 4195759 entries, 0 to 4012564\n",
      "Data columns (total 9 columns):\n",
      " #   Column     Dtype   \n",
      "---  ------     -----   \n",
      " 0   ID_graph   int64   \n",
      " 1   stnode     int64   \n",
      " 2   endnode    int64   \n",
      " 3   time       float64 \n",
      " 4   length     float64 \n",
      " 5   highway    object  \n",
      " 6   osmid      int64   \n",
      " 7   geometry   geometry\n",
      " 8   PFU_1in50  float64 \n",
      "dtypes: float64(3), geometry(1), int64(4), object(1)\n",
      "memory usage: 320.1+ MB\n"
     ]
    }
   ],
   "source": [
    "# Fewer errors farther down when using dataframe instead of gdf\n",
    "join50_df = pd.DataFrame(join50)\n",
    "join50_df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_graph</th>\n",
       "      <th>stnode</th>\n",
       "      <th>endnode</th>\n",
       "      <th>time</th>\n",
       "      <th>length</th>\n",
       "      <th>highway</th>\n",
       "      <th>osmid</th>\n",
       "      <th>geometry</th>\n",
       "      <th>PFU_1in50</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5217543379</td>\n",
       "      <td>2.385144</td>\n",
       "      <td>33.127</td>\n",
       "      <td>unclassified</td>\n",
       "      <td>59618174</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32368 12.3...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>358284990</td>\n",
       "      <td>1888282175</td>\n",
       "      <td>0.769920</td>\n",
       "      <td>12.832</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32351 12.3...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5329792467</td>\n",
       "      <td>2.926860</td>\n",
       "      <td>48.781</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32317 12.3...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888282575</td>\n",
       "      <td>7.594620</td>\n",
       "      <td>126.577</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28205 12.4...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888198886</td>\n",
       "      <td>7.234680</td>\n",
       "      <td>120.578</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28061 12.4...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012560</th>\n",
       "      <td>4012560</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48521 14.72445, -17.48451 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012561</th>\n",
       "      <td>4012561</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48521 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012562</th>\n",
       "      <td>4012562</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>3700438702</td>\n",
       "      <td>0.076140</td>\n",
       "      <td>0.846</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48451 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012563</th>\n",
       "      <td>4012563</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>6048975958</td>\n",
       "      <td>8.840674</td>\n",
       "      <td>171.902</td>\n",
       "      <td>secondary</td>\n",
       "      <td>177950649</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50357 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012564</th>\n",
       "      <td>4012564</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>7493947541</td>\n",
       "      <td>1.501740</td>\n",
       "      <td>16.686</td>\n",
       "      <td>residential</td>\n",
       "      <td>801279967</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50490 14.7...</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4195759 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID_graph      stnode     endnode      time   length       highway  \\\n",
       "0               0   358284990  5217543379  2.385144   33.127  unclassified   \n",
       "1               1   358284990  1888282175  0.769920   12.832      tertiary   \n",
       "2               2   358284990  5329792467  2.926860   48.781      tertiary   \n",
       "3               3   358284993  1888282575  7.594620  126.577      tertiary   \n",
       "4               4   358284993  1888198886  7.234680  120.578      tertiary   \n",
       "...           ...         ...         ...       ...      ...           ...   \n",
       "4012560   4012560  9246539941  9246539942  6.906780   76.742   residential   \n",
       "4012561   4012561  9246539942  9246539941  6.906780   76.742   residential   \n",
       "4012562   4012562  9246539942  3700438702  0.076140    0.846   residential   \n",
       "4012563   4012563  9276108905  6048975958  8.840674  171.902     secondary   \n",
       "4012564   4012564  9276108905  7493947541  1.501740   16.686   residential   \n",
       "\n",
       "             osmid                                           geometry  \\\n",
       "0         59618174  LINESTRING (-12.32347 12.38119, -12.32368 12.3...   \n",
       "1        178482063  LINESTRING (-12.32347 12.38119, -12.32351 12.3...   \n",
       "2        178482063  LINESTRING (-12.32347 12.38119, -12.32317 12.3...   \n",
       "3        178470940  LINESTRING (-12.28135 12.41380, -12.28205 12.4...   \n",
       "4        178470940  LINESTRING (-12.28135 12.41380, -12.28061 12.4...   \n",
       "...            ...                                                ...   \n",
       "4012560  366052716  LINESTRING (-17.48521 14.72445, -17.48451 14.7...   \n",
       "4012561  366052716  LINESTRING (-17.48451 14.72456, -17.48521 14.7...   \n",
       "4012562  366052716  LINESTRING (-17.48451 14.72456, -17.48451 14.7...   \n",
       "4012563  177950649  LINESTRING (-17.50499 14.74971, -17.50357 14.7...   \n",
       "4012564  801279967  LINESTRING (-17.50499 14.74971, -17.50490 14.7...   \n",
       "\n",
       "         PFU_1in50  \n",
       "0              NaN  \n",
       "1              NaN  \n",
       "2              NaN  \n",
       "3              NaN  \n",
       "4              NaN  \n",
       "...            ...  \n",
       "4012560        NaN  \n",
       "4012561        NaN  \n",
       "4012562        NaN  \n",
       "4012563        NaN  \n",
       "4012564        NaN  \n",
       "\n",
       "[4195759 rows x 9 columns]"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "join50_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "join50_df.to_csv(os.path.join(out_pth, 'gTime_flood50_intermediate.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create speed penalties.\n",
    "Note: Flood depths are in centimeters. FATHOM uses meters, but conversion process to vector required some finessing. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Give a depth to the nodes that don't cross a flood point. \n",
    "join50.loc[join50['PFU_1in50'].isnull(), 'PFU_1in50'] = -1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_graph</th>\n",
       "      <th>stnode</th>\n",
       "      <th>endnode</th>\n",
       "      <th>time</th>\n",
       "      <th>length</th>\n",
       "      <th>highway</th>\n",
       "      <th>osmid</th>\n",
       "      <th>geometry</th>\n",
       "      <th>PFU_1in50</th>\n",
       "      <th>t50</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5217543379</td>\n",
       "      <td>2.385144</td>\n",
       "      <td>33.127</td>\n",
       "      <td>unclassified</td>\n",
       "      <td>59618174</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32368 12.3...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>358284990</td>\n",
       "      <td>1888282175</td>\n",
       "      <td>0.769920</td>\n",
       "      <td>12.832</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32351 12.3...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5329792467</td>\n",
       "      <td>2.926860</td>\n",
       "      <td>48.781</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32317 12.3...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888282575</td>\n",
       "      <td>7.594620</td>\n",
       "      <td>126.577</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28205 12.4...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888198886</td>\n",
       "      <td>7.234680</td>\n",
       "      <td>120.578</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28061 12.4...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012560</th>\n",
       "      <td>4012560</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48521 14.72445, -17.48451 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012561</th>\n",
       "      <td>4012561</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48521 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012562</th>\n",
       "      <td>4012562</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>3700438702</td>\n",
       "      <td>0.076140</td>\n",
       "      <td>0.846</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48451 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012563</th>\n",
       "      <td>4012563</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>6048975958</td>\n",
       "      <td>8.840674</td>\n",
       "      <td>171.902</td>\n",
       "      <td>secondary</td>\n",
       "      <td>177950649</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50357 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012564</th>\n",
       "      <td>4012564</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>7493947541</td>\n",
       "      <td>1.501740</td>\n",
       "      <td>16.686</td>\n",
       "      <td>residential</td>\n",
       "      <td>801279967</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50490 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4195759 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID_graph      stnode     endnode      time   length       highway  \\\n",
       "0               0   358284990  5217543379  2.385144   33.127  unclassified   \n",
       "1               1   358284990  1888282175  0.769920   12.832      tertiary   \n",
       "2               2   358284990  5329792467  2.926860   48.781      tertiary   \n",
       "3               3   358284993  1888282575  7.594620  126.577      tertiary   \n",
       "4               4   358284993  1888198886  7.234680  120.578      tertiary   \n",
       "...           ...         ...         ...       ...      ...           ...   \n",
       "4012560   4012560  9246539941  9246539942  6.906780   76.742   residential   \n",
       "4012561   4012561  9246539942  9246539941  6.906780   76.742   residential   \n",
       "4012562   4012562  9246539942  3700438702  0.076140    0.846   residential   \n",
       "4012563   4012563  9276108905  6048975958  8.840674  171.902     secondary   \n",
       "4012564   4012564  9276108905  7493947541  1.501740   16.686   residential   \n",
       "\n",
       "             osmid                                           geometry  \\\n",
       "0         59618174  LINESTRING (-12.32347 12.38119, -12.32368 12.3...   \n",
       "1        178482063  LINESTRING (-12.32347 12.38119, -12.32351 12.3...   \n",
       "2        178482063  LINESTRING (-12.32347 12.38119, -12.32317 12.3...   \n",
       "3        178470940  LINESTRING (-12.28135 12.41380, -12.28205 12.4...   \n",
       "4        178470940  LINESTRING (-12.28135 12.41380, -12.28061 12.4...   \n",
       "...            ...                                                ...   \n",
       "4012560  366052716  LINESTRING (-17.48521 14.72445, -17.48451 14.7...   \n",
       "4012561  366052716  LINESTRING (-17.48451 14.72456, -17.48521 14.7...   \n",
       "4012562  366052716  LINESTRING (-17.48451 14.72456, -17.48451 14.7...   \n",
       "4012563  177950649  LINESTRING (-17.50499 14.74971, -17.50357 14.7...   \n",
       "4012564  801279967  LINESTRING (-17.50499 14.74971, -17.50490 14.7...   \n",
       "\n",
       "         PFU_1in50  t50  \n",
       "0             -1.0  1.0  \n",
       "1             -1.0  1.0  \n",
       "2             -1.0  1.0  \n",
       "3             -1.0  1.0  \n",
       "4             -1.0  1.0  \n",
       "...            ...  ...  \n",
       "4012560       -1.0  1.0  \n",
       "4012561       -1.0  1.0  \n",
       "4012562       -1.0  1.0  \n",
       "4012563       -1.0  1.0  \n",
       "4012564       -1.0  1.0  \n",
       "\n",
       "[4195759 rows x 10 columns]"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "join50[\"t50\"] = 1 # This is the penalty column.\n",
    "join50.loc[join50['PFU_1in50'] < 0, 't50'] = 1 # Where no flood crosses, keep the default value (no penalty).\n",
    "join50.loc[(join50['PFU_1in50'] > 10) & (join50['PFU_1in50'] <= 30), 't50'] = 1.25\n",
    "join50.loc[(join50['PFU_1in50'] > 30) & (join50['PFU_1in50'] <= 60), 't50'] = 2\n",
    "join50.loc[(join50['PFU_1in50'] > 60) & (join50['PFU_1in50'] <= 90), 't50'] = 5\n",
    "join50.loc[(join50['PFU_1in50'] > 90), 't50'] = 9999\n",
    "join50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_graph</th>\n",
       "      <th>stnode</th>\n",
       "      <th>endnode</th>\n",
       "      <th>time</th>\n",
       "      <th>length</th>\n",
       "      <th>highway</th>\n",
       "      <th>osmid</th>\n",
       "      <th>geometry</th>\n",
       "      <th>PFU_1in50</th>\n",
       "      <th>t50</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5217543379</td>\n",
       "      <td>2.385144</td>\n",
       "      <td>33.127</td>\n",
       "      <td>unclassified</td>\n",
       "      <td>59618174</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32368 12.3...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>2.385144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>358284990</td>\n",
       "      <td>1888282175</td>\n",
       "      <td>0.769920</td>\n",
       "      <td>12.832</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32351 12.3...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>0.769920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>358284990</td>\n",
       "      <td>5329792467</td>\n",
       "      <td>2.926860</td>\n",
       "      <td>48.781</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178482063</td>\n",
       "      <td>LINESTRING (-12.32347 12.38119, -12.32317 12.3...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>2.926860</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888282575</td>\n",
       "      <td>7.594620</td>\n",
       "      <td>126.577</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28205 12.4...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>7.594620</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>358284993</td>\n",
       "      <td>1888198886</td>\n",
       "      <td>7.234680</td>\n",
       "      <td>120.578</td>\n",
       "      <td>tertiary</td>\n",
       "      <td>178470940</td>\n",
       "      <td>LINESTRING (-12.28135 12.41380, -12.28061 12.4...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>7.234680</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012560</th>\n",
       "      <td>4012560</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48521 14.72445, -17.48451 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>6.906780</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012561</th>\n",
       "      <td>4012561</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>9246539941</td>\n",
       "      <td>6.906780</td>\n",
       "      <td>76.742</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48521 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>6.906780</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012562</th>\n",
       "      <td>4012562</td>\n",
       "      <td>9246539942</td>\n",
       "      <td>3700438702</td>\n",
       "      <td>0.076140</td>\n",
       "      <td>0.846</td>\n",
       "      <td>residential</td>\n",
       "      <td>366052716</td>\n",
       "      <td>LINESTRING (-17.48451 14.72456, -17.48451 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>0.076140</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012563</th>\n",
       "      <td>4012563</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>6048975958</td>\n",
       "      <td>8.840674</td>\n",
       "      <td>171.902</td>\n",
       "      <td>secondary</td>\n",
       "      <td>177950649</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50357 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>8.840674</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4012564</th>\n",
       "      <td>4012564</td>\n",
       "      <td>9276108905</td>\n",
       "      <td>7493947541</td>\n",
       "      <td>1.501740</td>\n",
       "      <td>16.686</td>\n",
       "      <td>residential</td>\n",
       "      <td>801279967</td>\n",
       "      <td>LINESTRING (-17.50499 14.74971, -17.50490 14.7...</td>\n",
       "      <td>-1.0</td>\n",
       "      <td>1.501740</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>4195759 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         ID_graph      stnode     endnode      time   length       highway  \\\n",
       "0               0   358284990  5217543379  2.385144   33.127  unclassified   \n",
       "1               1   358284990  1888282175  0.769920   12.832      tertiary   \n",
       "2               2   358284990  5329792467  2.926860   48.781      tertiary   \n",
       "3               3   358284993  1888282575  7.594620  126.577      tertiary   \n",
       "4               4   358284993  1888198886  7.234680  120.578      tertiary   \n",
       "...           ...         ...         ...       ...      ...           ...   \n",
       "4012560   4012560  9246539941  9246539942  6.906780   76.742   residential   \n",
       "4012561   4012561  9246539942  9246539941  6.906780   76.742   residential   \n",
       "4012562   4012562  9246539942  3700438702  0.076140    0.846   residential   \n",
       "4012563   4012563  9276108905  6048975958  8.840674  171.902     secondary   \n",
       "4012564   4012564  9276108905  7493947541  1.501740   16.686   residential   \n",
       "\n",
       "             osmid                                           geometry  \\\n",
       "0         59618174  LINESTRING (-12.32347 12.38119, -12.32368 12.3...   \n",
       "1        178482063  LINESTRING (-12.32347 12.38119, -12.32351 12.3...   \n",
       "2        178482063  LINESTRING (-12.32347 12.38119, -12.32317 12.3...   \n",
       "3        178470940  LINESTRING (-12.28135 12.41380, -12.28205 12.4...   \n",
       "4        178470940  LINESTRING (-12.28135 12.41380, -12.28061 12.4...   \n",
       "...            ...                                                ...   \n",
       "4012560  366052716  LINESTRING (-17.48521 14.72445, -17.48451 14.7...   \n",
       "4012561  366052716  LINESTRING (-17.48451 14.72456, -17.48521 14.7...   \n",
       "4012562  366052716  LINESTRING (-17.48451 14.72456, -17.48451 14.7...   \n",
       "4012563  177950649  LINESTRING (-17.50499 14.74971, -17.50357 14.7...   \n",
       "4012564  801279967  LINESTRING (-17.50499 14.74971, -17.50490 14.7...   \n",
       "\n",
       "         PFU_1in50       t50  \n",
       "0             -1.0  2.385144  \n",
       "1             -1.0  0.769920  \n",
       "2             -1.0  2.926860  \n",
       "3             -1.0  7.594620  \n",
       "4             -1.0  7.234680  \n",
       "...            ...       ...  \n",
       "4012560       -1.0  6.906780  \n",
       "4012561       -1.0  6.906780  \n",
       "4012562       -1.0  0.076140  \n",
       "4012563       -1.0  8.840674  \n",
       "4012564       -1.0  1.501740  \n",
       "\n",
       "[4195759 rows x 10 columns]"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Turn the penalty column into a flood-affected time column.\n",
    "join50['t50'] = join50['t50'] * join50['time']\n",
    "join50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "join50.to_csv(os.path.join(out_pth, 'join50.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert back to graph object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\wb527163\\.conda\\envs\\geo\\lib\\site-packages\\IPython\\core\\interactiveshell.py:3444: DtypeWarning: Columns (5) have mixed types.Specify dtype option on import or set low_memory=False.\n",
      "  exec(code_obj, self.user_global_ns, self.user_ns)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 1829568 entries, 0 to 1829567\n",
      "Data columns (total 7 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   node_ID     int64  \n",
      " 2   y           float64\n",
      " 3   highway     object \n",
      " 4   x           float64\n",
      " 5   ref         object \n",
      " 6   geometry    object \n",
      "dtypes: float64(2), int64(2), object(3)\n",
      "memory usage: 97.7+ MB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 4195759 entries, 0 to 4195758\n",
      "Data columns (total 11 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID_graph    int64  \n",
      " 2   stnode      int64  \n",
      " 3   endnode     int64  \n",
      " 4   time        float64\n",
      " 5   length      float64\n",
      " 6   highway     object \n",
      " 7   osmid       int64  \n",
      " 8   geometry    object \n",
      " 9   PFU_1in50   float64\n",
      " 10  t50         float64\n",
      "dtypes: float64(4), int64(5), object(2)\n",
      "memory usage: 352.1+ MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# Converting back to graph can cause memory errors. Suggested to restart the kernel and reload the nodes and revised edges at this point.\n",
    "gTime_node = os.path.join(pth, \"gTime_nodes.csv\")\n",
    "gTime_node = pd.read_csv(gTime_node)\n",
    "join50 = os.path.join(out_pth, \"join50.csv\")\n",
    "join50 = pd.read_csv(join50)\n",
    "print(gTime_node.info())\n",
    "print(join50.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Wed Dec 22 13:12:40 2021\n",
      "\n",
      "(358284990, 5217543379, {'geometry': <shapely.geometry.linestring.LineString object at 0x000001411CF7FF40>, 'Unnamed: 0': 0, 'ID_graph': 0, 'time': 2.3851440000000004, 'length': 33.127, 'highway': 'unclassified', 'osmid': 59618174, 'PFU_1in50': -1.0, 't50': 2.3851440000000004})\n",
      "(358284990, 1888282175, {'geometry': <shapely.geometry.linestring.LineString object at 0x000001411CF7FEB0>, 'Unnamed: 0': 1, 'ID_graph': 1, 'time': 0.76992, 'length': 12.832, 'highway': 'tertiary', 'osmid': 178482063, 'PFU_1in50': -1.0, 't50': 0.76992})\n",
      "(358284990, 5329792467, {'geometry': <shapely.geometry.linestring.LineString object at 0x000001411CF7FE80>, 'Unnamed: 0': 2, 'ID_graph': 2, 'time': 2.92686, 'length': 48.781, 'highway': 'tertiary', 'osmid': 178482063, 'PFU_1in50': -1.0, 't50': 2.92686})\n",
      "(5217543379, 358284990, {'geometry': <shapely.geometry.linestring.LineString object at 0x0000014189C99340>, 'Unnamed: 0': 16698, 'ID_graph': 16698, 'time': 2.3851440000000004, 'length': 33.127, 'highway': 'unclassified', 'osmid': 59618174, 'PFU_1in50': -1.0, 't50': 2.3851440000000004})\n",
      "(5217543379, 5329928981, {'geometry': <shapely.geometry.linestring.LineString object at 0x0000014189C993A0>, 'Unnamed: 0': 16699, 'ID_graph': 16699, 'time': 1.232928, 'length': 17.124, 'highway': 'unclassified', 'osmid': 59618174, 'PFU_1in50': -1.0, 't50': 1.232928})\n",
      "(1888282175, 1888282174, {'geometry': <shapely.geometry.linestring.LineString object at 0x00000141893265B0>, 'Unnamed: 0': 1988, 'ID_graph': 1988, 'time': 0.9322200000000002, 'length': 15.537, 'highway': 'tertiary', 'osmid': 178482063, 'PFU_1in50': -1.0, 't50': 0.9322200000000002})\n",
      "(1888282175, 358284990, {'geometry': <shapely.geometry.linestring.LineString object at 0x0000014189326610>, 'Unnamed: 0': 1989, 'ID_graph': 1989, 'time': 0.76992, 'length': 12.832, 'highway': 'tertiary', 'osmid': 178482063, 'PFU_1in50': -1.0, 't50': 0.76992})\n",
      "(5329792467, 358284990, {'geometry': <shapely.geometry.linestring.LineString object at 0x0000014189CCFBE0>, 'Unnamed: 0': 17057, 'ID_graph': 17057, 'time': 2.92686, 'length': 48.781, 'highway': 'tertiary', 'osmid': 178482063, 'PFU_1in50': -1.0, 't50': 2.92686})\n",
      "(5329792467, 1888282177, {'geometry': <shapely.geometry.linestring.LineString object at 0x0000014189CCFC40>, 'Unnamed: 0': 17058, 'ID_graph': 17058, 'time': 2.7629999999999995, 'length': 46.05, 'highway': 'tertiary', 'osmid': 178482063, 'PFU_1in50': -1.0, 't50': 2.7629999999999995})\n",
      "(358284993, 1888282575, {'geometry': <shapely.geometry.linestring.LineString object at 0x00000141155290A0>, 'Unnamed: 0': 3, 'ID_graph': 3, 'time': 7.59462, 'length': 126.577, 'highway': 'tertiary', 'osmid': 178470940, 'PFU_1in50': -1.0, 't50': 7.59462})\n",
      "\n",
      "end: Wed Dec 22 13:21:57 2021\n",
      "\n",
      "--- processing complete\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "G_flood = gn.edges_and_nodes_gdf_to_graph(gTime_node, join50, node_tag='node_ID', u_tag='stnode', v_tag='endnode', geometry_tag='geometry')\n",
    "gn.example_edge(G_flood, 10)\n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Wed Dec 22 13:21:57 2021\n",
      "\n",
      "\n",
      "end: Wed Dec 22 13:34:54 2021\n",
      "\n",
      "--- processing complete\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "gn.save(G_flood, 'gTime_flood50', out_pth, edges = True, nodes = True)\n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create travel time values for the road nodes nearest to each service.\n",
    "\n",
    "Using calculate_OD."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If starting a new session, load from file.\n",
    "HDurban_snap = os.path.join(out_pth, \"HDurban_snap.csv\")\n",
    "HDurban_snap = pd.read_csv(HDurban_snap)\n",
    "hamlet_snap = os.path.join(out_pth, \"hamlet_snap.csv\")\n",
    "hamlet_snap = pd.read_csv(hamlet_snap)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Tue Dec 28 14:07:59 2021\n",
      "\n",
      "\n",
      "end: Tue Dec 28 14:08:45 2021\n",
      "\n",
      "--- processing complete\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "ag_snap = os.path.join(out_pth, \"ag_snap.csv\")\n",
    "ag_snap = pd.read_csv(ag_snap)\n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "G_flood = nx.read_gpickle(\"SEN-Cdrive/outputs/gTime_flood10.pickle\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We only need to find the origin-destination pairs for nodes closest to the origins and services,\n",
    "# and some nodes will be the nearest for more than one service (and definitely for multiple origins).\n",
    "list_hamlet = list(hamlet_snap.NN.unique())\n",
    "list_ag = list(ag_snap.NN.unique())\n",
    "originslist = list_hamlet + list_ag\n",
    "origins = list(set(originslist))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "dests = list(HDurban_snap.NN.unique()) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "637854"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(origins)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "58"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(dests) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "fail_value = 999999999 # If there is no shortest path, the OD pair will be assigned the fail value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Tue Dec 28 14:10:29 2021\n",
      "\n",
      "\n",
      "end: Tue Dec 28 14:30:15 2021\n",
      "\n",
      "--- processing complete\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "OD = gn.calculate_OD(G_flood, origins, dests, fail_value, weight = 't10')\n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "OD_df = pd.DataFrame(OD, index = origins, columns = dests)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 637854 entries, 3571449893 to 8925478824\n",
      "Data columns (total 58 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   6058226279  637854 non-null  float64\n",
      " 1   6029307183  637854 non-null  float64\n",
      " 2   4998093094  637854 non-null  float64\n",
      " 3   2201506815  637854 non-null  float64\n",
      " 4   3474499811  637854 non-null  float64\n",
      " 5   1697006012  637854 non-null  float64\n",
      " 6   1901689169  637854 non-null  float64\n",
      " 7   6032060028  637854 non-null  float64\n",
      " 8   6040927878  637854 non-null  float64\n",
      " 9   3449495495  637854 non-null  float64\n",
      " 10  3990543961  637854 non-null  float64\n",
      " 11  8972391475  637854 non-null  float64\n",
      " 12  3418418812  637854 non-null  float64\n",
      " 13  1983641803  637854 non-null  float64\n",
      " 14  6014451367  637854 non-null  float64\n",
      " 15  6027163276  637854 non-null  float64\n",
      " 16  2833577858  637854 non-null  float64\n",
      " 17  4656728818  637854 non-null  float64\n",
      " 18  6045659373  637854 non-null  float64\n",
      " 19  5528866190  637854 non-null  float64\n",
      " 20  6027939517  637854 non-null  float64\n",
      " 21  6031093257  637854 non-null  float64\n",
      " 22  1883155712  637854 non-null  float64\n",
      " 23  6026417971  637854 non-null  float64\n",
      " 24  6031245697  637854 non-null  float64\n",
      " 25  7493947593  637854 non-null  float64\n",
      " 26  1859359090  637854 non-null  float64\n",
      " 27  6053816462  637854 non-null  float64\n",
      " 28  6027530676  637854 non-null  float64\n",
      " 29  2988435647  637854 non-null  float64\n",
      " 30  1332801944  637854 non-null  float64\n",
      " 31  6053825179  637854 non-null  float64\n",
      " 32  2988435792  637854 non-null  float64\n",
      " 33  3036877534  637854 non-null  float64\n",
      " 34  6056472223  637854 non-null  float64\n",
      " 35  7460241128  637854 non-null  float64\n",
      " 36  6028595154  637854 non-null  float64\n",
      " 37  4053739078  637854 non-null  float64\n",
      " 38  4071261250  637854 non-null  float64\n",
      " 39  6014815762  637854 non-null  float64\n",
      " 40  6024430122  637854 non-null  float64\n",
      " 41  8229317581  637854 non-null  float64\n",
      " 42  6027834438  637854 non-null  float64\n",
      " 43  6024894515  637854 non-null  float64\n",
      " 44  6024894530  637854 non-null  float64\n",
      " 45  6024894513  637854 non-null  float64\n",
      " 46  6031993937  637854 non-null  float64\n",
      " 47  8200298999  637854 non-null  float64\n",
      " 48  1968458114  637854 non-null  float64\n",
      " 49  1936967272  637854 non-null  float64\n",
      " 50  3496518021  637854 non-null  float64\n",
      " 51  6027615161  637854 non-null  float64\n",
      " 52  6027276892  637854 non-null  float64\n",
      " 53  6041228287  637854 non-null  float64\n",
      " 54  5536661253  637854 non-null  float64\n",
      " 55  7357630367  637854 non-null  float64\n",
      " 56  8178147277  637854 non-null  float64\n",
      " 57  6026834850  637854 non-null  float64\n",
      "dtypes: float64(58)\n",
      "memory usage: 287.1 MB\n"
     ]
    }
   ],
   "source": [
    "OD_df.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>6058226279</th>\n",
       "      <th>6029307183</th>\n",
       "      <th>4998093094</th>\n",
       "      <th>2201506815</th>\n",
       "      <th>3474499811</th>\n",
       "      <th>1697006012</th>\n",
       "      <th>1901689169</th>\n",
       "      <th>6032060028</th>\n",
       "      <th>6040927878</th>\n",
       "      <th>3449495495</th>\n",
       "      <th>...</th>\n",
       "      <th>1968458114</th>\n",
       "      <th>1936967272</th>\n",
       "      <th>3496518021</th>\n",
       "      <th>6027615161</th>\n",
       "      <th>6027276892</th>\n",
       "      <th>6041228287</th>\n",
       "      <th>5536661253</th>\n",
       "      <th>7357630367</th>\n",
       "      <th>8178147277</th>\n",
       "      <th>6026834850</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3571449893</th>\n",
       "      <td>212.584065</td>\n",
       "      <td>72.370016</td>\n",
       "      <td>263.336586</td>\n",
       "      <td>178.755639</td>\n",
       "      <td>161.903071</td>\n",
       "      <td>296.154013</td>\n",
       "      <td>309.228517</td>\n",
       "      <td>293.150349</td>\n",
       "      <td>227.295815</td>\n",
       "      <td>261.330804</td>\n",
       "      <td>...</td>\n",
       "      <td>350.466140</td>\n",
       "      <td>403.469275</td>\n",
       "      <td>155.710545</td>\n",
       "      <td>155.527216</td>\n",
       "      <td>267.669804</td>\n",
       "      <td>306.069012</td>\n",
       "      <td>306.987423</td>\n",
       "      <td>312.310460</td>\n",
       "      <td>259.120605</td>\n",
       "      <td>271.617579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3571449966</th>\n",
       "      <td>212.970977</td>\n",
       "      <td>72.756927</td>\n",
       "      <td>263.723498</td>\n",
       "      <td>179.142551</td>\n",
       "      <td>162.289982</td>\n",
       "      <td>296.540925</td>\n",
       "      <td>309.615428</td>\n",
       "      <td>293.537260</td>\n",
       "      <td>227.682727</td>\n",
       "      <td>261.717716</td>\n",
       "      <td>...</td>\n",
       "      <td>350.853051</td>\n",
       "      <td>403.856187</td>\n",
       "      <td>156.097456</td>\n",
       "      <td>155.914128</td>\n",
       "      <td>268.056716</td>\n",
       "      <td>306.455923</td>\n",
       "      <td>307.374334</td>\n",
       "      <td>312.697372</td>\n",
       "      <td>259.507517</td>\n",
       "      <td>272.004490</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3405774993</th>\n",
       "      <td>84.548330</td>\n",
       "      <td>164.723247</td>\n",
       "      <td>253.520017</td>\n",
       "      <td>168.939070</td>\n",
       "      <td>152.086502</td>\n",
       "      <td>286.337444</td>\n",
       "      <td>299.411948</td>\n",
       "      <td>283.333780</td>\n",
       "      <td>217.479246</td>\n",
       "      <td>251.514235</td>\n",
       "      <td>...</td>\n",
       "      <td>340.649571</td>\n",
       "      <td>393.652706</td>\n",
       "      <td>290.908473</td>\n",
       "      <td>290.725144</td>\n",
       "      <td>257.853235</td>\n",
       "      <td>296.252443</td>\n",
       "      <td>297.170854</td>\n",
       "      <td>302.493891</td>\n",
       "      <td>249.304036</td>\n",
       "      <td>261.801010</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3405774994</th>\n",
       "      <td>84.949079</td>\n",
       "      <td>164.322498</td>\n",
       "      <td>253.119268</td>\n",
       "      <td>168.538321</td>\n",
       "      <td>151.685753</td>\n",
       "      <td>285.936695</td>\n",
       "      <td>299.011199</td>\n",
       "      <td>282.933031</td>\n",
       "      <td>217.078497</td>\n",
       "      <td>251.113486</td>\n",
       "      <td>...</td>\n",
       "      <td>340.248822</td>\n",
       "      <td>393.251957</td>\n",
       "      <td>290.507724</td>\n",
       "      <td>290.324395</td>\n",
       "      <td>257.452486</td>\n",
       "      <td>295.851694</td>\n",
       "      <td>296.770105</td>\n",
       "      <td>302.093142</td>\n",
       "      <td>248.903287</td>\n",
       "      <td>261.400261</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3405774995</th>\n",
       "      <td>85.282625</td>\n",
       "      <td>163.988952</td>\n",
       "      <td>252.785722</td>\n",
       "      <td>168.204775</td>\n",
       "      <td>151.352207</td>\n",
       "      <td>285.603149</td>\n",
       "      <td>298.677653</td>\n",
       "      <td>282.599485</td>\n",
       "      <td>216.744951</td>\n",
       "      <td>250.779940</td>\n",
       "      <td>...</td>\n",
       "      <td>339.915276</td>\n",
       "      <td>392.918411</td>\n",
       "      <td>290.174178</td>\n",
       "      <td>289.990849</td>\n",
       "      <td>257.118940</td>\n",
       "      <td>295.518148</td>\n",
       "      <td>296.436559</td>\n",
       "      <td>301.759596</td>\n",
       "      <td>248.569741</td>\n",
       "      <td>261.066715</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8633974649</th>\n",
       "      <td>289.719581</td>\n",
       "      <td>146.027718</td>\n",
       "      <td>436.060128</td>\n",
       "      <td>351.479181</td>\n",
       "      <td>334.626612</td>\n",
       "      <td>468.877555</td>\n",
       "      <td>481.952058</td>\n",
       "      <td>465.873890</td>\n",
       "      <td>400.019357</td>\n",
       "      <td>434.054346</td>\n",
       "      <td>...</td>\n",
       "      <td>523.189681</td>\n",
       "      <td>576.192817</td>\n",
       "      <td>188.633881</td>\n",
       "      <td>188.450553</td>\n",
       "      <td>440.393346</td>\n",
       "      <td>478.792554</td>\n",
       "      <td>479.710964</td>\n",
       "      <td>485.034002</td>\n",
       "      <td>431.844147</td>\n",
       "      <td>444.341121</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8633974652</th>\n",
       "      <td>289.629165</td>\n",
       "      <td>145.937301</td>\n",
       "      <td>435.969711</td>\n",
       "      <td>351.388764</td>\n",
       "      <td>334.536196</td>\n",
       "      <td>468.787138</td>\n",
       "      <td>481.861642</td>\n",
       "      <td>465.783474</td>\n",
       "      <td>399.928940</td>\n",
       "      <td>433.963930</td>\n",
       "      <td>...</td>\n",
       "      <td>523.099265</td>\n",
       "      <td>576.102400</td>\n",
       "      <td>188.543465</td>\n",
       "      <td>188.360136</td>\n",
       "      <td>440.302929</td>\n",
       "      <td>478.702137</td>\n",
       "      <td>479.620548</td>\n",
       "      <td>484.943585</td>\n",
       "      <td>431.753731</td>\n",
       "      <td>444.250704</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8633974653</th>\n",
       "      <td>289.563206</td>\n",
       "      <td>145.871342</td>\n",
       "      <td>435.903752</td>\n",
       "      <td>351.322805</td>\n",
       "      <td>334.470237</td>\n",
       "      <td>468.721179</td>\n",
       "      <td>481.795683</td>\n",
       "      <td>465.717515</td>\n",
       "      <td>399.862981</td>\n",
       "      <td>433.897970</td>\n",
       "      <td>...</td>\n",
       "      <td>523.033306</td>\n",
       "      <td>576.036441</td>\n",
       "      <td>188.477505</td>\n",
       "      <td>188.294177</td>\n",
       "      <td>440.236970</td>\n",
       "      <td>478.636178</td>\n",
       "      <td>479.554589</td>\n",
       "      <td>484.877626</td>\n",
       "      <td>431.687771</td>\n",
       "      <td>444.184745</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8633974654</th>\n",
       "      <td>289.530430</td>\n",
       "      <td>145.838566</td>\n",
       "      <td>435.870977</td>\n",
       "      <td>351.290029</td>\n",
       "      <td>334.437461</td>\n",
       "      <td>468.688404</td>\n",
       "      <td>481.762907</td>\n",
       "      <td>465.684739</td>\n",
       "      <td>399.830206</td>\n",
       "      <td>433.865195</td>\n",
       "      <td>...</td>\n",
       "      <td>523.000530</td>\n",
       "      <td>576.003665</td>\n",
       "      <td>188.444730</td>\n",
       "      <td>188.261401</td>\n",
       "      <td>440.204195</td>\n",
       "      <td>478.603402</td>\n",
       "      <td>479.521813</td>\n",
       "      <td>484.844851</td>\n",
       "      <td>431.654996</td>\n",
       "      <td>444.151969</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8925478824</th>\n",
       "      <td>336.683970</td>\n",
       "      <td>298.639720</td>\n",
       "      <td>93.717882</td>\n",
       "      <td>94.267267</td>\n",
       "      <td>181.970075</td>\n",
       "      <td>56.833006</td>\n",
       "      <td>96.320762</td>\n",
       "      <td>46.648633</td>\n",
       "      <td>49.228073</td>\n",
       "      <td>79.486425</td>\n",
       "      <td>...</td>\n",
       "      <td>82.527629</td>\n",
       "      <td>135.530765</td>\n",
       "      <td>424.824946</td>\n",
       "      <td>424.641618</td>\n",
       "      <td>95.234620</td>\n",
       "      <td>91.379411</td>\n",
       "      <td>90.851147</td>\n",
       "      <td>90.763607</td>\n",
       "      <td>81.587664</td>\n",
       "      <td>82.593782</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>637854 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            6058226279  6029307183  4998093094  2201506815  3474499811  \\\n",
       "3571449893  212.584065   72.370016  263.336586  178.755639  161.903071   \n",
       "3571449966  212.970977   72.756927  263.723498  179.142551  162.289982   \n",
       "3405774993   84.548330  164.723247  253.520017  168.939070  152.086502   \n",
       "3405774994   84.949079  164.322498  253.119268  168.538321  151.685753   \n",
       "3405774995   85.282625  163.988952  252.785722  168.204775  151.352207   \n",
       "...                ...         ...         ...         ...         ...   \n",
       "8633974649  289.719581  146.027718  436.060128  351.479181  334.626612   \n",
       "8633974652  289.629165  145.937301  435.969711  351.388764  334.536196   \n",
       "8633974653  289.563206  145.871342  435.903752  351.322805  334.470237   \n",
       "8633974654  289.530430  145.838566  435.870977  351.290029  334.437461   \n",
       "8925478824  336.683970  298.639720   93.717882   94.267267  181.970075   \n",
       "\n",
       "            1697006012  1901689169  6032060028  6040927878  3449495495  ...  \\\n",
       "3571449893  296.154013  309.228517  293.150349  227.295815  261.330804  ...   \n",
       "3571449966  296.540925  309.615428  293.537260  227.682727  261.717716  ...   \n",
       "3405774993  286.337444  299.411948  283.333780  217.479246  251.514235  ...   \n",
       "3405774994  285.936695  299.011199  282.933031  217.078497  251.113486  ...   \n",
       "3405774995  285.603149  298.677653  282.599485  216.744951  250.779940  ...   \n",
       "...                ...         ...         ...         ...         ...  ...   \n",
       "8633974649  468.877555  481.952058  465.873890  400.019357  434.054346  ...   \n",
       "8633974652  468.787138  481.861642  465.783474  399.928940  433.963930  ...   \n",
       "8633974653  468.721179  481.795683  465.717515  399.862981  433.897970  ...   \n",
       "8633974654  468.688404  481.762907  465.684739  399.830206  433.865195  ...   \n",
       "8925478824   56.833006   96.320762   46.648633   49.228073   79.486425  ...   \n",
       "\n",
       "            1968458114  1936967272  3496518021  6027615161  6027276892  \\\n",
       "3571449893  350.466140  403.469275  155.710545  155.527216  267.669804   \n",
       "3571449966  350.853051  403.856187  156.097456  155.914128  268.056716   \n",
       "3405774993  340.649571  393.652706  290.908473  290.725144  257.853235   \n",
       "3405774994  340.248822  393.251957  290.507724  290.324395  257.452486   \n",
       "3405774995  339.915276  392.918411  290.174178  289.990849  257.118940   \n",
       "...                ...         ...         ...         ...         ...   \n",
       "8633974649  523.189681  576.192817  188.633881  188.450553  440.393346   \n",
       "8633974652  523.099265  576.102400  188.543465  188.360136  440.302929   \n",
       "8633974653  523.033306  576.036441  188.477505  188.294177  440.236970   \n",
       "8633974654  523.000530  576.003665  188.444730  188.261401  440.204195   \n",
       "8925478824   82.527629  135.530765  424.824946  424.641618   95.234620   \n",
       "\n",
       "            6041228287  5536661253  7357630367  8178147277  6026834850  \n",
       "3571449893  306.069012  306.987423  312.310460  259.120605  271.617579  \n",
       "3571449966  306.455923  307.374334  312.697372  259.507517  272.004490  \n",
       "3405774993  296.252443  297.170854  302.493891  249.304036  261.801010  \n",
       "3405774994  295.851694  296.770105  302.093142  248.903287  261.400261  \n",
       "3405774995  295.518148  296.436559  301.759596  248.569741  261.066715  \n",
       "...                ...         ...         ...         ...         ...  \n",
       "8633974649  478.792554  479.710964  485.034002  431.844147  444.341121  \n",
       "8633974652  478.702137  479.620548  484.943585  431.753731  444.250704  \n",
       "8633974653  478.636178  479.554589  484.877626  431.687771  444.184745  \n",
       "8633974654  478.603402  479.521813  484.844851  431.654996  444.151969  \n",
       "8925478824   91.379411   90.851147   90.763607   81.587664   82.593782  \n",
       "\n",
       "[637854 rows x 58 columns]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Convert to minutes and save to file.\n",
    "OD_min = OD_df[OD_df <fail_value] / 60\n",
    "OD_min.to_csv(os.path.join(out_pth, 'OD_flood10_allorigins.csv'))\n",
    "OD_min"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>6058226279</th>\n",
       "      <th>6029307183</th>\n",
       "      <th>4998093094</th>\n",
       "      <th>2201506815</th>\n",
       "      <th>3474499811</th>\n",
       "      <th>1697006012</th>\n",
       "      <th>1901689169</th>\n",
       "      <th>6032060028</th>\n",
       "      <th>6040927878</th>\n",
       "      <th>3449495495</th>\n",
       "      <th>...</th>\n",
       "      <th>1968458114</th>\n",
       "      <th>1936967272</th>\n",
       "      <th>3496518021</th>\n",
       "      <th>6027615161</th>\n",
       "      <th>6027276892</th>\n",
       "      <th>6041228287</th>\n",
       "      <th>5536661253</th>\n",
       "      <th>7357630367</th>\n",
       "      <th>8178147277</th>\n",
       "      <th>6026834850</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3507831609</th>\n",
       "      <td>83.755636</td>\n",
       "      <td>232.445763</td>\n",
       "      <td>411.937336</td>\n",
       "      <td>327.356389</td>\n",
       "      <td>310.503821</td>\n",
       "      <td>444.754763</td>\n",
       "      <td>457.829267</td>\n",
       "      <td>441.751099</td>\n",
       "      <td>375.896565</td>\n",
       "      <td>409.931555</td>\n",
       "      <td>...</td>\n",
       "      <td>499.066890</td>\n",
       "      <td>552.070025</td>\n",
       "      <td>427.474675</td>\n",
       "      <td>427.291346</td>\n",
       "      <td>416.270554</td>\n",
       "      <td>454.669762</td>\n",
       "      <td>455.588173</td>\n",
       "      <td>460.911210</td>\n",
       "      <td>407.721356</td>\n",
       "      <td>420.218329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3507831510</th>\n",
       "      <td>90.114132</td>\n",
       "      <td>238.804260</td>\n",
       "      <td>418.295833</td>\n",
       "      <td>333.714886</td>\n",
       "      <td>316.862317</td>\n",
       "      <td>451.113260</td>\n",
       "      <td>464.187763</td>\n",
       "      <td>448.109595</td>\n",
       "      <td>382.255062</td>\n",
       "      <td>416.290051</td>\n",
       "      <td>...</td>\n",
       "      <td>505.425386</td>\n",
       "      <td>558.428522</td>\n",
       "      <td>433.833171</td>\n",
       "      <td>433.649843</td>\n",
       "      <td>422.629051</td>\n",
       "      <td>461.028259</td>\n",
       "      <td>461.946669</td>\n",
       "      <td>467.269707</td>\n",
       "      <td>414.079852</td>\n",
       "      <td>426.576826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6188134127</th>\n",
       "      <td>25.237636</td>\n",
       "      <td>173.927763</td>\n",
       "      <td>353.419336</td>\n",
       "      <td>268.838389</td>\n",
       "      <td>251.985821</td>\n",
       "      <td>386.236763</td>\n",
       "      <td>399.311267</td>\n",
       "      <td>383.233099</td>\n",
       "      <td>317.378565</td>\n",
       "      <td>351.413554</td>\n",
       "      <td>...</td>\n",
       "      <td>440.548890</td>\n",
       "      <td>493.552025</td>\n",
       "      <td>368.956675</td>\n",
       "      <td>368.773346</td>\n",
       "      <td>357.752554</td>\n",
       "      <td>396.151762</td>\n",
       "      <td>397.070173</td>\n",
       "      <td>402.393210</td>\n",
       "      <td>349.203355</td>\n",
       "      <td>361.700329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8631201421</th>\n",
       "      <td>96.422509</td>\n",
       "      <td>176.244632</td>\n",
       "      <td>424.604209</td>\n",
       "      <td>340.023262</td>\n",
       "      <td>323.170694</td>\n",
       "      <td>457.421636</td>\n",
       "      <td>470.496140</td>\n",
       "      <td>454.417972</td>\n",
       "      <td>388.563438</td>\n",
       "      <td>422.598428</td>\n",
       "      <td>...</td>\n",
       "      <td>511.733763</td>\n",
       "      <td>564.736898</td>\n",
       "      <td>367.072725</td>\n",
       "      <td>366.889397</td>\n",
       "      <td>428.937427</td>\n",
       "      <td>467.336635</td>\n",
       "      <td>468.255046</td>\n",
       "      <td>473.578083</td>\n",
       "      <td>420.388229</td>\n",
       "      <td>432.885202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8598305977</th>\n",
       "      <td>53.438771</td>\n",
       "      <td>155.021674</td>\n",
       "      <td>381.620471</td>\n",
       "      <td>297.039524</td>\n",
       "      <td>280.186956</td>\n",
       "      <td>414.437898</td>\n",
       "      <td>427.512402</td>\n",
       "      <td>411.434234</td>\n",
       "      <td>345.579700</td>\n",
       "      <td>379.614690</td>\n",
       "      <td>...</td>\n",
       "      <td>468.750025</td>\n",
       "      <td>521.753160</td>\n",
       "      <td>350.050585</td>\n",
       "      <td>349.867257</td>\n",
       "      <td>385.953689</td>\n",
       "      <td>424.352897</td>\n",
       "      <td>425.271308</td>\n",
       "      <td>430.594345</td>\n",
       "      <td>377.404491</td>\n",
       "      <td>389.901464</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3651042474</th>\n",
       "      <td>95949.366111</td>\n",
       "      <td>95911.321862</td>\n",
       "      <td>95670.088416</td>\n",
       "      <td>95706.949409</td>\n",
       "      <td>95794.652216</td>\n",
       "      <td>95634.223092</td>\n",
       "      <td>95672.691296</td>\n",
       "      <td>95617.275126</td>\n",
       "      <td>95661.910215</td>\n",
       "      <td>95629.657776</td>\n",
       "      <td>...</td>\n",
       "      <td>95541.272988</td>\n",
       "      <td>95480.310462</td>\n",
       "      <td>95989.024473</td>\n",
       "      <td>95988.666461</td>\n",
       "      <td>95671.605154</td>\n",
       "      <td>95664.516122</td>\n",
       "      <td>95663.626244</td>\n",
       "      <td>95663.519466</td>\n",
       "      <td>95631.759015</td>\n",
       "      <td>95632.765133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3651042508</th>\n",
       "      <td>95941.215857</td>\n",
       "      <td>95903.171607</td>\n",
       "      <td>95661.938161</td>\n",
       "      <td>95698.799154</td>\n",
       "      <td>95786.501962</td>\n",
       "      <td>95626.072838</td>\n",
       "      <td>95664.541042</td>\n",
       "      <td>95609.124871</td>\n",
       "      <td>95653.759960</td>\n",
       "      <td>95621.507521</td>\n",
       "      <td>...</td>\n",
       "      <td>95533.122733</td>\n",
       "      <td>95472.160207</td>\n",
       "      <td>95980.874218</td>\n",
       "      <td>95980.516206</td>\n",
       "      <td>95663.454899</td>\n",
       "      <td>95656.365867</td>\n",
       "      <td>95655.475989</td>\n",
       "      <td>95655.369211</td>\n",
       "      <td>95623.608760</td>\n",
       "      <td>95624.614878</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3651042501</th>\n",
       "      <td>95946.710972</td>\n",
       "      <td>95908.666723</td>\n",
       "      <td>95667.433277</td>\n",
       "      <td>95704.294269</td>\n",
       "      <td>95791.997077</td>\n",
       "      <td>95631.567953</td>\n",
       "      <td>95670.036157</td>\n",
       "      <td>95614.619986</td>\n",
       "      <td>95659.255076</td>\n",
       "      <td>95627.002637</td>\n",
       "      <td>...</td>\n",
       "      <td>95538.617849</td>\n",
       "      <td>95477.655323</td>\n",
       "      <td>95986.369334</td>\n",
       "      <td>95986.011322</td>\n",
       "      <td>95668.950015</td>\n",
       "      <td>95661.860983</td>\n",
       "      <td>95660.971105</td>\n",
       "      <td>95660.864327</td>\n",
       "      <td>95629.103876</td>\n",
       "      <td>95630.109994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3651042393</th>\n",
       "      <td>95933.180670</td>\n",
       "      <td>95895.136420</td>\n",
       "      <td>95653.902975</td>\n",
       "      <td>95690.763967</td>\n",
       "      <td>95778.466775</td>\n",
       "      <td>95618.037651</td>\n",
       "      <td>95656.505855</td>\n",
       "      <td>95601.089684</td>\n",
       "      <td>95645.724773</td>\n",
       "      <td>95613.472335</td>\n",
       "      <td>...</td>\n",
       "      <td>95525.087547</td>\n",
       "      <td>95464.125021</td>\n",
       "      <td>95972.839031</td>\n",
       "      <td>95972.481019</td>\n",
       "      <td>95655.419712</td>\n",
       "      <td>95648.330680</td>\n",
       "      <td>95647.440802</td>\n",
       "      <td>95647.334025</td>\n",
       "      <td>95615.573574</td>\n",
       "      <td>95616.579692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6323434997</th>\n",
       "      <td>95963.039430</td>\n",
       "      <td>95924.995180</td>\n",
       "      <td>95683.761734</td>\n",
       "      <td>95720.622727</td>\n",
       "      <td>95808.325535</td>\n",
       "      <td>95647.896411</td>\n",
       "      <td>95686.364615</td>\n",
       "      <td>95630.948444</td>\n",
       "      <td>95675.583533</td>\n",
       "      <td>95643.331094</td>\n",
       "      <td>...</td>\n",
       "      <td>95554.946306</td>\n",
       "      <td>95493.983780</td>\n",
       "      <td>96002.697791</td>\n",
       "      <td>96002.339779</td>\n",
       "      <td>95685.278472</td>\n",
       "      <td>95678.189440</td>\n",
       "      <td>95677.299562</td>\n",
       "      <td>95677.192784</td>\n",
       "      <td>95645.432333</td>\n",
       "      <td>95646.438451</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625851 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              6058226279    6029307183    4998093094    2201506815  \\\n",
       "3507831609     83.755636    232.445763    411.937336    327.356389   \n",
       "3507831510     90.114132    238.804260    418.295833    333.714886   \n",
       "6188134127     25.237636    173.927763    353.419336    268.838389   \n",
       "8631201421     96.422509    176.244632    424.604209    340.023262   \n",
       "8598305977     53.438771    155.021674    381.620471    297.039524   \n",
       "...                  ...           ...           ...           ...   \n",
       "3651042474  95949.366111  95911.321862  95670.088416  95706.949409   \n",
       "3651042508  95941.215857  95903.171607  95661.938161  95698.799154   \n",
       "3651042501  95946.710972  95908.666723  95667.433277  95704.294269   \n",
       "3651042393  95933.180670  95895.136420  95653.902975  95690.763967   \n",
       "6323434997  95963.039430  95924.995180  95683.761734  95720.622727   \n",
       "\n",
       "              3474499811    1697006012    1901689169    6032060028  \\\n",
       "3507831609    310.503821    444.754763    457.829267    441.751099   \n",
       "3507831510    316.862317    451.113260    464.187763    448.109595   \n",
       "6188134127    251.985821    386.236763    399.311267    383.233099   \n",
       "8631201421    323.170694    457.421636    470.496140    454.417972   \n",
       "8598305977    280.186956    414.437898    427.512402    411.434234   \n",
       "...                  ...           ...           ...           ...   \n",
       "3651042474  95794.652216  95634.223092  95672.691296  95617.275126   \n",
       "3651042508  95786.501962  95626.072838  95664.541042  95609.124871   \n",
       "3651042501  95791.997077  95631.567953  95670.036157  95614.619986   \n",
       "3651042393  95778.466775  95618.037651  95656.505855  95601.089684   \n",
       "6323434997  95808.325535  95647.896411  95686.364615  95630.948444   \n",
       "\n",
       "              6040927878    3449495495  ...    1968458114    1936967272  \\\n",
       "3507831609    375.896565    409.931555  ...    499.066890    552.070025   \n",
       "3507831510    382.255062    416.290051  ...    505.425386    558.428522   \n",
       "6188134127    317.378565    351.413554  ...    440.548890    493.552025   \n",
       "8631201421    388.563438    422.598428  ...    511.733763    564.736898   \n",
       "8598305977    345.579700    379.614690  ...    468.750025    521.753160   \n",
       "...                  ...           ...  ...           ...           ...   \n",
       "3651042474  95661.910215  95629.657776  ...  95541.272988  95480.310462   \n",
       "3651042508  95653.759960  95621.507521  ...  95533.122733  95472.160207   \n",
       "3651042501  95659.255076  95627.002637  ...  95538.617849  95477.655323   \n",
       "3651042393  95645.724773  95613.472335  ...  95525.087547  95464.125021   \n",
       "6323434997  95675.583533  95643.331094  ...  95554.946306  95493.983780   \n",
       "\n",
       "              3496518021    6027615161    6027276892    6041228287  \\\n",
       "3507831609    427.474675    427.291346    416.270554    454.669762   \n",
       "3507831510    433.833171    433.649843    422.629051    461.028259   \n",
       "6188134127    368.956675    368.773346    357.752554    396.151762   \n",
       "8631201421    367.072725    366.889397    428.937427    467.336635   \n",
       "8598305977    350.050585    349.867257    385.953689    424.352897   \n",
       "...                  ...           ...           ...           ...   \n",
       "3651042474  95989.024473  95988.666461  95671.605154  95664.516122   \n",
       "3651042508  95980.874218  95980.516206  95663.454899  95656.365867   \n",
       "3651042501  95986.369334  95986.011322  95668.950015  95661.860983   \n",
       "3651042393  95972.839031  95972.481019  95655.419712  95648.330680   \n",
       "6323434997  96002.697791  96002.339779  95685.278472  95678.189440   \n",
       "\n",
       "              5536661253    7357630367    8178147277    6026834850  \n",
       "3507831609    455.588173    460.911210    407.721356    420.218329  \n",
       "3507831510    461.946669    467.269707    414.079852    426.576826  \n",
       "6188134127    397.070173    402.393210    349.203355    361.700329  \n",
       "8631201421    468.255046    473.578083    420.388229    432.885202  \n",
       "8598305977    425.271308    430.594345    377.404491    389.901464  \n",
       "...                  ...           ...           ...           ...  \n",
       "3651042474  95663.626244  95663.519466  95631.759015  95632.765133  \n",
       "3651042508  95655.475989  95655.369211  95623.608760  95624.614878  \n",
       "3651042501  95660.971105  95660.864327  95629.103876  95630.109994  \n",
       "3651042393  95647.440802  95647.334025  95615.573574  95616.579692  \n",
       "6323434997  95677.299562  95677.192784  95645.432333  95646.438451  \n",
       "\n",
       "[625851 rows x 58 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create origin-specific matrix and save to file.\n",
    "OD_ag = OD_df.loc[list_ag,: ]\n",
    "OD_ag = OD_ag[OD_ag < fail_value] / 60 \n",
    "OD_ag.to_csv(os.path.join(out_pth, 'OD_flood10_ag.csv'))\n",
    "OD_ag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>6058226279</th>\n",
       "      <th>6029307183</th>\n",
       "      <th>4998093094</th>\n",
       "      <th>2201506815</th>\n",
       "      <th>3474499811</th>\n",
       "      <th>1697006012</th>\n",
       "      <th>1901689169</th>\n",
       "      <th>6032060028</th>\n",
       "      <th>6040927878</th>\n",
       "      <th>3449495495</th>\n",
       "      <th>...</th>\n",
       "      <th>1968458114</th>\n",
       "      <th>1936967272</th>\n",
       "      <th>3496518021</th>\n",
       "      <th>6027615161</th>\n",
       "      <th>6027276892</th>\n",
       "      <th>6041228287</th>\n",
       "      <th>5536661253</th>\n",
       "      <th>7357630367</th>\n",
       "      <th>8178147277</th>\n",
       "      <th>6026834850</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>7761872870</th>\n",
       "      <td>62.891073</td>\n",
       "      <td>245.798986</td>\n",
       "      <td>389.498891</td>\n",
       "      <td>304.917944</td>\n",
       "      <td>288.065376</td>\n",
       "      <td>422.316318</td>\n",
       "      <td>435.390822</td>\n",
       "      <td>419.312654</td>\n",
       "      <td>353.458120</td>\n",
       "      <td>387.493110</td>\n",
       "      <td>...</td>\n",
       "      <td>476.628445</td>\n",
       "      <td>529.631580</td>\n",
       "      <td>426.887347</td>\n",
       "      <td>426.704018</td>\n",
       "      <td>393.832109</td>\n",
       "      <td>432.231317</td>\n",
       "      <td>433.149728</td>\n",
       "      <td>438.472765</td>\n",
       "      <td>385.282911</td>\n",
       "      <td>397.779884</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7761872869</th>\n",
       "      <td>62.872513</td>\n",
       "      <td>245.780427</td>\n",
       "      <td>389.480332</td>\n",
       "      <td>304.899385</td>\n",
       "      <td>288.046817</td>\n",
       "      <td>422.297759</td>\n",
       "      <td>435.372262</td>\n",
       "      <td>419.294095</td>\n",
       "      <td>353.439561</td>\n",
       "      <td>387.474550</td>\n",
       "      <td>...</td>\n",
       "      <td>476.609885</td>\n",
       "      <td>529.613021</td>\n",
       "      <td>426.868787</td>\n",
       "      <td>426.685459</td>\n",
       "      <td>393.813550</td>\n",
       "      <td>432.212758</td>\n",
       "      <td>433.131168</td>\n",
       "      <td>438.454206</td>\n",
       "      <td>385.264351</td>\n",
       "      <td>397.761325</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6442044321</th>\n",
       "      <td>61.862681</td>\n",
       "      <td>244.770595</td>\n",
       "      <td>388.470500</td>\n",
       "      <td>303.889553</td>\n",
       "      <td>287.036985</td>\n",
       "      <td>421.287927</td>\n",
       "      <td>434.362430</td>\n",
       "      <td>418.284263</td>\n",
       "      <td>352.429729</td>\n",
       "      <td>386.464718</td>\n",
       "      <td>...</td>\n",
       "      <td>475.600053</td>\n",
       "      <td>528.603189</td>\n",
       "      <td>425.858955</td>\n",
       "      <td>425.675627</td>\n",
       "      <td>392.803718</td>\n",
       "      <td>431.202926</td>\n",
       "      <td>432.121336</td>\n",
       "      <td>437.444374</td>\n",
       "      <td>384.254519</td>\n",
       "      <td>396.751493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2142496418</th>\n",
       "      <td>63.316203</td>\n",
       "      <td>246.224117</td>\n",
       "      <td>389.924022</td>\n",
       "      <td>305.343075</td>\n",
       "      <td>288.490507</td>\n",
       "      <td>422.741449</td>\n",
       "      <td>435.815952</td>\n",
       "      <td>419.737785</td>\n",
       "      <td>353.883251</td>\n",
       "      <td>387.918240</td>\n",
       "      <td>...</td>\n",
       "      <td>477.053575</td>\n",
       "      <td>530.056711</td>\n",
       "      <td>427.312477</td>\n",
       "      <td>427.129149</td>\n",
       "      <td>394.257240</td>\n",
       "      <td>432.656448</td>\n",
       "      <td>433.574858</td>\n",
       "      <td>438.897896</td>\n",
       "      <td>385.708041</td>\n",
       "      <td>398.205015</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2142496429</th>\n",
       "      <td>63.698577</td>\n",
       "      <td>246.606491</td>\n",
       "      <td>390.306396</td>\n",
       "      <td>305.725449</td>\n",
       "      <td>288.872881</td>\n",
       "      <td>423.123823</td>\n",
       "      <td>436.198326</td>\n",
       "      <td>420.120159</td>\n",
       "      <td>354.265625</td>\n",
       "      <td>388.300614</td>\n",
       "      <td>...</td>\n",
       "      <td>477.435949</td>\n",
       "      <td>530.439085</td>\n",
       "      <td>427.694851</td>\n",
       "      <td>427.511523</td>\n",
       "      <td>394.639614</td>\n",
       "      <td>433.038822</td>\n",
       "      <td>433.957232</td>\n",
       "      <td>439.280270</td>\n",
       "      <td>386.090415</td>\n",
       "      <td>398.587389</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9207762346</th>\n",
       "      <td>590.010067</td>\n",
       "      <td>538.743620</td>\n",
       "      <td>392.316477</td>\n",
       "      <td>344.627112</td>\n",
       "      <td>435.296172</td>\n",
       "      <td>372.302310</td>\n",
       "      <td>411.790066</td>\n",
       "      <td>365.139842</td>\n",
       "      <td>297.697339</td>\n",
       "      <td>262.766567</td>\n",
       "      <td>...</td>\n",
       "      <td>284.677977</td>\n",
       "      <td>244.782814</td>\n",
       "      <td>348.428662</td>\n",
       "      <td>348.070650</td>\n",
       "      <td>396.649695</td>\n",
       "      <td>406.848714</td>\n",
       "      <td>406.320451</td>\n",
       "      <td>406.232911</td>\n",
       "      <td>272.057049</td>\n",
       "      <td>265.128454</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8463584916</th>\n",
       "      <td>495.475639</td>\n",
       "      <td>457.431389</td>\n",
       "      <td>221.796126</td>\n",
       "      <td>250.092684</td>\n",
       "      <td>340.761744</td>\n",
       "      <td>185.930802</td>\n",
       "      <td>224.399006</td>\n",
       "      <td>168.982836</td>\n",
       "      <td>203.162911</td>\n",
       "      <td>169.766119</td>\n",
       "      <td>...</td>\n",
       "      <td>91.112914</td>\n",
       "      <td>64.552430</td>\n",
       "      <td>505.146696</td>\n",
       "      <td>504.788684</td>\n",
       "      <td>223.312864</td>\n",
       "      <td>219.457655</td>\n",
       "      <td>218.929391</td>\n",
       "      <td>218.841851</td>\n",
       "      <td>171.867358</td>\n",
       "      <td>172.873476</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8463593882</th>\n",
       "      <td>499.234844</td>\n",
       "      <td>461.190594</td>\n",
       "      <td>225.764336</td>\n",
       "      <td>253.851889</td>\n",
       "      <td>344.520949</td>\n",
       "      <td>189.899012</td>\n",
       "      <td>228.367216</td>\n",
       "      <td>172.951045</td>\n",
       "      <td>206.922116</td>\n",
       "      <td>173.525324</td>\n",
       "      <td>...</td>\n",
       "      <td>94.872119</td>\n",
       "      <td>68.659633</td>\n",
       "      <td>492.534686</td>\n",
       "      <td>492.176674</td>\n",
       "      <td>227.281073</td>\n",
       "      <td>223.425864</td>\n",
       "      <td>222.897601</td>\n",
       "      <td>222.810061</td>\n",
       "      <td>175.626563</td>\n",
       "      <td>176.632681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9208004175</th>\n",
       "      <td>95942.692657</td>\n",
       "      <td>95904.648407</td>\n",
       "      <td>95663.414961</td>\n",
       "      <td>95700.275954</td>\n",
       "      <td>95787.978761</td>\n",
       "      <td>95627.549637</td>\n",
       "      <td>95666.017842</td>\n",
       "      <td>95610.601671</td>\n",
       "      <td>95655.236760</td>\n",
       "      <td>95622.984321</td>\n",
       "      <td>...</td>\n",
       "      <td>95534.599533</td>\n",
       "      <td>95473.637007</td>\n",
       "      <td>95982.351018</td>\n",
       "      <td>95981.993006</td>\n",
       "      <td>95664.931699</td>\n",
       "      <td>95657.842667</td>\n",
       "      <td>95656.952789</td>\n",
       "      <td>95656.846011</td>\n",
       "      <td>95625.085560</td>\n",
       "      <td>95626.091678</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8592243457</th>\n",
       "      <td>584.246592</td>\n",
       "      <td>533.274406</td>\n",
       "      <td>374.700459</td>\n",
       "      <td>338.863637</td>\n",
       "      <td>429.532697</td>\n",
       "      <td>338.835135</td>\n",
       "      <td>377.303339</td>\n",
       "      <td>321.887168</td>\n",
       "      <td>291.933864</td>\n",
       "      <td>257.003092</td>\n",
       "      <td>...</td>\n",
       "      <td>245.767564</td>\n",
       "      <td>186.661697</td>\n",
       "      <td>342.959449</td>\n",
       "      <td>342.601437</td>\n",
       "      <td>376.217196</td>\n",
       "      <td>370.867357</td>\n",
       "      <td>369.977478</td>\n",
       "      <td>369.870701</td>\n",
       "      <td>266.293574</td>\n",
       "      <td>259.364979</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61230 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              6058226279    6029307183    4998093094    2201506815  \\\n",
       "7761872870     62.891073    245.798986    389.498891    304.917944   \n",
       "7761872869     62.872513    245.780427    389.480332    304.899385   \n",
       "6442044321     61.862681    244.770595    388.470500    303.889553   \n",
       "2142496418     63.316203    246.224117    389.924022    305.343075   \n",
       "2142496429     63.698577    246.606491    390.306396    305.725449   \n",
       "...                  ...           ...           ...           ...   \n",
       "9207762346    590.010067    538.743620    392.316477    344.627112   \n",
       "8463584916    495.475639    457.431389    221.796126    250.092684   \n",
       "8463593882    499.234844    461.190594    225.764336    253.851889   \n",
       "9208004175  95942.692657  95904.648407  95663.414961  95700.275954   \n",
       "8592243457    584.246592    533.274406    374.700459    338.863637   \n",
       "\n",
       "              3474499811    1697006012    1901689169    6032060028  \\\n",
       "7761872870    288.065376    422.316318    435.390822    419.312654   \n",
       "7761872869    288.046817    422.297759    435.372262    419.294095   \n",
       "6442044321    287.036985    421.287927    434.362430    418.284263   \n",
       "2142496418    288.490507    422.741449    435.815952    419.737785   \n",
       "2142496429    288.872881    423.123823    436.198326    420.120159   \n",
       "...                  ...           ...           ...           ...   \n",
       "9207762346    435.296172    372.302310    411.790066    365.139842   \n",
       "8463584916    340.761744    185.930802    224.399006    168.982836   \n",
       "8463593882    344.520949    189.899012    228.367216    172.951045   \n",
       "9208004175  95787.978761  95627.549637  95666.017842  95610.601671   \n",
       "8592243457    429.532697    338.835135    377.303339    321.887168   \n",
       "\n",
       "              6040927878    3449495495  ...    1968458114    1936967272  \\\n",
       "7761872870    353.458120    387.493110  ...    476.628445    529.631580   \n",
       "7761872869    353.439561    387.474550  ...    476.609885    529.613021   \n",
       "6442044321    352.429729    386.464718  ...    475.600053    528.603189   \n",
       "2142496418    353.883251    387.918240  ...    477.053575    530.056711   \n",
       "2142496429    354.265625    388.300614  ...    477.435949    530.439085   \n",
       "...                  ...           ...  ...           ...           ...   \n",
       "9207762346    297.697339    262.766567  ...    284.677977    244.782814   \n",
       "8463584916    203.162911    169.766119  ...     91.112914     64.552430   \n",
       "8463593882    206.922116    173.525324  ...     94.872119     68.659633   \n",
       "9208004175  95655.236760  95622.984321  ...  95534.599533  95473.637007   \n",
       "8592243457    291.933864    257.003092  ...    245.767564    186.661697   \n",
       "\n",
       "              3496518021    6027615161    6027276892    6041228287  \\\n",
       "7761872870    426.887347    426.704018    393.832109    432.231317   \n",
       "7761872869    426.868787    426.685459    393.813550    432.212758   \n",
       "6442044321    425.858955    425.675627    392.803718    431.202926   \n",
       "2142496418    427.312477    427.129149    394.257240    432.656448   \n",
       "2142496429    427.694851    427.511523    394.639614    433.038822   \n",
       "...                  ...           ...           ...           ...   \n",
       "9207762346    348.428662    348.070650    396.649695    406.848714   \n",
       "8463584916    505.146696    504.788684    223.312864    219.457655   \n",
       "8463593882    492.534686    492.176674    227.281073    223.425864   \n",
       "9208004175  95982.351018  95981.993006  95664.931699  95657.842667   \n",
       "8592243457    342.959449    342.601437    376.217196    370.867357   \n",
       "\n",
       "              5536661253    7357630367    8178147277    6026834850  \n",
       "7761872870    433.149728    438.472765    385.282911    397.779884  \n",
       "7761872869    433.131168    438.454206    385.264351    397.761325  \n",
       "6442044321    432.121336    437.444374    384.254519    396.751493  \n",
       "2142496418    433.574858    438.897896    385.708041    398.205015  \n",
       "2142496429    433.957232    439.280270    386.090415    398.587389  \n",
       "...                  ...           ...           ...           ...  \n",
       "9207762346    406.320451    406.232911    272.057049    265.128454  \n",
       "8463584916    218.929391    218.841851    171.867358    172.873476  \n",
       "8463593882    222.897601    222.810061    175.626563    176.632681  \n",
       "9208004175  95656.952789  95656.846011  95625.085560  95626.091678  \n",
       "8592243457    369.977478    369.870701    266.293574    259.364979  \n",
       "\n",
       "[61230 rows x 58 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "OD_hamlet = OD_df.loc[list_hamlet,: ]\n",
    "OD_hamlet = OD_hamlet[OD_hamlet < fail_value] / 60 \n",
    "OD_hamlet.to_csv(os.path.join(out_pth, 'OD_flood10_hamlet.csv'))\n",
    "OD_hamlet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Filter 1st nearest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Check each file to make sure nearest neighbor column is named correctly. If not, rename."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reload from file even if already loaded. Quickest way to ensure NN is a column rather than only the index.\n",
    "OD_hamlet = os.path.join(out_pth, \"OD_flood20_hamlet.csv\")\n",
    "OD_hamlet = pd.read_csv(OD_hamlet)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "OD_ag = os.path.join(out_pth, \"OD_flood10_ag.csv\")\n",
    "OD_ag = pd.read_csv(OD_ag)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>6058226279</th>\n",
       "      <th>6029307183</th>\n",
       "      <th>4998093094</th>\n",
       "      <th>2201506815</th>\n",
       "      <th>3474499811</th>\n",
       "      <th>1697006012</th>\n",
       "      <th>1901689169</th>\n",
       "      <th>6032060028</th>\n",
       "      <th>6040927878</th>\n",
       "      <th>...</th>\n",
       "      <th>1968458114</th>\n",
       "      <th>1936967272</th>\n",
       "      <th>3496518021</th>\n",
       "      <th>6027615161</th>\n",
       "      <th>6027276892</th>\n",
       "      <th>6041228287</th>\n",
       "      <th>5536661253</th>\n",
       "      <th>7357630367</th>\n",
       "      <th>8178147277</th>\n",
       "      <th>6026834850</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3507831609</td>\n",
       "      <td>83.755636</td>\n",
       "      <td>232.445763</td>\n",
       "      <td>411.937336</td>\n",
       "      <td>327.356389</td>\n",
       "      <td>310.503821</td>\n",
       "      <td>444.754763</td>\n",
       "      <td>457.829267</td>\n",
       "      <td>441.751099</td>\n",
       "      <td>375.896565</td>\n",
       "      <td>...</td>\n",
       "      <td>499.066890</td>\n",
       "      <td>552.070025</td>\n",
       "      <td>427.474675</td>\n",
       "      <td>427.291346</td>\n",
       "      <td>416.270554</td>\n",
       "      <td>454.669762</td>\n",
       "      <td>455.588173</td>\n",
       "      <td>460.911210</td>\n",
       "      <td>407.721356</td>\n",
       "      <td>420.218329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3507831510</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>238.804260</td>\n",
       "      <td>418.295833</td>\n",
       "      <td>333.714886</td>\n",
       "      <td>316.862317</td>\n",
       "      <td>451.113260</td>\n",
       "      <td>464.187763</td>\n",
       "      <td>448.109595</td>\n",
       "      <td>382.255062</td>\n",
       "      <td>...</td>\n",
       "      <td>505.425386</td>\n",
       "      <td>558.428522</td>\n",
       "      <td>433.833171</td>\n",
       "      <td>433.649843</td>\n",
       "      <td>422.629051</td>\n",
       "      <td>461.028259</td>\n",
       "      <td>461.946669</td>\n",
       "      <td>467.269707</td>\n",
       "      <td>414.079852</td>\n",
       "      <td>426.576826</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6188134127</td>\n",
       "      <td>25.237636</td>\n",
       "      <td>173.927763</td>\n",
       "      <td>353.419336</td>\n",
       "      <td>268.838389</td>\n",
       "      <td>251.985821</td>\n",
       "      <td>386.236763</td>\n",
       "      <td>399.311267</td>\n",
       "      <td>383.233099</td>\n",
       "      <td>317.378565</td>\n",
       "      <td>...</td>\n",
       "      <td>440.548890</td>\n",
       "      <td>493.552025</td>\n",
       "      <td>368.956675</td>\n",
       "      <td>368.773346</td>\n",
       "      <td>357.752554</td>\n",
       "      <td>396.151762</td>\n",
       "      <td>397.070173</td>\n",
       "      <td>402.393210</td>\n",
       "      <td>349.203355</td>\n",
       "      <td>361.700329</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8631201421</td>\n",
       "      <td>96.422509</td>\n",
       "      <td>176.244632</td>\n",
       "      <td>424.604209</td>\n",
       "      <td>340.023262</td>\n",
       "      <td>323.170694</td>\n",
       "      <td>457.421636</td>\n",
       "      <td>470.496140</td>\n",
       "      <td>454.417972</td>\n",
       "      <td>388.563438</td>\n",
       "      <td>...</td>\n",
       "      <td>511.733763</td>\n",
       "      <td>564.736898</td>\n",
       "      <td>367.072725</td>\n",
       "      <td>366.889397</td>\n",
       "      <td>428.937427</td>\n",
       "      <td>467.336635</td>\n",
       "      <td>468.255046</td>\n",
       "      <td>473.578083</td>\n",
       "      <td>420.388229</td>\n",
       "      <td>432.885202</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8598305977</td>\n",
       "      <td>53.438771</td>\n",
       "      <td>155.021674</td>\n",
       "      <td>381.620471</td>\n",
       "      <td>297.039524</td>\n",
       "      <td>280.186956</td>\n",
       "      <td>414.437898</td>\n",
       "      <td>427.512402</td>\n",
       "      <td>411.434234</td>\n",
       "      <td>345.579700</td>\n",
       "      <td>...</td>\n",
       "      <td>468.750025</td>\n",
       "      <td>521.753160</td>\n",
       "      <td>350.050585</td>\n",
       "      <td>349.867257</td>\n",
       "      <td>385.953689</td>\n",
       "      <td>424.352897</td>\n",
       "      <td>425.271308</td>\n",
       "      <td>430.594345</td>\n",
       "      <td>377.404491</td>\n",
       "      <td>389.901464</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625846</th>\n",
       "      <td>3651042474</td>\n",
       "      <td>95949.366111</td>\n",
       "      <td>95911.321862</td>\n",
       "      <td>95670.088416</td>\n",
       "      <td>95706.949409</td>\n",
       "      <td>95794.652216</td>\n",
       "      <td>95634.223092</td>\n",
       "      <td>95672.691296</td>\n",
       "      <td>95617.275126</td>\n",
       "      <td>95661.910215</td>\n",
       "      <td>...</td>\n",
       "      <td>95541.272988</td>\n",
       "      <td>95480.310462</td>\n",
       "      <td>95989.024473</td>\n",
       "      <td>95988.666461</td>\n",
       "      <td>95671.605154</td>\n",
       "      <td>95664.516122</td>\n",
       "      <td>95663.626244</td>\n",
       "      <td>95663.519466</td>\n",
       "      <td>95631.759015</td>\n",
       "      <td>95632.765133</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625847</th>\n",
       "      <td>3651042508</td>\n",
       "      <td>95941.215857</td>\n",
       "      <td>95903.171607</td>\n",
       "      <td>95661.938161</td>\n",
       "      <td>95698.799154</td>\n",
       "      <td>95786.501962</td>\n",
       "      <td>95626.072838</td>\n",
       "      <td>95664.541042</td>\n",
       "      <td>95609.124871</td>\n",
       "      <td>95653.759960</td>\n",
       "      <td>...</td>\n",
       "      <td>95533.122733</td>\n",
       "      <td>95472.160207</td>\n",
       "      <td>95980.874218</td>\n",
       "      <td>95980.516206</td>\n",
       "      <td>95663.454899</td>\n",
       "      <td>95656.365867</td>\n",
       "      <td>95655.475989</td>\n",
       "      <td>95655.369211</td>\n",
       "      <td>95623.608760</td>\n",
       "      <td>95624.614878</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625848</th>\n",
       "      <td>3651042501</td>\n",
       "      <td>95946.710972</td>\n",
       "      <td>95908.666723</td>\n",
       "      <td>95667.433277</td>\n",
       "      <td>95704.294269</td>\n",
       "      <td>95791.997077</td>\n",
       "      <td>95631.567953</td>\n",
       "      <td>95670.036157</td>\n",
       "      <td>95614.619986</td>\n",
       "      <td>95659.255076</td>\n",
       "      <td>...</td>\n",
       "      <td>95538.617849</td>\n",
       "      <td>95477.655323</td>\n",
       "      <td>95986.369334</td>\n",
       "      <td>95986.011322</td>\n",
       "      <td>95668.950015</td>\n",
       "      <td>95661.860983</td>\n",
       "      <td>95660.971105</td>\n",
       "      <td>95660.864327</td>\n",
       "      <td>95629.103876</td>\n",
       "      <td>95630.109994</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625849</th>\n",
       "      <td>3651042393</td>\n",
       "      <td>95933.180670</td>\n",
       "      <td>95895.136420</td>\n",
       "      <td>95653.902975</td>\n",
       "      <td>95690.763967</td>\n",
       "      <td>95778.466775</td>\n",
       "      <td>95618.037651</td>\n",
       "      <td>95656.505855</td>\n",
       "      <td>95601.089684</td>\n",
       "      <td>95645.724773</td>\n",
       "      <td>...</td>\n",
       "      <td>95525.087547</td>\n",
       "      <td>95464.125021</td>\n",
       "      <td>95972.839031</td>\n",
       "      <td>95972.481019</td>\n",
       "      <td>95655.419712</td>\n",
       "      <td>95648.330680</td>\n",
       "      <td>95647.440802</td>\n",
       "      <td>95647.334025</td>\n",
       "      <td>95615.573574</td>\n",
       "      <td>95616.579692</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625850</th>\n",
       "      <td>6323434997</td>\n",
       "      <td>95963.039430</td>\n",
       "      <td>95924.995180</td>\n",
       "      <td>95683.761734</td>\n",
       "      <td>95720.622727</td>\n",
       "      <td>95808.325535</td>\n",
       "      <td>95647.896411</td>\n",
       "      <td>95686.364615</td>\n",
       "      <td>95630.948444</td>\n",
       "      <td>95675.583533</td>\n",
       "      <td>...</td>\n",
       "      <td>95554.946306</td>\n",
       "      <td>95493.983780</td>\n",
       "      <td>96002.697791</td>\n",
       "      <td>96002.339779</td>\n",
       "      <td>95685.278472</td>\n",
       "      <td>95678.189440</td>\n",
       "      <td>95677.299562</td>\n",
       "      <td>95677.192784</td>\n",
       "      <td>95645.432333</td>\n",
       "      <td>95646.438451</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625851 rows × 59 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                NN    6058226279    6029307183    4998093094    2201506815  \\\n",
       "0       3507831609     83.755636    232.445763    411.937336    327.356389   \n",
       "1       3507831510     90.114132    238.804260    418.295833    333.714886   \n",
       "2       6188134127     25.237636    173.927763    353.419336    268.838389   \n",
       "3       8631201421     96.422509    176.244632    424.604209    340.023262   \n",
       "4       8598305977     53.438771    155.021674    381.620471    297.039524   \n",
       "...            ...           ...           ...           ...           ...   \n",
       "625846  3651042474  95949.366111  95911.321862  95670.088416  95706.949409   \n",
       "625847  3651042508  95941.215857  95903.171607  95661.938161  95698.799154   \n",
       "625848  3651042501  95946.710972  95908.666723  95667.433277  95704.294269   \n",
       "625849  3651042393  95933.180670  95895.136420  95653.902975  95690.763967   \n",
       "625850  6323434997  95963.039430  95924.995180  95683.761734  95720.622727   \n",
       "\n",
       "          3474499811    1697006012    1901689169    6032060028    6040927878  \\\n",
       "0         310.503821    444.754763    457.829267    441.751099    375.896565   \n",
       "1         316.862317    451.113260    464.187763    448.109595    382.255062   \n",
       "2         251.985821    386.236763    399.311267    383.233099    317.378565   \n",
       "3         323.170694    457.421636    470.496140    454.417972    388.563438   \n",
       "4         280.186956    414.437898    427.512402    411.434234    345.579700   \n",
       "...              ...           ...           ...           ...           ...   \n",
       "625846  95794.652216  95634.223092  95672.691296  95617.275126  95661.910215   \n",
       "625847  95786.501962  95626.072838  95664.541042  95609.124871  95653.759960   \n",
       "625848  95791.997077  95631.567953  95670.036157  95614.619986  95659.255076   \n",
       "625849  95778.466775  95618.037651  95656.505855  95601.089684  95645.724773   \n",
       "625850  95808.325535  95647.896411  95686.364615  95630.948444  95675.583533   \n",
       "\n",
       "        ...    1968458114    1936967272    3496518021    6027615161  \\\n",
       "0       ...    499.066890    552.070025    427.474675    427.291346   \n",
       "1       ...    505.425386    558.428522    433.833171    433.649843   \n",
       "2       ...    440.548890    493.552025    368.956675    368.773346   \n",
       "3       ...    511.733763    564.736898    367.072725    366.889397   \n",
       "4       ...    468.750025    521.753160    350.050585    349.867257   \n",
       "...     ...           ...           ...           ...           ...   \n",
       "625846  ...  95541.272988  95480.310462  95989.024473  95988.666461   \n",
       "625847  ...  95533.122733  95472.160207  95980.874218  95980.516206   \n",
       "625848  ...  95538.617849  95477.655323  95986.369334  95986.011322   \n",
       "625849  ...  95525.087547  95464.125021  95972.839031  95972.481019   \n",
       "625850  ...  95554.946306  95493.983780  96002.697791  96002.339779   \n",
       "\n",
       "          6027276892    6041228287    5536661253    7357630367    8178147277  \\\n",
       "0         416.270554    454.669762    455.588173    460.911210    407.721356   \n",
       "1         422.629051    461.028259    461.946669    467.269707    414.079852   \n",
       "2         357.752554    396.151762    397.070173    402.393210    349.203355   \n",
       "3         428.937427    467.336635    468.255046    473.578083    420.388229   \n",
       "4         385.953689    424.352897    425.271308    430.594345    377.404491   \n",
       "...              ...           ...           ...           ...           ...   \n",
       "625846  95671.605154  95664.516122  95663.626244  95663.519466  95631.759015   \n",
       "625847  95663.454899  95656.365867  95655.475989  95655.369211  95623.608760   \n",
       "625848  95668.950015  95661.860983  95660.971105  95660.864327  95629.103876   \n",
       "625849  95655.419712  95648.330680  95647.440802  95647.334025  95615.573574   \n",
       "625850  95685.278472  95678.189440  95677.299562  95677.192784  95645.432333   \n",
       "\n",
       "          6026834850  \n",
       "0         420.218329  \n",
       "1         426.576826  \n",
       "2         361.700329  \n",
       "3         432.885202  \n",
       "4         389.901464  \n",
       "...              ...  \n",
       "625846  95632.765133  \n",
       "625847  95624.614878  \n",
       "625848  95630.109994  \n",
       "625849  95616.579692  \n",
       "625850  95646.438451  \n",
       "\n",
       "[625851 rows x 59 columns]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "OD_ag.rename(columns={'Unnamed: 0': 'NN'}, inplace=True) \n",
    "OD_ag"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>6058226279</th>\n",
       "      <th>6029307183</th>\n",
       "      <th>4998093094</th>\n",
       "      <th>2201506815</th>\n",
       "      <th>3474499811</th>\n",
       "      <th>1697006012</th>\n",
       "      <th>1901689169</th>\n",
       "      <th>6032060028</th>\n",
       "      <th>6040927878</th>\n",
       "      <th>...</th>\n",
       "      <th>1968458114</th>\n",
       "      <th>1936967272</th>\n",
       "      <th>3496518021</th>\n",
       "      <th>6027615161</th>\n",
       "      <th>6027276892</th>\n",
       "      <th>6041228287</th>\n",
       "      <th>5536661253</th>\n",
       "      <th>7357630367</th>\n",
       "      <th>8178147277</th>\n",
       "      <th>6026834850</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7761872870</td>\n",
       "      <td>63.403221</td>\n",
       "      <td>874.359225</td>\n",
       "      <td>829.335161</td>\n",
       "      <td>708.179991</td>\n",
       "      <td>291.019814</td>\n",
       "      <td>792.126517</td>\n",
       "      <td>831.614273</td>\n",
       "      <td>788.395031</td>\n",
       "      <td>707.248449</td>\n",
       "      <td>...</td>\n",
       "      <td>706.998875</td>\n",
       "      <td>763.004594</td>\n",
       "      <td>428.940834</td>\n",
       "      <td>428.757505</td>\n",
       "      <td>830.851899</td>\n",
       "      <td>826.672921</td>\n",
       "      <td>826.144657</td>\n",
       "      <td>826.057118</td>\n",
       "      <td>686.470963</td>\n",
       "      <td>685.749430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7761872869</td>\n",
       "      <td>63.384661</td>\n",
       "      <td>874.340665</td>\n",
       "      <td>829.316602</td>\n",
       "      <td>708.161432</td>\n",
       "      <td>291.001255</td>\n",
       "      <td>792.107957</td>\n",
       "      <td>831.595713</td>\n",
       "      <td>788.376471</td>\n",
       "      <td>707.229889</td>\n",
       "      <td>...</td>\n",
       "      <td>706.980315</td>\n",
       "      <td>762.986034</td>\n",
       "      <td>428.922274</td>\n",
       "      <td>428.738946</td>\n",
       "      <td>830.833339</td>\n",
       "      <td>826.654362</td>\n",
       "      <td>826.126098</td>\n",
       "      <td>826.038558</td>\n",
       "      <td>686.452404</td>\n",
       "      <td>685.730871</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6442044321</td>\n",
       "      <td>62.374829</td>\n",
       "      <td>873.330833</td>\n",
       "      <td>828.306770</td>\n",
       "      <td>707.151600</td>\n",
       "      <td>289.991423</td>\n",
       "      <td>791.098125</td>\n",
       "      <td>830.585881</td>\n",
       "      <td>787.366639</td>\n",
       "      <td>706.220057</td>\n",
       "      <td>...</td>\n",
       "      <td>705.970483</td>\n",
       "      <td>761.976202</td>\n",
       "      <td>427.912442</td>\n",
       "      <td>427.729114</td>\n",
       "      <td>829.823507</td>\n",
       "      <td>825.644530</td>\n",
       "      <td>825.116266</td>\n",
       "      <td>825.028726</td>\n",
       "      <td>685.442572</td>\n",
       "      <td>684.721039</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2142496418</td>\n",
       "      <td>63.828351</td>\n",
       "      <td>874.784355</td>\n",
       "      <td>829.760292</td>\n",
       "      <td>708.605122</td>\n",
       "      <td>291.444945</td>\n",
       "      <td>792.551647</td>\n",
       "      <td>832.039403</td>\n",
       "      <td>788.820161</td>\n",
       "      <td>707.673579</td>\n",
       "      <td>...</td>\n",
       "      <td>707.424005</td>\n",
       "      <td>763.429724</td>\n",
       "      <td>429.365964</td>\n",
       "      <td>429.182636</td>\n",
       "      <td>831.277029</td>\n",
       "      <td>827.098052</td>\n",
       "      <td>826.569788</td>\n",
       "      <td>826.482248</td>\n",
       "      <td>686.896094</td>\n",
       "      <td>686.174561</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2142496429</td>\n",
       "      <td>64.210725</td>\n",
       "      <td>875.166729</td>\n",
       "      <td>830.142666</td>\n",
       "      <td>708.987496</td>\n",
       "      <td>291.827319</td>\n",
       "      <td>792.934021</td>\n",
       "      <td>832.421777</td>\n",
       "      <td>789.202535</td>\n",
       "      <td>708.055953</td>\n",
       "      <td>...</td>\n",
       "      <td>707.806379</td>\n",
       "      <td>763.812098</td>\n",
       "      <td>429.748338</td>\n",
       "      <td>429.565010</td>\n",
       "      <td>831.659403</td>\n",
       "      <td>827.480426</td>\n",
       "      <td>826.952162</td>\n",
       "      <td>826.864622</td>\n",
       "      <td>687.278468</td>\n",
       "      <td>686.556935</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61225</th>\n",
       "      <td>9207762346</td>\n",
       "      <td>759.815291</td>\n",
       "      <td>1270.957620</td>\n",
       "      <td>441.918070</td>\n",
       "      <td>366.535448</td>\n",
       "      <td>671.915052</td>\n",
       "      <td>404.709426</td>\n",
       "      <td>444.197182</td>\n",
       "      <td>391.234121</td>\n",
       "      <td>319.831358</td>\n",
       "      <td>...</td>\n",
       "      <td>305.620498</td>\n",
       "      <td>249.148226</td>\n",
       "      <td>398.206819</td>\n",
       "      <td>398.023490</td>\n",
       "      <td>443.434808</td>\n",
       "      <td>433.572803</td>\n",
       "      <td>432.682925</td>\n",
       "      <td>432.576147</td>\n",
       "      <td>293.401199</td>\n",
       "      <td>286.416565</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61226</th>\n",
       "      <td>8463584916</td>\n",
       "      <td>719.145603</td>\n",
       "      <td>1428.350095</td>\n",
       "      <td>230.836629</td>\n",
       "      <td>251.844536</td>\n",
       "      <td>595.818867</td>\n",
       "      <td>194.647536</td>\n",
       "      <td>233.115741</td>\n",
       "      <td>177.606300</td>\n",
       "      <td>205.140446</td>\n",
       "      <td>...</td>\n",
       "      <td>92.201681</td>\n",
       "      <td>64.552430</td>\n",
       "      <td>555.599294</td>\n",
       "      <td>555.415966</td>\n",
       "      <td>232.353367</td>\n",
       "      <td>228.174389</td>\n",
       "      <td>227.646125</td>\n",
       "      <td>227.558585</td>\n",
       "      <td>173.050546</td>\n",
       "      <td>174.056664</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61227</th>\n",
       "      <td>8463593882</td>\n",
       "      <td>722.904808</td>\n",
       "      <td>1415.738086</td>\n",
       "      <td>234.804839</td>\n",
       "      <td>255.603741</td>\n",
       "      <td>599.578072</td>\n",
       "      <td>198.615746</td>\n",
       "      <td>237.083950</td>\n",
       "      <td>181.574510</td>\n",
       "      <td>208.899651</td>\n",
       "      <td>...</td>\n",
       "      <td>95.960887</td>\n",
       "      <td>68.659633</td>\n",
       "      <td>542.987285</td>\n",
       "      <td>542.803956</td>\n",
       "      <td>236.321576</td>\n",
       "      <td>232.142599</td>\n",
       "      <td>231.614335</td>\n",
       "      <td>231.526795</td>\n",
       "      <td>176.809751</td>\n",
       "      <td>177.815869</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61228</th>\n",
       "      <td>9208004175</td>\n",
       "      <td>163905.632985</td>\n",
       "      <td>164639.432529</td>\n",
       "      <td>163399.717166</td>\n",
       "      <td>163435.116360</td>\n",
       "      <td>163782.306249</td>\n",
       "      <td>163363.528073</td>\n",
       "      <td>163399.830202</td>\n",
       "      <td>163346.486837</td>\n",
       "      <td>163388.412269</td>\n",
       "      <td>...</td>\n",
       "      <td>163265.217738</td>\n",
       "      <td>163203.751207</td>\n",
       "      <td>163766.681729</td>\n",
       "      <td>163766.498400</td>\n",
       "      <td>163401.233904</td>\n",
       "      <td>163388.175784</td>\n",
       "      <td>163387.285906</td>\n",
       "      <td>163387.179128</td>\n",
       "      <td>163356.322369</td>\n",
       "      <td>163357.328487</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>61229</th>\n",
       "      <td>8592243457</td>\n",
       "      <td>754.346078</td>\n",
       "      <td>1265.488406</td>\n",
       "      <td>386.343333</td>\n",
       "      <td>362.651877</td>\n",
       "      <td>668.031481</td>\n",
       "      <td>350.154240</td>\n",
       "      <td>387.106104</td>\n",
       "      <td>333.113004</td>\n",
       "      <td>315.947787</td>\n",
       "      <td>...</td>\n",
       "      <td>247.499381</td>\n",
       "      <td>191.027109</td>\n",
       "      <td>392.737605</td>\n",
       "      <td>392.554277</td>\n",
       "      <td>387.860071</td>\n",
       "      <td>375.451686</td>\n",
       "      <td>374.561808</td>\n",
       "      <td>374.455030</td>\n",
       "      <td>289.517628</td>\n",
       "      <td>282.532993</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61230 rows × 59 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               NN     6058226279     6029307183     4998093094     2201506815  \\\n",
       "0      7761872870      63.403221     874.359225     829.335161     708.179991   \n",
       "1      7761872869      63.384661     874.340665     829.316602     708.161432   \n",
       "2      6442044321      62.374829     873.330833     828.306770     707.151600   \n",
       "3      2142496418      63.828351     874.784355     829.760292     708.605122   \n",
       "4      2142496429      64.210725     875.166729     830.142666     708.987496   \n",
       "...           ...            ...            ...            ...            ...   \n",
       "61225  9207762346     759.815291    1270.957620     441.918070     366.535448   \n",
       "61226  8463584916     719.145603    1428.350095     230.836629     251.844536   \n",
       "61227  8463593882     722.904808    1415.738086     234.804839     255.603741   \n",
       "61228  9208004175  163905.632985  164639.432529  163399.717166  163435.116360   \n",
       "61229  8592243457     754.346078    1265.488406     386.343333     362.651877   \n",
       "\n",
       "          3474499811     1697006012     1901689169     6032060028  \\\n",
       "0         291.019814     792.126517     831.614273     788.395031   \n",
       "1         291.001255     792.107957     831.595713     788.376471   \n",
       "2         289.991423     791.098125     830.585881     787.366639   \n",
       "3         291.444945     792.551647     832.039403     788.820161   \n",
       "4         291.827319     792.934021     832.421777     789.202535   \n",
       "...              ...            ...            ...            ...   \n",
       "61225     671.915052     404.709426     444.197182     391.234121   \n",
       "61226     595.818867     194.647536     233.115741     177.606300   \n",
       "61227     599.578072     198.615746     237.083950     181.574510   \n",
       "61228  163782.306249  163363.528073  163399.830202  163346.486837   \n",
       "61229     668.031481     350.154240     387.106104     333.113004   \n",
       "\n",
       "          6040927878  ...     1968458114     1936967272     3496518021  \\\n",
       "0         707.248449  ...     706.998875     763.004594     428.940834   \n",
       "1         707.229889  ...     706.980315     762.986034     428.922274   \n",
       "2         706.220057  ...     705.970483     761.976202     427.912442   \n",
       "3         707.673579  ...     707.424005     763.429724     429.365964   \n",
       "4         708.055953  ...     707.806379     763.812098     429.748338   \n",
       "...              ...  ...            ...            ...            ...   \n",
       "61225     319.831358  ...     305.620498     249.148226     398.206819   \n",
       "61226     205.140446  ...      92.201681      64.552430     555.599294   \n",
       "61227     208.899651  ...      95.960887      68.659633     542.987285   \n",
       "61228  163388.412269  ...  163265.217738  163203.751207  163766.681729   \n",
       "61229     315.947787  ...     247.499381     191.027109     392.737605   \n",
       "\n",
       "          6027615161     6027276892     6041228287     5536661253  \\\n",
       "0         428.757505     830.851899     826.672921     826.144657   \n",
       "1         428.738946     830.833339     826.654362     826.126098   \n",
       "2         427.729114     829.823507     825.644530     825.116266   \n",
       "3         429.182636     831.277029     827.098052     826.569788   \n",
       "4         429.565010     831.659403     827.480426     826.952162   \n",
       "...              ...            ...            ...            ...   \n",
       "61225     398.023490     443.434808     433.572803     432.682925   \n",
       "61226     555.415966     232.353367     228.174389     227.646125   \n",
       "61227     542.803956     236.321576     232.142599     231.614335   \n",
       "61228  163766.498400  163401.233904  163388.175784  163387.285906   \n",
       "61229     392.554277     387.860071     375.451686     374.561808   \n",
       "\n",
       "          7357630367     8178147277     6026834850  \n",
       "0         826.057118     686.470963     685.749430  \n",
       "1         826.038558     686.452404     685.730871  \n",
       "2         825.028726     685.442572     684.721039  \n",
       "3         826.482248     686.896094     686.174561  \n",
       "4         826.864622     687.278468     686.556935  \n",
       "...              ...            ...            ...  \n",
       "61225     432.576147     293.401199     286.416565  \n",
       "61226     227.558585     173.050546     174.056664  \n",
       "61227     231.526795     176.809751     177.815869  \n",
       "61228  163387.179128  163356.322369  163357.328487  \n",
       "61229     374.455030     289.517628     282.532993  \n",
       "\n",
       "[61230 rows x 59 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "OD_hamlet.rename(columns={'Unnamed: 0': 'NN'}, inplace=True) \n",
    "OD_hamlet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Find first, second, and third nearest destination for each origin node. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "fail_value = 999999999"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>ag_HD1F10</th>\n",
       "      <th>ag_HD2F10</th>\n",
       "      <th>ag_HD3F10</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3507831609</td>\n",
       "      <td>82.054823</td>\n",
       "      <td>83.755636</td>\n",
       "      <td>232.445763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3507831510</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>238.804260</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6188134127</td>\n",
       "      <td>23.536822</td>\n",
       "      <td>25.237636</td>\n",
       "      <td>173.927763</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8631201421</td>\n",
       "      <td>94.721696</td>\n",
       "      <td>96.422509</td>\n",
       "      <td>176.244632</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8598305977</td>\n",
       "      <td>51.737958</td>\n",
       "      <td>53.438771</td>\n",
       "      <td>155.021674</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           NN  ag_HD1F10  ag_HD2F10   ag_HD3F10\n",
       "0  3507831609  82.054823  83.755636  232.445763\n",
       "1  3507831510  88.413319  90.114132  238.804260\n",
       "2  6188134127  23.536822  25.237636  173.927763\n",
       "3  8631201421  94.721696  96.422509  176.244632\n",
       "4  8598305977  51.737958  53.438771  155.021674"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Nearest\n",
    "OD_ag[\"ag_HD1F10\"] = 0\n",
    "sub = OD_ag.iloc[:,1:-1] # Filtering out the newly created field and the node ID column. (\"include everything between column 0 and the last column\")\n",
    "OD_ag[\"ag_HD1F10\"] = sub.min(axis=1) # Default is axis=0, meaning min value of each column selected. We want min of each row.\n",
    "ag1 = OD_ag[['NN', 'ag_HD1F10']] # Remove unnecessary OD values.\n",
    "\n",
    "\n",
    "# Second nearest\n",
    "dupes = OD_ag.apply(pd.Series.duplicated, axis = 1, keep=False) # If a number is repeated within a row, value is True. If not, False.\n",
    "# The first time this is done, there should be two True values per row, unless any POIs are equidistant.\n",
    "dupes = OD_ag.where(~dupes, fail_value) # For any value that appears more than once in its row, it is replaced with the fail_value.\n",
    "OD_ag[\"ag_HD2F10\"] = 0\n",
    "Dsub = dupes.iloc[:,1:] # Filtering out the node ID column. No need to filter 1st nearest as its new \"dupes\" value is too high to be caught.\n",
    "OD_ag[\"ag_HD2F10\"] = Dsub.min(axis=1) \n",
    "ag2 = OD_ag.loc[:,['NN', 'ag_HD2F10']] \n",
    "\n",
    "\n",
    "# Third nearest\n",
    "dupes = OD_ag.apply(pd.Series.duplicated, axis = 1, keep=False)\n",
    "# Since this includes both first and second nearest columns, there should be four True values per row, unless POIs are equidistant.\n",
    "dupes = OD_ag.where(~dupes, fail_value)\n",
    "\n",
    "OD_ag[\"ag_HD3F10\"] = 0\n",
    "Dsub = dupes.iloc[:,1:] # Filtering out the node ID column.\n",
    "OD_ag[\"ag_HD3F10\"] = Dsub.min(axis=1)\n",
    "ag3 = OD_ag.loc[:,['NN', 'ag_HD3F10']]\n",
    "\n",
    "# Combine and write to file\n",
    "ag_all = OD_ag.loc[:,['NN', 'ag_HD1F10', 'ag_HD2F10', 'ag_HD3F10']]\n",
    "ag_all.to_csv(os.path.join(out_pth, 'ag_to_HDurban_flood10.csv'))\n",
    "ag_all.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>ha_HD1F20</th>\n",
       "      <th>ha_HD2F20</th>\n",
       "      <th>ha_HD3F20</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>7761872870</td>\n",
       "      <td>63.403221</td>\n",
       "      <td>66.771529</td>\n",
       "      <td>291.019814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>7761872869</td>\n",
       "      <td>63.384661</td>\n",
       "      <td>66.752970</td>\n",
       "      <td>291.001255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6442044321</td>\n",
       "      <td>62.374829</td>\n",
       "      <td>65.743138</td>\n",
       "      <td>289.991423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>2142496418</td>\n",
       "      <td>63.828351</td>\n",
       "      <td>67.196660</td>\n",
       "      <td>291.444945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>2142496429</td>\n",
       "      <td>64.210725</td>\n",
       "      <td>67.579034</td>\n",
       "      <td>291.827319</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           NN  ha_HD1F20  ha_HD2F20   ha_HD3F20\n",
       "0  7761872870  63.403221  66.771529  291.019814\n",
       "1  7761872869  63.384661  66.752970  291.001255\n",
       "2  6442044321  62.374829  65.743138  289.991423\n",
       "3  2142496418  63.828351  67.196660  291.444945\n",
       "4  2142496429  64.210725  67.579034  291.827319"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Nearest\n",
    "OD_hamlet[\"ha_HD1F20\"] = 0\n",
    "sub = OD_hamlet.iloc[:,1:-1] # Filtering out the newly created field and the node ID column. (\"include everything between column 0 and the last column\")\n",
    "OD_hamlet[\"ha_HD1F20\"] = sub.min(axis=1) # Default is axis=0, meaning min value of each column selected. We want min of each row.\n",
    "hamlet1 = OD_hamlet[['NN', 'ha_HD1F20']] # Remove unnecessary OD values.\n",
    "\n",
    "\n",
    "# Second nearest\n",
    "dupes = OD_hamlet.apply(pd.Series.duplicated, axis = 1, keep=False) # If a number is repeated within a row, value is True. If not, False.\n",
    "# The first time this is done, there should be two True values per row, unless any POIs are equidistant.\n",
    "dupes = OD_hamlet.where(~dupes, fail_value) # For any value that appears more than once in its row, it is replaced with the fail_value.\n",
    "OD_hamlet[\"ha_HD2F20\"] = 0\n",
    "Dsub = dupes.iloc[:,1:] # Filtering out the node ID column. No need to filter 1st nearest as its new \"dupes\" value is too high to be caught.\n",
    "OD_hamlet[\"ha_HD2F20\"] = Dsub.min(axis=1) \n",
    "hamlet2 = OD_hamlet.loc[:,['NN', 'ha_HD2F20']] \n",
    "\n",
    "\n",
    "# Third nearest\n",
    "dupes = OD_hamlet.apply(pd.Series.duplicated, axis = 1, keep=False)\n",
    "# Since this includes both first and second nearest columns, there should be four True values per row, unless POIs are equidistant.\n",
    "dupes = OD_hamlet.where(~dupes, fail_value)\n",
    "OD_hamlet[\"ha_HD3F20\"] = 0\n",
    "Dsub = dupes.iloc[:,1:] # Filtering out the node ID column.\n",
    "OD_hamlet[\"ha_HD3F20\"] = Dsub.min(axis=1)\n",
    "hamlet3 = OD_hamlet.loc[:,['NN', 'ha_HD3F20']]\n",
    "\n",
    "\n",
    "# Combine and write to file\n",
    "hamlet_all = OD_hamlet.loc[:,['NN', 'ha_HD1F20', 'ha_HD2F20', 'ha_HD3F20']]\n",
    "hamlet_all.to_csv(os.path.join(out_pth, 'hamlet_to_HDurban_flood20.csv'))\n",
    "hamlet_all.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Join back to georeferenced _snap file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 541461 entries, 0 to 541460\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   Unnamed: 0  541461 non-null  int64  \n",
      " 1   NN          541461 non-null  int64  \n",
      " 2   ag_HD1F10   541453 non-null  float64\n",
      " 3   ag_HD2F10   541461 non-null  float64\n",
      " 4   ag_HD3F10   541461 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 20.7 MB\n"
     ]
    }
   ],
   "source": [
    "# If starting new session, reload from file.\n",
    "ag_all = os.path.join(out_pth, \"ag_to_HDurban_flood20.csv\")\n",
    "ag_all = pd.read_csv(ag_all)\n",
    "ag_all.info() # Check to make sure NN data type matches its corresponding _snap file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet_all = os.path.join(out_pth, \"hamlet_to_HDurban_flood20.csv\")\n",
    "hamlet_all = pd.read_csv(hamlet_all)\n",
    "hamlet_all.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 6852701 entries, 0 to 6852700\n",
      "Data columns (total 6 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID          int64  \n",
      " 2   LC_90m      int64  \n",
      " 3   geometry    object \n",
      " 4   NN          int64  \n",
      " 5   NN_dist     float64\n",
      "dtypes: float64(1), int64(4), object(1)\n",
      "memory usage: 313.7+ MB\n"
     ]
    }
   ],
   "source": [
    "ag_snap = os.path.join(out_pth, \"ag_snap.csv\")\n",
    "ag_snap = pd.read_csv(ag_snap)\n",
    "ag_snap.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet_snap = os.path.join(out_pth, \"hamlet_snap.csv\")\n",
    "hamlet_snap = pd.read_csv(hamlet_snap)\n",
    "hamlet_snap.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>ID_spam</th>\n",
       "      <th>grid_val</th>\n",
       "      <th>ID_LC</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>geometry</th>\n",
       "      <th>NN</th>\n",
       "      <th>NN_dist</th>\n",
       "      <th>ag_HD1F20</th>\n",
       "      <th>ag_HD2F20</th>\n",
       "      <th>ag_HD3F20</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>13941103</td>\n",
       "      <td>2344.0</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>-16.458988</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.458987999999977 12.208646000000044)</td>\n",
       "      <td>3507831609</td>\n",
       "      <td>397.198533</td>\n",
       "      <td>82.054823</td>\n",
       "      <td>83.755636</td>\n",
       "      <td>312.946111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13941122</td>\n",
       "      <td>2345.0</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>-16.375655</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.375654999999938 12.208646000000044)</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>2881.578460</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>319.304608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>13941017</td>\n",
       "      <td>2346.0</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>-16.208989</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.208988999999974 12.208646000000044)</td>\n",
       "      <td>6188134127</td>\n",
       "      <td>17711.673781</td>\n",
       "      <td>23.536822</td>\n",
       "      <td>25.237636</td>\n",
       "      <td>254.428111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>13941036</td>\n",
       "      <td>2402.0</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>-16.208492</td>\n",
       "      <td>12.152815</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.20849199999998 12.152815000000032)</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>21562.215777</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>319.304608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>13941044</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>-16.294213</td>\n",
       "      <td>12.149487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.294212999999957 12.149487000000022)</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>12874.077085</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>319.304608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941119</th>\n",
       "      <td>13941119</td>\n",
       "      <td>13941081</td>\n",
       "      <td>3.0</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>-15.291536</td>\n",
       "      <td>16.874047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.291535999999951 16.874047000000076)</td>\n",
       "      <td>3651042474</td>\n",
       "      <td>2658.421843</td>\n",
       "      <td>139785.574025</td>\n",
       "      <td>139786.528309</td>\n",
       "      <td>139792.211856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941120</th>\n",
       "      <td>13941120</td>\n",
       "      <td>13941093</td>\n",
       "      <td>6.0</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>-15.292326</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.292325999999946 16.79196100000007)</td>\n",
       "      <td>3651042508</td>\n",
       "      <td>1234.146014</td>\n",
       "      <td>139777.423770</td>\n",
       "      <td>139778.378054</td>\n",
       "      <td>139784.061601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941121</th>\n",
       "      <td>13941121</td>\n",
       "      <td>13941109</td>\n",
       "      <td>2.0</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>-15.374255</td>\n",
       "      <td>16.867753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.374254999999948 16.86775300000005)</td>\n",
       "      <td>3651042501</td>\n",
       "      <td>9172.299324</td>\n",
       "      <td>139782.918886</td>\n",
       "      <td>139783.873169</td>\n",
       "      <td>139789.556716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941122</th>\n",
       "      <td>13941122</td>\n",
       "      <td>13941119</td>\n",
       "      <td>5.0</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>-15.375659</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.375658999999928 16.79196100000007)</td>\n",
       "      <td>3651042393</td>\n",
       "      <td>4280.425149</td>\n",
       "      <td>139769.388584</td>\n",
       "      <td>139770.342867</td>\n",
       "      <td>139776.026414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941123</th>\n",
       "      <td>13941123</td>\n",
       "      <td>13941114</td>\n",
       "      <td>7.0</td>\n",
       "      <td>901.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>901.0</td>\n",
       "      <td>-14.875661</td>\n",
       "      <td>16.785873</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-14.87566099999998 16.785873000000038)</td>\n",
       "      <td>6323434997</td>\n",
       "      <td>2922.119100</td>\n",
       "      <td>158123.044333</td>\n",
       "      <td>158123.998616</td>\n",
       "      <td>158129.682163</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>13941124 rows × 15 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Unnamed: 0     ID_ag  ID_spam   grid_val  ID_LC        val  \\\n",
       "0                  0  13941103   2344.0  2008851.0    0.1  2008851.0   \n",
       "1                  1  13941122   2345.0    94605.0    0.1    94605.0   \n",
       "2                  2  13941017   2346.0   708413.0    0.1   708413.0   \n",
       "3                  3  13941036   2402.0   315719.0    0.1   315719.0   \n",
       "4                  4  13941044   2401.0   375288.0    0.1   375288.0   \n",
       "...              ...       ...      ...        ...    ...        ...   \n",
       "13941119    13941119  13941081      3.0    87412.0    0.1    87412.0   \n",
       "13941120    13941120  13941093      6.0    21660.0    0.1    21660.0   \n",
       "13941121    13941121  13941109      2.0    70993.0    0.1    70993.0   \n",
       "13941122    13941122  13941119      5.0    54822.0    0.1    54822.0   \n",
       "13941123    13941123  13941114      7.0      901.0    0.1      901.0   \n",
       "\n",
       "                  x          y  NSnomax  \\\n",
       "0        -16.458988  12.208646      NaN   \n",
       "1        -16.375655  12.208646      NaN   \n",
       "2        -16.208989  12.208646      NaN   \n",
       "3        -16.208492  12.152815      NaN   \n",
       "4        -16.294213  12.149487      NaN   \n",
       "...             ...        ...      ...   \n",
       "13941119 -15.291536  16.874047      NaN   \n",
       "13941120 -15.292326  16.791961      NaN   \n",
       "13941121 -15.374255  16.867753      NaN   \n",
       "13941122 -15.375659  16.791961      NaN   \n",
       "13941123 -14.875661  16.785873      NaN   \n",
       "\n",
       "                                                geometry          NN  \\\n",
       "0         POINT (-16.458987999999977 12.208646000000044)  3507831609   \n",
       "1         POINT (-16.375654999999938 12.208646000000044)  3507831510   \n",
       "2         POINT (-16.208988999999974 12.208646000000044)  6188134127   \n",
       "3          POINT (-16.20849199999998 12.152815000000032)  3507831510   \n",
       "4         POINT (-16.294212999999957 12.149487000000022)  3507831510   \n",
       "...                                                  ...         ...   \n",
       "13941119  POINT (-15.291535999999951 16.874047000000076)  3651042474   \n",
       "13941120   POINT (-15.292325999999946 16.79196100000007)  3651042508   \n",
       "13941121   POINT (-15.374254999999948 16.86775300000005)  3651042501   \n",
       "13941122   POINT (-15.375658999999928 16.79196100000007)  3651042393   \n",
       "13941123   POINT (-14.87566099999998 16.785873000000038)  6323434997   \n",
       "\n",
       "               NN_dist      ag_HD1F20      ag_HD2F20      ag_HD3F20  \n",
       "0           397.198533      82.054823      83.755636     312.946111  \n",
       "1          2881.578460      88.413319      90.114132     319.304608  \n",
       "2         17711.673781      23.536822      25.237636     254.428111  \n",
       "3         21562.215777      88.413319      90.114132     319.304608  \n",
       "4         12874.077085      88.413319      90.114132     319.304608  \n",
       "...                ...            ...            ...            ...  \n",
       "13941119   2658.421843  139785.574025  139786.528309  139792.211856  \n",
       "13941120   1234.146014  139777.423770  139778.378054  139784.061601  \n",
       "13941121   9172.299324  139782.918886  139783.873169  139789.556716  \n",
       "13941122   4280.425149  139769.388584  139770.342867  139776.026414  \n",
       "13941123   2922.119100  158123.044333  158123.998616  158129.682163  \n",
       "\n",
       "[13941124 rows x 15 columns]"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag_to_HDurban = pd.merge(ag_snap, ag_all, on='NN',how='left')\n",
    "ag_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed_ 0</th>\n",
       "      <th>mgrs_code</th>\n",
       "      <th>type</th>\n",
       "      <th>GlobalID</th>\n",
       "      <th>Shape_Leng</th>\n",
       "      <th>Shape_Area</th>\n",
       "      <th>geometry</th>\n",
       "      <th>NN</th>\n",
       "      <th>NN_dist</th>\n",
       "      <th>ha_HD1F20</th>\n",
       "      <th>ha_HD2F20</th>\n",
       "      <th>ha_HD3F20</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28PCU1265_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}</td>\n",
       "      <td>0.004314</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-16.721473282454415 12.348636090165247)</td>\n",
       "      <td>7761872870</td>\n",
       "      <td>307.058089</td>\n",
       "      <td>63.403221</td>\n",
       "      <td>66.771529</td>\n",
       "      <td>291.019814</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>28PCU1365_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{372B104B-B208-4D14-84E2-8ABFD4D8C37A}</td>\n",
       "      <td>0.009910</td>\n",
       "      <td>0.000006</td>\n",
       "      <td>POINT (-16.716456507935607 12.34788723564788)</td>\n",
       "      <td>7761872869</td>\n",
       "      <td>801.450257</td>\n",
       "      <td>63.384661</td>\n",
       "      <td>66.752970</td>\n",
       "      <td>291.001255</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>28PCU1365_02</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{D03C2B85-5F35-4EE8-8346-B83494628F26}</td>\n",
       "      <td>0.003754</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-16.713855008830738 12.350880992129111)</td>\n",
       "      <td>6442044321</td>\n",
       "      <td>694.273717</td>\n",
       "      <td>62.374829</td>\n",
       "      <td>65.743138</td>\n",
       "      <td>289.991423</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>28PCU1566_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{5EAFF1C3-6EE5-4F96-99FC-78F924454480}</td>\n",
       "      <td>0.004401</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-16.701275174874546 12.355585269999269)</td>\n",
       "      <td>2142496418</td>\n",
       "      <td>689.246791</td>\n",
       "      <td>63.828351</td>\n",
       "      <td>67.196660</td>\n",
       "      <td>291.444945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>28PCU1566_02</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{1D6A9E17-0D49-446D-A23B-7A47B155DC64}</td>\n",
       "      <td>0.005357</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-16.698773736706396 12.356804484409668)</td>\n",
       "      <td>2142496429</td>\n",
       "      <td>607.912599</td>\n",
       "      <td>64.210725</td>\n",
       "      <td>67.579034</td>\n",
       "      <td>291.827319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125881</th>\n",
       "      <td>125881</td>\n",
       "      <td>125881</td>\n",
       "      <td>28QED6412_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{5555A010-36B2-47D2-96C4-BDD1E59111ED}</td>\n",
       "      <td>0.005397</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-14.397827933065358 16.394142941310925)</td>\n",
       "      <td>8592243241</td>\n",
       "      <td>5089.086102</td>\n",
       "      <td>191.325867</td>\n",
       "      <td>193.414105</td>\n",
       "      <td>199.472924</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125882</th>\n",
       "      <td>125882</td>\n",
       "      <td>125882</td>\n",
       "      <td>28QED6413_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{20205A44-8B9D-4FCE-B14C-53826594DB5A}</td>\n",
       "      <td>0.003610</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-14.397473236932905 16.405676938062538)</td>\n",
       "      <td>6375187769</td>\n",
       "      <td>3949.180081</td>\n",
       "      <td>189.396436</td>\n",
       "      <td>191.484673</td>\n",
       "      <td>197.543492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125883</th>\n",
       "      <td>125883</td>\n",
       "      <td>125883</td>\n",
       "      <td>28QED6413_04</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}</td>\n",
       "      <td>0.015471</td>\n",
       "      <td>0.000008</td>\n",
       "      <td>POINT (-14.400364192239715 16.404303134272737)</td>\n",
       "      <td>8592243457</td>\n",
       "      <td>4008.851333</td>\n",
       "      <td>191.027109</td>\n",
       "      <td>193.115346</td>\n",
       "      <td>199.174166</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125884</th>\n",
       "      <td>125884</td>\n",
       "      <td>125884</td>\n",
       "      <td>28QED6424_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{51593C65-B268-4BA1-8212-E43232C021FF}</td>\n",
       "      <td>0.003883</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-14.396243884215123 16.49715066208162)</td>\n",
       "      <td>3646207611</td>\n",
       "      <td>1826.174300</td>\n",
       "      <td>187.250035</td>\n",
       "      <td>189.338272</td>\n",
       "      <td>195.397091</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125885</th>\n",
       "      <td>125885</td>\n",
       "      <td>125885</td>\n",
       "      <td>28QED6438_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{05114E26-63E9-4384-9F9E-8533D8AE2F1F}</td>\n",
       "      <td>0.007869</td>\n",
       "      <td>0.000004</td>\n",
       "      <td>POINT (-14.395109427617555 16.631548057253372)</td>\n",
       "      <td>7798350276</td>\n",
       "      <td>708.512647</td>\n",
       "      <td>203.882049</td>\n",
       "      <td>205.970286</td>\n",
       "      <td>212.029105</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>125886 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Unnamed: 0  Unnamed_ 0     mgrs_code    type  \\\n",
       "0                0           0  28PCU1265_01  hamlet   \n",
       "1                1           1  28PCU1365_01  hamlet   \n",
       "2                2           2  28PCU1365_02  hamlet   \n",
       "3                3           3  28PCU1566_01  hamlet   \n",
       "4                4           4  28PCU1566_02  hamlet   \n",
       "...            ...         ...           ...     ...   \n",
       "125881      125881      125881  28QED6412_03  hamlet   \n",
       "125882      125882      125882  28QED6413_03  hamlet   \n",
       "125883      125883      125883  28QED6413_04  hamlet   \n",
       "125884      125884      125884  28QED6424_03  hamlet   \n",
       "125885      125885      125885  28QED6438_03  hamlet   \n",
       "\n",
       "                                      GlobalID  Shape_Leng  Shape_Area  \\\n",
       "0       {ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}    0.004314    0.000001   \n",
       "1       {372B104B-B208-4D14-84E2-8ABFD4D8C37A}    0.009910    0.000006   \n",
       "2       {D03C2B85-5F35-4EE8-8346-B83494628F26}    0.003754    0.000001   \n",
       "3       {5EAFF1C3-6EE5-4F96-99FC-78F924454480}    0.004401    0.000002   \n",
       "4       {1D6A9E17-0D49-446D-A23B-7A47B155DC64}    0.005357    0.000002   \n",
       "...                                        ...         ...         ...   \n",
       "125881  {5555A010-36B2-47D2-96C4-BDD1E59111ED}    0.005397    0.000002   \n",
       "125882  {20205A44-8B9D-4FCE-B14C-53826594DB5A}    0.003610    0.000001   \n",
       "125883  {AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}    0.015471    0.000008   \n",
       "125884  {51593C65-B268-4BA1-8212-E43232C021FF}    0.003883    0.000001   \n",
       "125885  {05114E26-63E9-4384-9F9E-8533D8AE2F1F}    0.007869    0.000004   \n",
       "\n",
       "                                              geometry          NN  \\\n",
       "0       POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
       "1        POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
       "2       POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
       "3       POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
       "4       POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
       "...                                                ...         ...   \n",
       "125881  POINT (-14.397827933065358 16.394142941310925)  8592243241   \n",
       "125882  POINT (-14.397473236932905 16.405676938062538)  6375187769   \n",
       "125883  POINT (-14.400364192239715 16.404303134272737)  8592243457   \n",
       "125884   POINT (-14.396243884215123 16.49715066208162)  3646207611   \n",
       "125885  POINT (-14.395109427617555 16.631548057253372)  7798350276   \n",
       "\n",
       "            NN_dist   ha_HD1F20   ha_HD2F20   ha_HD3F20  \n",
       "0        307.058089   63.403221   66.771529  291.019814  \n",
       "1        801.450257   63.384661   66.752970  291.001255  \n",
       "2        694.273717   62.374829   65.743138  289.991423  \n",
       "3        689.246791   63.828351   67.196660  291.444945  \n",
       "4        607.912599   64.210725   67.579034  291.827319  \n",
       "...             ...         ...         ...         ...  \n",
       "125881  5089.086102  191.325867  193.414105  199.472924  \n",
       "125882  3949.180081  189.396436  191.484673  197.543492  \n",
       "125883  4008.851333  191.027109  193.115346  199.174166  \n",
       "125884  1826.174300  187.250035  189.338272  195.397091  \n",
       "125885   708.512647  203.882049  205.970286  212.029105  \n",
       "\n",
       "[125886 rows x 13 columns]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hamlet_to_HDurban = pd.merge(hamlet_snap, hamlet_all, on='NN',how='left')\n",
    "hamlet_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet_to_HDurban.to_csv(os.path.join(out_pth, 'hamlet_to_HDurban_flood20.csv'))\n",
    "ag_to_HDurban.to_csv(os.path.join(out_pth, 'ag_to_HDurban_flood20.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combine with cost-distance raster travel times from origins to road node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 16 columns):\n",
      " #   Column        Dtype  \n",
      "---  ------        -----  \n",
      " 0   Unnamed: 0    int64  \n",
      " 1   Unnamed: 0.1  int64  \n",
      " 2   ID_ag         int64  \n",
      " 3   ID_spam       float64\n",
      " 4   grid_val      float64\n",
      " 5   ID_LC         float64\n",
      " 6   val           float64\n",
      " 7   x             float64\n",
      " 8   y             float64\n",
      " 9   NSnomax       float64\n",
      " 10  geometry      object \n",
      " 11  NN            int64  \n",
      " 12  NN_dist       float64\n",
      " 13  ag_HD1F20     float64\n",
      " 14  ag_HD2F20     float64\n",
      " 15  ag_HD3F20     float64\n",
      "dtypes: float64(11), int64(4), object(1)\n",
      "memory usage: 1.7+ GB\n"
     ]
    }
   ],
   "source": [
    "# If reloading\n",
    "ag_to_HDurban = os.path.join(out_pth, \"ag_to_HDurban_flood20.csv\")\n",
    "ag_to_HDurban = pd.read_csv(ag_to_HDurban)\n",
    "ag_to_HDurban.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>ID_spam</th>\n",
       "      <th>grid_val</th>\n",
       "      <th>ID_LC</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>geometry</th>\n",
       "      <th>NN</th>\n",
       "      <th>NN_dist</th>\n",
       "      <th>ag_HD1F20</th>\n",
       "      <th>ag_HD2F20</th>\n",
       "      <th>ag_HD3F20</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13941103</td>\n",
       "      <td>2344.0</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>-16.458988</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.458987999999977 12.208646000000044)</td>\n",
       "      <td>3507831609</td>\n",
       "      <td>397.198533</td>\n",
       "      <td>82.054823</td>\n",
       "      <td>83.755636</td>\n",
       "      <td>312.946111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13941122</td>\n",
       "      <td>2345.0</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>-16.375655</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.375654999999938 12.208646000000044)</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>2881.578460</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>319.304608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>13941017</td>\n",
       "      <td>2346.0</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>-16.208989</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.208988999999974 12.208646000000044)</td>\n",
       "      <td>6188134127</td>\n",
       "      <td>17711.673781</td>\n",
       "      <td>23.536822</td>\n",
       "      <td>25.237636</td>\n",
       "      <td>254.428111</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>13941036</td>\n",
       "      <td>2402.0</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>-16.208492</td>\n",
       "      <td>12.152815</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.20849199999998 12.152815000000032)</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>21562.215777</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>319.304608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>13941044</td>\n",
       "      <td>2401.0</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>-16.294213</td>\n",
       "      <td>12.149487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.294212999999957 12.149487000000022)</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>12874.077085</td>\n",
       "      <td>88.413319</td>\n",
       "      <td>90.114132</td>\n",
       "      <td>319.304608</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941119</th>\n",
       "      <td>13941119</td>\n",
       "      <td>13941119</td>\n",
       "      <td>13941081</td>\n",
       "      <td>3.0</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>-15.291536</td>\n",
       "      <td>16.874047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.291535999999951 16.874047000000076)</td>\n",
       "      <td>3651042474</td>\n",
       "      <td>2658.421843</td>\n",
       "      <td>139785.574025</td>\n",
       "      <td>139786.528309</td>\n",
       "      <td>139792.211856</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941120</th>\n",
       "      <td>13941120</td>\n",
       "      <td>13941120</td>\n",
       "      <td>13941093</td>\n",
       "      <td>6.0</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>-15.292326</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.292325999999946 16.79196100000007)</td>\n",
       "      <td>3651042508</td>\n",
       "      <td>1234.146014</td>\n",
       "      <td>139777.423770</td>\n",
       "      <td>139778.378054</td>\n",
       "      <td>139784.061601</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941121</th>\n",
       "      <td>13941121</td>\n",
       "      <td>13941121</td>\n",
       "      <td>13941109</td>\n",
       "      <td>2.0</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>-15.374255</td>\n",
       "      <td>16.867753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.374254999999948 16.86775300000005)</td>\n",
       "      <td>3651042501</td>\n",
       "      <td>9172.299324</td>\n",
       "      <td>139782.918886</td>\n",
       "      <td>139783.873169</td>\n",
       "      <td>139789.556716</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941122</th>\n",
       "      <td>13941122</td>\n",
       "      <td>13941122</td>\n",
       "      <td>13941119</td>\n",
       "      <td>5.0</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>-15.375659</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.375658999999928 16.79196100000007)</td>\n",
       "      <td>3651042393</td>\n",
       "      <td>4280.425149</td>\n",
       "      <td>139769.388584</td>\n",
       "      <td>139770.342867</td>\n",
       "      <td>139776.026414</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941123</th>\n",
       "      <td>13941123</td>\n",
       "      <td>13941123</td>\n",
       "      <td>13941114</td>\n",
       "      <td>7.0</td>\n",
       "      <td>901.0</td>\n",
       "      <td>0.1</td>\n",
       "      <td>901.0</td>\n",
       "      <td>-14.875661</td>\n",
       "      <td>16.785873</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-14.87566099999998 16.785873000000038)</td>\n",
       "      <td>6323434997</td>\n",
       "      <td>2922.119100</td>\n",
       "      <td>158123.044333</td>\n",
       "      <td>158123.998616</td>\n",
       "      <td>158129.682163</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>13941124 rows × 16 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Unnamed: 0  Unnamed: 0.1     ID_ag  ID_spam   grid_val  ID_LC  \\\n",
       "0                  0             0  13941103   2344.0  2008851.0    0.1   \n",
       "1                  1             1  13941122   2345.0    94605.0    0.1   \n",
       "2                  2             2  13941017   2346.0   708413.0    0.1   \n",
       "3                  3             3  13941036   2402.0   315719.0    0.1   \n",
       "4                  4             4  13941044   2401.0   375288.0    0.1   \n",
       "...              ...           ...       ...      ...        ...    ...   \n",
       "13941119    13941119      13941119  13941081      3.0    87412.0    0.1   \n",
       "13941120    13941120      13941120  13941093      6.0    21660.0    0.1   \n",
       "13941121    13941121      13941121  13941109      2.0    70993.0    0.1   \n",
       "13941122    13941122      13941122  13941119      5.0    54822.0    0.1   \n",
       "13941123    13941123      13941123  13941114      7.0      901.0    0.1   \n",
       "\n",
       "                val          x          y  NSnomax  \\\n",
       "0         2008851.0 -16.458988  12.208646      NaN   \n",
       "1           94605.0 -16.375655  12.208646      NaN   \n",
       "2          708413.0 -16.208989  12.208646      NaN   \n",
       "3          315719.0 -16.208492  12.152815      NaN   \n",
       "4          375288.0 -16.294213  12.149487      NaN   \n",
       "...             ...        ...        ...      ...   \n",
       "13941119    87412.0 -15.291536  16.874047      NaN   \n",
       "13941120    21660.0 -15.292326  16.791961      NaN   \n",
       "13941121    70993.0 -15.374255  16.867753      NaN   \n",
       "13941122    54822.0 -15.375659  16.791961      NaN   \n",
       "13941123      901.0 -14.875661  16.785873      NaN   \n",
       "\n",
       "                                                geometry          NN  \\\n",
       "0         POINT (-16.458987999999977 12.208646000000044)  3507831609   \n",
       "1         POINT (-16.375654999999938 12.208646000000044)  3507831510   \n",
       "2         POINT (-16.208988999999974 12.208646000000044)  6188134127   \n",
       "3          POINT (-16.20849199999998 12.152815000000032)  3507831510   \n",
       "4         POINT (-16.294212999999957 12.149487000000022)  3507831510   \n",
       "...                                                  ...         ...   \n",
       "13941119  POINT (-15.291535999999951 16.874047000000076)  3651042474   \n",
       "13941120   POINT (-15.292325999999946 16.79196100000007)  3651042508   \n",
       "13941121   POINT (-15.374254999999948 16.86775300000005)  3651042501   \n",
       "13941122   POINT (-15.375658999999928 16.79196100000007)  3651042393   \n",
       "13941123   POINT (-14.87566099999998 16.785873000000038)  6323434997   \n",
       "\n",
       "               NN_dist      ag_HD1F20      ag_HD2F20      ag_HD3F20  \n",
       "0           397.198533      82.054823      83.755636     312.946111  \n",
       "1          2881.578460      88.413319      90.114132     319.304608  \n",
       "2         17711.673781      23.536822      25.237636     254.428111  \n",
       "3         21562.215777      88.413319      90.114132     319.304608  \n",
       "4         12874.077085      88.413319      90.114132     319.304608  \n",
       "...                ...            ...            ...            ...  \n",
       "13941119   2658.421843  139785.574025  139786.528309  139792.211856  \n",
       "13941120   1234.146014  139777.423770  139778.378054  139784.061601  \n",
       "13941121   9172.299324  139782.918886  139783.873169  139789.556716  \n",
       "13941122   4280.425149  139769.388584  139770.342867  139776.026414  \n",
       "13941123   2922.119100  158123.044333  158123.998616  158129.682163  \n",
       "\n",
       "[13941124 rows x 16 columns]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag_to_HDurban['NSnomax'].replace({-9999: np.nan},inplace =True)\n",
    "ag_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "ag_to_HDurban['HD1mmF20'] = 0 # mm for multi-modal\n",
    "ag_to_HDurban['HD1mmF20'] = ag_to_HDurban['ag_HD1F20'] + ag_to_HDurban['NSnomax']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>HD1mmF20</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>13941103</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>-16.458988</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831609</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13941122</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>-16.375655</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13941017</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>-16.208989</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6188134127</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>13941036</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>-16.208492</td>\n",
       "      <td>12.152815</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13941044</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>-16.294213</td>\n",
       "      <td>12.149487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941119</th>\n",
       "      <td>13941081</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>-15.291536</td>\n",
       "      <td>16.874047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042474</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941120</th>\n",
       "      <td>13941093</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>-15.292326</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042508</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941121</th>\n",
       "      <td>13941109</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>-15.374255</td>\n",
       "      <td>16.867753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042501</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941122</th>\n",
       "      <td>13941119</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>-15.375659</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042393</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941123</th>\n",
       "      <td>13941114</td>\n",
       "      <td>901.0</td>\n",
       "      <td>-14.875661</td>\n",
       "      <td>16.785873</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6323434997</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>13941124 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             ID_ag        val          x          y  NSnomax          NN  \\\n",
       "0         13941103  2008851.0 -16.458988  12.208646      NaN  3507831609   \n",
       "1         13941122    94605.0 -16.375655  12.208646      NaN  3507831510   \n",
       "2         13941017   708413.0 -16.208989  12.208646      NaN  6188134127   \n",
       "3         13941036   315719.0 -16.208492  12.152815      NaN  3507831510   \n",
       "4         13941044   375288.0 -16.294213  12.149487      NaN  3507831510   \n",
       "...            ...        ...        ...        ...      ...         ...   \n",
       "13941119  13941081    87412.0 -15.291536  16.874047      NaN  3651042474   \n",
       "13941120  13941093    21660.0 -15.292326  16.791961      NaN  3651042508   \n",
       "13941121  13941109    70993.0 -15.374255  16.867753      NaN  3651042501   \n",
       "13941122  13941119    54822.0 -15.375659  16.791961      NaN  3651042393   \n",
       "13941123  13941114      901.0 -14.875661  16.785873      NaN  6323434997   \n",
       "\n",
       "          HD1mmF20  \n",
       "0              NaN  \n",
       "1              NaN  \n",
       "2              NaN  \n",
       "3              NaN  \n",
       "4              NaN  \n",
       "...            ...  \n",
       "13941119       NaN  \n",
       "13941120       NaN  \n",
       "13941121       NaN  \n",
       "13941122       NaN  \n",
       "13941123       NaN  \n",
       "\n",
       "[13941124 rows x 7 columns]"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remove unnecessary data. These data are still saved in the _to_HDurban files.\n",
    "ag_to_HDurban = ag_to_HDurban[['ID_ag', 'val', 'x', 'y', 'NSnomax', 'NN', 'HD1mmF20']]\n",
    "ag_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1163857\n",
      "755160\n"
     ]
    }
   ],
   "source": [
    "print(len(ag_to_HDurban.loc[ag_to_HDurban['HD1mmF20']>240]), end=\"\\n\") # Number of origins isolated by excessive travel time\n",
    "print(len(ag_to_HDurban.loc[ag_to_HDurban['HD1mmF20'].isnull()])) # Number of origins isolated by inability to access road\n",
    "# Second value should match the non-flooded results (755160), as the cost-distance raster does not incorporate the flood penalties."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 5 columns):\n",
      " #   Column    Dtype  \n",
      "---  ------    -----  \n",
      " 0   ID_ag     int64  \n",
      " 1   val       float64\n",
      " 2   NSnomax   float64\n",
      " 3   NN        int64  \n",
      " 4   ag_HD1mm  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 531.8 MB\n"
     ]
    }
   ],
   "source": [
    "# Add non-flooded multi-modal values to table.\n",
    "ag_base = os.path.join(out_pth, \"temp_fromag_drive-CD.csv\")\n",
    "ag_base = pd.read_csv(ag_base)\n",
    "ag_base.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 2 columns):\n",
      " #   Column  Dtype  \n",
      "---  ------  -----  \n",
      " 0   ID_ag   int64  \n",
      " 1   HD1mm   float64\n",
      "dtypes: float64(1), int64(1)\n",
      "memory usage: 212.7 MB\n"
     ]
    }
   ],
   "source": [
    "ag_base = ag_base[['ID_ag', 'ag_HD1mm']]\n",
    "ag_base.rename(columns={'ag_HD1mm': 'HD1mm'}, inplace=True)\n",
    "ag_base.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 8 columns):\n",
      " #   Column    Dtype  \n",
      "---  ------    -----  \n",
      " 0   ID_ag     int64  \n",
      " 1   val       float64\n",
      " 2   x         float64\n",
      " 3   y         float64\n",
      " 4   NSnomax   float64\n",
      " 5   NN        int64  \n",
      " 6   HD1mmF20  float64\n",
      " 7   HD1mm     float64\n",
      "dtypes: float64(6), int64(2)\n",
      "memory usage: 957.3 MB\n"
     ]
    }
   ],
   "source": [
    "ag_to_HDurban = pd.merge(ag_to_HDurban, ag_base, on='ID_ag',how='left')\n",
    "ag_to_HDurban.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>HD1mmF20</th>\n",
       "      <th>HD1mm</th>\n",
       "      <th>HD1dif</th>\n",
       "      <th>HD1pc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>13941103</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>-16.458988</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831609</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>13941122</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>-16.375655</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13941017</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>-16.208989</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6188134127</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>13941036</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>-16.208492</td>\n",
       "      <td>12.152815</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>13941044</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>-16.294213</td>\n",
       "      <td>12.149487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941119</th>\n",
       "      <td>13941081</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>-15.291536</td>\n",
       "      <td>16.874047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042474</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941120</th>\n",
       "      <td>13941093</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>-15.292326</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042508</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941121</th>\n",
       "      <td>13941109</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>-15.374255</td>\n",
       "      <td>16.867753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042501</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941122</th>\n",
       "      <td>13941119</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>-15.375659</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042393</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941123</th>\n",
       "      <td>13941114</td>\n",
       "      <td>901.0</td>\n",
       "      <td>-14.875661</td>\n",
       "      <td>16.785873</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6323434997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>13941124 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             ID_ag        val          x          y  NSnomax          NN  \\\n",
       "0         13941103  2008851.0 -16.458988  12.208646      NaN  3507831609   \n",
       "1         13941122    94605.0 -16.375655  12.208646      NaN  3507831510   \n",
       "2         13941017   708413.0 -16.208989  12.208646      NaN  6188134127   \n",
       "3         13941036   315719.0 -16.208492  12.152815      NaN  3507831510   \n",
       "4         13941044   375288.0 -16.294213  12.149487      NaN  3507831510   \n",
       "...            ...        ...        ...        ...      ...         ...   \n",
       "13941119  13941081    87412.0 -15.291536  16.874047      NaN  3651042474   \n",
       "13941120  13941093    21660.0 -15.292326  16.791961      NaN  3651042508   \n",
       "13941121  13941109    70993.0 -15.374255  16.867753      NaN  3651042501   \n",
       "13941122  13941119    54822.0 -15.375659  16.791961      NaN  3651042393   \n",
       "13941123  13941114      901.0 -14.875661  16.785873      NaN  6323434997   \n",
       "\n",
       "          HD1mmF20  HD1mm  HD1dif  HD1pc  \n",
       "0              NaN    NaN     NaN    NaN  \n",
       "1              NaN    NaN     NaN    NaN  \n",
       "2              NaN    NaN     NaN    NaN  \n",
       "3              NaN    NaN     NaN    NaN  \n",
       "4              NaN    NaN     NaN    NaN  \n",
       "...            ...    ...     ...    ...  \n",
       "13941119       NaN    NaN     NaN    NaN  \n",
       "13941120       NaN    NaN     NaN    NaN  \n",
       "13941121       NaN    NaN     NaN    NaN  \n",
       "13941122       NaN    NaN     NaN    NaN  \n",
       "13941123       NaN    NaN     NaN    NaN  \n",
       "\n",
       "[13941124 rows x 10 columns]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ag_to_HDurban['HD1dif'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "ag_to_HDurban['HD1pc'] = 0 # pc for percent change\n",
    "ag_to_HDurban['HD1dif'] = ag_to_HDurban['HD1mmF20'] - ag_to_HDurban['HD1mm'] # Travel time is X minutes longer in flood conditions.\n",
    "ag_to_HDurban['HD1pc'] = ag_to_HDurban['HD1dif'] / ag_to_HDurban['HD1mm'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "ag_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1163857\n",
      "755160\n",
      "50507\n",
      "755160\n"
     ]
    }
   ],
   "source": [
    "print(len(ag_to_HDurban.loc[ag_to_HDurban['HD1mmF20']>240]), end=\"\\n\") # Number of origins isolated by excessive travel time\n",
    "print(len(ag_to_HDurban.loc[ag_to_HDurban['HD1mmF20'].isnull()])) # Number of origins isolated by inability to access road\n",
    "\n",
    "# Second value should match the non-flooded results, as the cost-distance raster does not incorporate the flood penalties.\n",
    "print(len(ag_to_HDurban.loc[ag_to_HDurban['HD1mm']>240]), end=\"\\n\") # Number of origins isolated by excessive travel time\n",
    "print(len(ag_to_HDurban.loc[ag_to_HDurban['HD1mm'].isnull()]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>HD1mmF20</th>\n",
       "      <th>HD1mm</th>\n",
       "      <th>HD1dif</th>\n",
       "      <th>HD1pc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.318626e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>1.219702e+02</td>\n",
       "      <td>-1.558418e+01</td>\n",
       "      <td>1.432788e+01</td>\n",
       "      <td>2.209557e+01</td>\n",
       "      <td>5.121305e+09</td>\n",
       "      <td>5.217945e+02</td>\n",
       "      <td>7.234398e+01</td>\n",
       "      <td>4.494505e+02</td>\n",
       "      <td>3.929297e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>4.024456e+06</td>\n",
       "      <td>3.760062e+03</td>\n",
       "      <td>9.437674e-01</td>\n",
       "      <td>9.172257e-01</td>\n",
       "      <td>2.796064e+01</td>\n",
       "      <td>2.262019e+09</td>\n",
       "      <td>5.061590e+03</td>\n",
       "      <td>4.196269e+01</td>\n",
       "      <td>5.052823e+03</td>\n",
       "      <td>2.803781e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>1.563126e-01</td>\n",
       "      <td>-1.748885e+01</td>\n",
       "      <td>1.204963e+01</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>2.214129e+07</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>-1.136868e-13</td>\n",
       "      <td>-1.000430e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.485281e+06</td>\n",
       "      <td>4.293236e+01</td>\n",
       "      <td>-1.622061e+01</td>\n",
       "      <td>1.368601e+01</td>\n",
       "      <td>5.456861e+00</td>\n",
       "      <td>3.656862e+09</td>\n",
       "      <td>4.999677e+01</td>\n",
       "      <td>4.336144e+01</td>\n",
       "      <td>4.113212e-01</td>\n",
       "      <td>6.045770e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>7.008440e+01</td>\n",
       "      <td>-1.578565e+01</td>\n",
       "      <td>1.436783e+01</td>\n",
       "      <td>1.162371e+01</td>\n",
       "      <td>4.519739e+09</td>\n",
       "      <td>8.137713e+01</td>\n",
       "      <td>6.605466e+01</td>\n",
       "      <td>2.693749e+00</td>\n",
       "      <td>4.421227e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.045584e+07</td>\n",
       "      <td>1.013998e+02</td>\n",
       "      <td>-1.516850e+01</td>\n",
       "      <td>1.491868e+01</td>\n",
       "      <td>2.869398e+01</td>\n",
       "      <td>6.968809e+09</td>\n",
       "      <td>1.274517e+02</td>\n",
       "      <td>9.401759e+01</td>\n",
       "      <td>3.236852e+01</td>\n",
       "      <td>3.900011e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>6.939670e+06</td>\n",
       "      <td>-1.115399e+01</td>\n",
       "      <td>1.687405e+01</td>\n",
       "      <td>1.713278e+03</td>\n",
       "      <td>9.285641e+09</td>\n",
       "      <td>2.218282e+05</td>\n",
       "      <td>1.801212e+03</td>\n",
       "      <td>2.213788e+05</td>\n",
       "      <td>2.456662e+05</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              ID_ag           val             x             y       NSnomax  \\\n",
       "count  1.394112e+07  1.394112e+07  1.394112e+07  1.394112e+07  1.318626e+07   \n",
       "mean   6.970562e+06  1.219702e+02 -1.558418e+01  1.432788e+01  2.209557e+01   \n",
       "std    4.024456e+06  3.760062e+03  9.437674e-01  9.172257e-01  2.796064e+01   \n",
       "min    0.000000e+00  1.563126e-01 -1.748885e+01  1.204963e+01  0.000000e+00   \n",
       "25%    3.485281e+06  4.293236e+01 -1.622061e+01  1.368601e+01  5.456861e+00   \n",
       "50%    6.970562e+06  7.008440e+01 -1.578565e+01  1.436783e+01  1.162371e+01   \n",
       "75%    1.045584e+07  1.013998e+02 -1.516850e+01  1.491868e+01  2.869398e+01   \n",
       "max    1.394112e+07  6.939670e+06 -1.115399e+01  1.687405e+01  1.713278e+03   \n",
       "\n",
       "                 NN      HD1mmF20         HD1mm        HD1dif         HD1pc  \n",
       "count  1.394112e+07  1.318596e+07  1.318596e+07  1.318596e+07  1.318596e+07  \n",
       "mean   5.121305e+09  5.217945e+02  7.234398e+01  4.494505e+02  3.929297e+02  \n",
       "std    2.262019e+09  5.061590e+03  4.196269e+01  5.052823e+03  2.803781e+03  \n",
       "min    2.214129e+07  0.000000e+00  0.000000e+00 -1.136868e-13 -1.000430e-13  \n",
       "25%    3.656862e+09  4.999677e+01  4.336144e+01  4.113212e-01  6.045770e-01  \n",
       "50%    4.519739e+09  8.137713e+01  6.605466e+01  2.693749e+00  4.421227e+00  \n",
       "75%    6.968809e+09  1.274517e+02  9.401759e+01  3.236852e+01  3.900011e+01  \n",
       "max    9.285641e+09  2.218282e+05  1.801212e+03  2.213788e+05  2.456662e+05  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(ag_to_HDurban.describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "521.7944964677994\n",
      "72.343983276716\n"
     ]
    }
   ],
   "source": [
    "print(ag_to_HDurban[\"HD1mmF20\"].mean())\n",
    "print(ag_to_HDurban[\"HD1mm\"].mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "ag_to_HDurban.to_csv(os.path.join(out_pth, 'ag_to_HDurban_mmF20.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hamlets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 125886 entries, 0 to 125885\n",
      "Data columns (total 14 columns):\n",
      " #   Column        Non-Null Count   Dtype  \n",
      "---  ------        --------------   -----  \n",
      " 0   Unnamed: 0    125886 non-null  int64  \n",
      " 1   Unnamed: 0.1  125886 non-null  int64  \n",
      " 2   Unnamed_ 0    125886 non-null  int64  \n",
      " 3   mgrs_code     125886 non-null  object \n",
      " 4   type          125886 non-null  object \n",
      " 5   GlobalID      125886 non-null  object \n",
      " 6   Shape_Leng    125886 non-null  float64\n",
      " 7   Shape_Area    125886 non-null  float64\n",
      " 8   geometry      125886 non-null  object \n",
      " 9   NN            125886 non-null  int64  \n",
      " 10  NN_dist       125886 non-null  float64\n",
      " 11  ha_HD1F20     125883 non-null  float64\n",
      " 12  ha_HD2F20     125886 non-null  float64\n",
      " 13  ha_HD3F20     125886 non-null  float64\n",
      "dtypes: float64(6), int64(4), object(4)\n",
      "memory usage: 13.4+ MB\n"
     ]
    }
   ],
   "source": [
    "hamlet_to_HDurban = os.path.join(out_pth, \"hamlet_to_HDurban_flood20.csv\")\n",
    "hamlet_to_HDurban = pd.read_csv(hamlet_to_HDurban)\n",
    "hamlet_to_HDurban.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 125886 entries, 0 to 125885\n",
      "Data columns (total 5 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   GlobalID   125886 non-null  object \n",
      " 2   HD1        125886 non-null  float64\n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   HD1mm      125552 non-null  float64\n",
      "dtypes: float64(3), object(2)\n",
      "memory usage: 4.8+ MB\n"
     ]
    }
   ],
   "source": [
    "# Hamlet file didn't already have the travel values from origin to road node. Merging now.\n",
    "costdist = os.path.join(out_pth, \"temp_fromhamlet_drive.csv\")\n",
    "costdist = pd.read_csv(costdist)\n",
    "costdist.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "125886\n",
      "125886\n"
     ]
    }
   ],
   "source": [
    "# Make sure the ID is unique and fully matched\n",
    "list1 = list(hamlet_to_HDurban['mgrs_code'].unique())\n",
    "list2 = list(costdist['mgrs_code'].unique())\n",
    "print(len(list1))\n",
    "print(len(list2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>Unnamed_ 0</th>\n",
       "      <th>mgrs_code</th>\n",
       "      <th>type</th>\n",
       "      <th>GlobalID_x</th>\n",
       "      <th>Shape_Leng</th>\n",
       "      <th>Shape_Area</th>\n",
       "      <th>geometry</th>\n",
       "      <th>NN</th>\n",
       "      <th>NN_dist</th>\n",
       "      <th>ha_HD1F20</th>\n",
       "      <th>ha_HD2F20</th>\n",
       "      <th>ha_HD3F20</th>\n",
       "      <th>GlobalID_y</th>\n",
       "      <th>HD1</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>HD1mm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28PCU1265_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}</td>\n",
       "      <td>0.004314</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-16.721473282454415 12.348636090165247)</td>\n",
       "      <td>7761872870</td>\n",
       "      <td>307.058089</td>\n",
       "      <td>63.403221</td>\n",
       "      <td>66.771529</td>\n",
       "      <td>291.019814</td>\n",
       "      <td>{ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}</td>\n",
       "      <td>62.539942</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>28PCU1365_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{372B104B-B208-4D14-84E2-8ABFD4D8C37A}</td>\n",
       "      <td>0.009910</td>\n",
       "      <td>0.000006</td>\n",
       "      <td>POINT (-16.716456507935607 12.34788723564788)</td>\n",
       "      <td>7761872869</td>\n",
       "      <td>801.450257</td>\n",
       "      <td>63.384661</td>\n",
       "      <td>66.752970</td>\n",
       "      <td>291.001255</td>\n",
       "      <td>{372B104B-B208-4D14-84E2-8ABFD4D8C37A}</td>\n",
       "      <td>62.521382</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>28PCU1365_02</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{D03C2B85-5F35-4EE8-8346-B83494628F26}</td>\n",
       "      <td>0.003754</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-16.713855008830738 12.350880992129111)</td>\n",
       "      <td>6442044321</td>\n",
       "      <td>694.273717</td>\n",
       "      <td>62.374829</td>\n",
       "      <td>65.743138</td>\n",
       "      <td>289.991423</td>\n",
       "      <td>{D03C2B85-5F35-4EE8-8346-B83494628F26}</td>\n",
       "      <td>61.511550</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>28PCU1566_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{5EAFF1C3-6EE5-4F96-99FC-78F924454480}</td>\n",
       "      <td>0.004401</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-16.701275174874546 12.355585269999269)</td>\n",
       "      <td>2142496418</td>\n",
       "      <td>689.246791</td>\n",
       "      <td>63.828351</td>\n",
       "      <td>67.196660</td>\n",
       "      <td>291.444945</td>\n",
       "      <td>{5EAFF1C3-6EE5-4F96-99FC-78F924454480}</td>\n",
       "      <td>62.965072</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>28PCU1566_02</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{1D6A9E17-0D49-446D-A23B-7A47B155DC64}</td>\n",
       "      <td>0.005357</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-16.698773736706396 12.356804484409668)</td>\n",
       "      <td>2142496429</td>\n",
       "      <td>607.912599</td>\n",
       "      <td>64.210725</td>\n",
       "      <td>67.579034</td>\n",
       "      <td>291.827319</td>\n",
       "      <td>{1D6A9E17-0D49-446D-A23B-7A47B155DC64}</td>\n",
       "      <td>63.347446</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125881</th>\n",
       "      <td>125881</td>\n",
       "      <td>125881</td>\n",
       "      <td>125881</td>\n",
       "      <td>28QED6412_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{5555A010-36B2-47D2-96C4-BDD1E59111ED}</td>\n",
       "      <td>0.005397</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-14.397827933065358 16.394142941310925)</td>\n",
       "      <td>8592243241</td>\n",
       "      <td>5089.086102</td>\n",
       "      <td>191.325867</td>\n",
       "      <td>193.414105</td>\n",
       "      <td>199.472924</td>\n",
       "      <td>{5555A010-36B2-47D2-96C4-BDD1E59111ED}</td>\n",
       "      <td>184.373909</td>\n",
       "      <td>64.9628</td>\n",
       "      <td>249.336709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125882</th>\n",
       "      <td>125882</td>\n",
       "      <td>125882</td>\n",
       "      <td>125882</td>\n",
       "      <td>28QED6413_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{20205A44-8B9D-4FCE-B14C-53826594DB5A}</td>\n",
       "      <td>0.003610</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-14.397473236932905 16.405676938062538)</td>\n",
       "      <td>6375187769</td>\n",
       "      <td>3949.180081</td>\n",
       "      <td>189.396436</td>\n",
       "      <td>191.484673</td>\n",
       "      <td>197.543492</td>\n",
       "      <td>{20205A44-8B9D-4FCE-B14C-53826594DB5A}</td>\n",
       "      <td>182.444477</td>\n",
       "      <td>83.4117</td>\n",
       "      <td>265.856177</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125883</th>\n",
       "      <td>125883</td>\n",
       "      <td>125883</td>\n",
       "      <td>125883</td>\n",
       "      <td>28QED6413_04</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}</td>\n",
       "      <td>0.015471</td>\n",
       "      <td>0.000008</td>\n",
       "      <td>POINT (-14.400364192239715 16.404303134272737)</td>\n",
       "      <td>8592243457</td>\n",
       "      <td>4008.851333</td>\n",
       "      <td>191.027109</td>\n",
       "      <td>193.115346</td>\n",
       "      <td>199.174166</td>\n",
       "      <td>{AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}</td>\n",
       "      <td>184.075151</td>\n",
       "      <td>86.7114</td>\n",
       "      <td>270.786551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125884</th>\n",
       "      <td>125884</td>\n",
       "      <td>125884</td>\n",
       "      <td>125884</td>\n",
       "      <td>28QED6424_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{51593C65-B268-4BA1-8212-E43232C021FF}</td>\n",
       "      <td>0.003883</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-14.396243884215123 16.49715066208162)</td>\n",
       "      <td>3646207611</td>\n",
       "      <td>1826.174300</td>\n",
       "      <td>187.250035</td>\n",
       "      <td>189.338272</td>\n",
       "      <td>195.397091</td>\n",
       "      <td>{51593C65-B268-4BA1-8212-E43232C021FF}</td>\n",
       "      <td>180.281911</td>\n",
       "      <td>56.9831</td>\n",
       "      <td>237.265011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125885</th>\n",
       "      <td>125885</td>\n",
       "      <td>125885</td>\n",
       "      <td>125885</td>\n",
       "      <td>28QED6438_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{05114E26-63E9-4384-9F9E-8533D8AE2F1F}</td>\n",
       "      <td>0.007869</td>\n",
       "      <td>0.000004</td>\n",
       "      <td>POINT (-14.395109427617555 16.631548057253372)</td>\n",
       "      <td>7798350276</td>\n",
       "      <td>708.512647</td>\n",
       "      <td>203.882049</td>\n",
       "      <td>205.970286</td>\n",
       "      <td>212.029105</td>\n",
       "      <td>{05114E26-63E9-4384-9F9E-8533D8AE2F1F}</td>\n",
       "      <td>195.264226</td>\n",
       "      <td>10.8968</td>\n",
       "      <td>206.161026</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>125886 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Unnamed: 0  Unnamed: 0.1  Unnamed_ 0     mgrs_code    type  \\\n",
       "0                0             0           0  28PCU1265_01  hamlet   \n",
       "1                1             1           1  28PCU1365_01  hamlet   \n",
       "2                2             2           2  28PCU1365_02  hamlet   \n",
       "3                3             3           3  28PCU1566_01  hamlet   \n",
       "4                4             4           4  28PCU1566_02  hamlet   \n",
       "...            ...           ...         ...           ...     ...   \n",
       "125881      125881        125881      125881  28QED6412_03  hamlet   \n",
       "125882      125882        125882      125882  28QED6413_03  hamlet   \n",
       "125883      125883        125883      125883  28QED6413_04  hamlet   \n",
       "125884      125884        125884      125884  28QED6424_03  hamlet   \n",
       "125885      125885        125885      125885  28QED6438_03  hamlet   \n",
       "\n",
       "                                    GlobalID_x  Shape_Leng  Shape_Area  \\\n",
       "0       {ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}    0.004314    0.000001   \n",
       "1       {372B104B-B208-4D14-84E2-8ABFD4D8C37A}    0.009910    0.000006   \n",
       "2       {D03C2B85-5F35-4EE8-8346-B83494628F26}    0.003754    0.000001   \n",
       "3       {5EAFF1C3-6EE5-4F96-99FC-78F924454480}    0.004401    0.000002   \n",
       "4       {1D6A9E17-0D49-446D-A23B-7A47B155DC64}    0.005357    0.000002   \n",
       "...                                        ...         ...         ...   \n",
       "125881  {5555A010-36B2-47D2-96C4-BDD1E59111ED}    0.005397    0.000002   \n",
       "125882  {20205A44-8B9D-4FCE-B14C-53826594DB5A}    0.003610    0.000001   \n",
       "125883  {AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}    0.015471    0.000008   \n",
       "125884  {51593C65-B268-4BA1-8212-E43232C021FF}    0.003883    0.000001   \n",
       "125885  {05114E26-63E9-4384-9F9E-8533D8AE2F1F}    0.007869    0.000004   \n",
       "\n",
       "                                              geometry          NN  \\\n",
       "0       POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
       "1        POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
       "2       POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
       "3       POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
       "4       POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
       "...                                                ...         ...   \n",
       "125881  POINT (-14.397827933065358 16.394142941310925)  8592243241   \n",
       "125882  POINT (-14.397473236932905 16.405676938062538)  6375187769   \n",
       "125883  POINT (-14.400364192239715 16.404303134272737)  8592243457   \n",
       "125884   POINT (-14.396243884215123 16.49715066208162)  3646207611   \n",
       "125885  POINT (-14.395109427617555 16.631548057253372)  7798350276   \n",
       "\n",
       "            NN_dist   ha_HD1F20   ha_HD2F20   ha_HD3F20  \\\n",
       "0        307.058089   63.403221   66.771529  291.019814   \n",
       "1        801.450257   63.384661   66.752970  291.001255   \n",
       "2        694.273717   62.374829   65.743138  289.991423   \n",
       "3        689.246791   63.828351   67.196660  291.444945   \n",
       "4        607.912599   64.210725   67.579034  291.827319   \n",
       "...             ...         ...         ...         ...   \n",
       "125881  5089.086102  191.325867  193.414105  199.472924   \n",
       "125882  3949.180081  189.396436  191.484673  197.543492   \n",
       "125883  4008.851333  191.027109  193.115346  199.174166   \n",
       "125884  1826.174300  187.250035  189.338272  195.397091   \n",
       "125885   708.512647  203.882049  205.970286  212.029105   \n",
       "\n",
       "                                    GlobalID_y         HD1  NSnomax  \\\n",
       "0       {ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}   62.539942      NaN   \n",
       "1       {372B104B-B208-4D14-84E2-8ABFD4D8C37A}   62.521382      NaN   \n",
       "2       {D03C2B85-5F35-4EE8-8346-B83494628F26}   61.511550      NaN   \n",
       "3       {5EAFF1C3-6EE5-4F96-99FC-78F924454480}   62.965072      NaN   \n",
       "4       {1D6A9E17-0D49-446D-A23B-7A47B155DC64}   63.347446      NaN   \n",
       "...                                        ...         ...      ...   \n",
       "125881  {5555A010-36B2-47D2-96C4-BDD1E59111ED}  184.373909  64.9628   \n",
       "125882  {20205A44-8B9D-4FCE-B14C-53826594DB5A}  182.444477  83.4117   \n",
       "125883  {AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}  184.075151  86.7114   \n",
       "125884  {51593C65-B268-4BA1-8212-E43232C021FF}  180.281911  56.9831   \n",
       "125885  {05114E26-63E9-4384-9F9E-8533D8AE2F1F}  195.264226  10.8968   \n",
       "\n",
       "             HD1mm  \n",
       "0              NaN  \n",
       "1              NaN  \n",
       "2              NaN  \n",
       "3              NaN  \n",
       "4              NaN  \n",
       "...            ...  \n",
       "125881  249.336709  \n",
       "125882  265.856177  \n",
       "125883  270.786551  \n",
       "125884  237.265011  \n",
       "125885  206.161026  \n",
       "\n",
       "[125886 rows x 18 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hamlet_to_HDurban = pd.merge(hamlet_to_HDurban, costdist, on='mgrs_code',how='left')\n",
    "hamlet_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>Unnamed_ 0</th>\n",
       "      <th>mgrs_code</th>\n",
       "      <th>type</th>\n",
       "      <th>GlobalID_x</th>\n",
       "      <th>Shape_Leng</th>\n",
       "      <th>Shape_Area</th>\n",
       "      <th>geometry</th>\n",
       "      <th>NN</th>\n",
       "      <th>NN_dist</th>\n",
       "      <th>ha_HD1F20</th>\n",
       "      <th>ha_HD2F20</th>\n",
       "      <th>ha_HD3F20</th>\n",
       "      <th>GlobalID_y</th>\n",
       "      <th>HD1</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>HD1mm</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>28PCU1265_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}</td>\n",
       "      <td>0.004314</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-16.721473282454415 12.348636090165247)</td>\n",
       "      <td>7761872870</td>\n",
       "      <td>307.058089</td>\n",
       "      <td>63.403221</td>\n",
       "      <td>66.771529</td>\n",
       "      <td>291.019814</td>\n",
       "      <td>{ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}</td>\n",
       "      <td>62.539942</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>28PCU1365_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{372B104B-B208-4D14-84E2-8ABFD4D8C37A}</td>\n",
       "      <td>0.009910</td>\n",
       "      <td>0.000006</td>\n",
       "      <td>POINT (-16.716456507935607 12.34788723564788)</td>\n",
       "      <td>7761872869</td>\n",
       "      <td>801.450257</td>\n",
       "      <td>63.384661</td>\n",
       "      <td>66.752970</td>\n",
       "      <td>291.001255</td>\n",
       "      <td>{372B104B-B208-4D14-84E2-8ABFD4D8C37A}</td>\n",
       "      <td>62.521382</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>28PCU1365_02</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{D03C2B85-5F35-4EE8-8346-B83494628F26}</td>\n",
       "      <td>0.003754</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-16.713855008830738 12.350880992129111)</td>\n",
       "      <td>6442044321</td>\n",
       "      <td>694.273717</td>\n",
       "      <td>62.374829</td>\n",
       "      <td>65.743138</td>\n",
       "      <td>289.991423</td>\n",
       "      <td>{D03C2B85-5F35-4EE8-8346-B83494628F26}</td>\n",
       "      <td>61.511550</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>28PCU1566_01</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{5EAFF1C3-6EE5-4F96-99FC-78F924454480}</td>\n",
       "      <td>0.004401</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-16.701275174874546 12.355585269999269)</td>\n",
       "      <td>2142496418</td>\n",
       "      <td>689.246791</td>\n",
       "      <td>63.828351</td>\n",
       "      <td>67.196660</td>\n",
       "      <td>291.444945</td>\n",
       "      <td>{5EAFF1C3-6EE5-4F96-99FC-78F924454480}</td>\n",
       "      <td>62.965072</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>28PCU1566_02</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{1D6A9E17-0D49-446D-A23B-7A47B155DC64}</td>\n",
       "      <td>0.005357</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-16.698773736706396 12.356804484409668)</td>\n",
       "      <td>2142496429</td>\n",
       "      <td>607.912599</td>\n",
       "      <td>64.210725</td>\n",
       "      <td>67.579034</td>\n",
       "      <td>291.827319</td>\n",
       "      <td>{1D6A9E17-0D49-446D-A23B-7A47B155DC64}</td>\n",
       "      <td>63.347446</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125881</th>\n",
       "      <td>125881</td>\n",
       "      <td>125881</td>\n",
       "      <td>125881</td>\n",
       "      <td>28QED6412_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{5555A010-36B2-47D2-96C4-BDD1E59111ED}</td>\n",
       "      <td>0.005397</td>\n",
       "      <td>0.000002</td>\n",
       "      <td>POINT (-14.397827933065358 16.394142941310925)</td>\n",
       "      <td>8592243241</td>\n",
       "      <td>5089.086102</td>\n",
       "      <td>191.325867</td>\n",
       "      <td>193.414105</td>\n",
       "      <td>199.472924</td>\n",
       "      <td>{5555A010-36B2-47D2-96C4-BDD1E59111ED}</td>\n",
       "      <td>184.373909</td>\n",
       "      <td>64.9628</td>\n",
       "      <td>249.336709</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125882</th>\n",
       "      <td>125882</td>\n",
       "      <td>125882</td>\n",
       "      <td>125882</td>\n",
       "      <td>28QED6413_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{20205A44-8B9D-4FCE-B14C-53826594DB5A}</td>\n",
       "      <td>0.003610</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-14.397473236932905 16.405676938062538)</td>\n",
       "      <td>6375187769</td>\n",
       "      <td>3949.180081</td>\n",
       "      <td>189.396436</td>\n",
       "      <td>191.484673</td>\n",
       "      <td>197.543492</td>\n",
       "      <td>{20205A44-8B9D-4FCE-B14C-53826594DB5A}</td>\n",
       "      <td>182.444477</td>\n",
       "      <td>83.4117</td>\n",
       "      <td>265.856177</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125883</th>\n",
       "      <td>125883</td>\n",
       "      <td>125883</td>\n",
       "      <td>125883</td>\n",
       "      <td>28QED6413_04</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}</td>\n",
       "      <td>0.015471</td>\n",
       "      <td>0.000008</td>\n",
       "      <td>POINT (-14.400364192239715 16.404303134272737)</td>\n",
       "      <td>8592243457</td>\n",
       "      <td>4008.851333</td>\n",
       "      <td>191.027109</td>\n",
       "      <td>193.115346</td>\n",
       "      <td>199.174166</td>\n",
       "      <td>{AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}</td>\n",
       "      <td>184.075151</td>\n",
       "      <td>86.7114</td>\n",
       "      <td>270.786551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125884</th>\n",
       "      <td>125884</td>\n",
       "      <td>125884</td>\n",
       "      <td>125884</td>\n",
       "      <td>28QED6424_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{51593C65-B268-4BA1-8212-E43232C021FF}</td>\n",
       "      <td>0.003883</td>\n",
       "      <td>0.000001</td>\n",
       "      <td>POINT (-14.396243884215123 16.49715066208162)</td>\n",
       "      <td>3646207611</td>\n",
       "      <td>1826.174300</td>\n",
       "      <td>187.250035</td>\n",
       "      <td>189.338272</td>\n",
       "      <td>195.397091</td>\n",
       "      <td>{51593C65-B268-4BA1-8212-E43232C021FF}</td>\n",
       "      <td>180.281911</td>\n",
       "      <td>56.9831</td>\n",
       "      <td>237.265011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125885</th>\n",
       "      <td>125885</td>\n",
       "      <td>125885</td>\n",
       "      <td>125885</td>\n",
       "      <td>28QED6438_03</td>\n",
       "      <td>hamlet</td>\n",
       "      <td>{05114E26-63E9-4384-9F9E-8533D8AE2F1F}</td>\n",
       "      <td>0.007869</td>\n",
       "      <td>0.000004</td>\n",
       "      <td>POINT (-14.395109427617555 16.631548057253372)</td>\n",
       "      <td>7798350276</td>\n",
       "      <td>708.512647</td>\n",
       "      <td>203.882049</td>\n",
       "      <td>205.970286</td>\n",
       "      <td>212.029105</td>\n",
       "      <td>{05114E26-63E9-4384-9F9E-8533D8AE2F1F}</td>\n",
       "      <td>195.264226</td>\n",
       "      <td>10.8968</td>\n",
       "      <td>206.161026</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>125886 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Unnamed: 0  Unnamed: 0.1  Unnamed_ 0     mgrs_code    type  \\\n",
       "0                0             0           0  28PCU1265_01  hamlet   \n",
       "1                1             1           1  28PCU1365_01  hamlet   \n",
       "2                2             2           2  28PCU1365_02  hamlet   \n",
       "3                3             3           3  28PCU1566_01  hamlet   \n",
       "4                4             4           4  28PCU1566_02  hamlet   \n",
       "...            ...           ...         ...           ...     ...   \n",
       "125881      125881        125881      125881  28QED6412_03  hamlet   \n",
       "125882      125882        125882      125882  28QED6413_03  hamlet   \n",
       "125883      125883        125883      125883  28QED6413_04  hamlet   \n",
       "125884      125884        125884      125884  28QED6424_03  hamlet   \n",
       "125885      125885        125885      125885  28QED6438_03  hamlet   \n",
       "\n",
       "                                    GlobalID_x  Shape_Leng  Shape_Area  \\\n",
       "0       {ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}    0.004314    0.000001   \n",
       "1       {372B104B-B208-4D14-84E2-8ABFD4D8C37A}    0.009910    0.000006   \n",
       "2       {D03C2B85-5F35-4EE8-8346-B83494628F26}    0.003754    0.000001   \n",
       "3       {5EAFF1C3-6EE5-4F96-99FC-78F924454480}    0.004401    0.000002   \n",
       "4       {1D6A9E17-0D49-446D-A23B-7A47B155DC64}    0.005357    0.000002   \n",
       "...                                        ...         ...         ...   \n",
       "125881  {5555A010-36B2-47D2-96C4-BDD1E59111ED}    0.005397    0.000002   \n",
       "125882  {20205A44-8B9D-4FCE-B14C-53826594DB5A}    0.003610    0.000001   \n",
       "125883  {AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}    0.015471    0.000008   \n",
       "125884  {51593C65-B268-4BA1-8212-E43232C021FF}    0.003883    0.000001   \n",
       "125885  {05114E26-63E9-4384-9F9E-8533D8AE2F1F}    0.007869    0.000004   \n",
       "\n",
       "                                              geometry          NN  \\\n",
       "0       POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
       "1        POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
       "2       POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
       "3       POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
       "4       POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
       "...                                                ...         ...   \n",
       "125881  POINT (-14.397827933065358 16.394142941310925)  8592243241   \n",
       "125882  POINT (-14.397473236932905 16.405676938062538)  6375187769   \n",
       "125883  POINT (-14.400364192239715 16.404303134272737)  8592243457   \n",
       "125884   POINT (-14.396243884215123 16.49715066208162)  3646207611   \n",
       "125885  POINT (-14.395109427617555 16.631548057253372)  7798350276   \n",
       "\n",
       "            NN_dist   ha_HD1F20   ha_HD2F20   ha_HD3F20  \\\n",
       "0        307.058089   63.403221   66.771529  291.019814   \n",
       "1        801.450257   63.384661   66.752970  291.001255   \n",
       "2        694.273717   62.374829   65.743138  289.991423   \n",
       "3        689.246791   63.828351   67.196660  291.444945   \n",
       "4        607.912599   64.210725   67.579034  291.827319   \n",
       "...             ...         ...         ...         ...   \n",
       "125881  5089.086102  191.325867  193.414105  199.472924   \n",
       "125882  3949.180081  189.396436  191.484673  197.543492   \n",
       "125883  4008.851333  191.027109  193.115346  199.174166   \n",
       "125884  1826.174300  187.250035  189.338272  195.397091   \n",
       "125885   708.512647  203.882049  205.970286  212.029105   \n",
       "\n",
       "                                    GlobalID_y         HD1  NSnomax  \\\n",
       "0       {ED2CCDD5-C78F-40B6-A18A-3A01B61A4998}   62.539942      NaN   \n",
       "1       {372B104B-B208-4D14-84E2-8ABFD4D8C37A}   62.521382      NaN   \n",
       "2       {D03C2B85-5F35-4EE8-8346-B83494628F26}   61.511550      NaN   \n",
       "3       {5EAFF1C3-6EE5-4F96-99FC-78F924454480}   62.965072      NaN   \n",
       "4       {1D6A9E17-0D49-446D-A23B-7A47B155DC64}   63.347446      NaN   \n",
       "...                                        ...         ...      ...   \n",
       "125881  {5555A010-36B2-47D2-96C4-BDD1E59111ED}  184.373909  64.9628   \n",
       "125882  {20205A44-8B9D-4FCE-B14C-53826594DB5A}  182.444477  83.4117   \n",
       "125883  {AC6A169C-FD0E-4DF6-BDAB-B69FBD04BFAF}  184.075151  86.7114   \n",
       "125884  {51593C65-B268-4BA1-8212-E43232C021FF}  180.281911  56.9831   \n",
       "125885  {05114E26-63E9-4384-9F9E-8533D8AE2F1F}  195.264226  10.8968   \n",
       "\n",
       "             HD1mm  \n",
       "0              NaN  \n",
       "1              NaN  \n",
       "2              NaN  \n",
       "3              NaN  \n",
       "4              NaN  \n",
       "...            ...  \n",
       "125881  249.336709  \n",
       "125882  265.856177  \n",
       "125883  270.786551  \n",
       "125884  237.265011  \n",
       "125885  206.161026  \n",
       "\n",
       "[125886 rows x 18 columns]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hamlet_to_HDurban['NSnomax'].replace({-9999: np.nan},inplace =True)\n",
    "hamlet_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet_to_HDurban['HD1mmF20'] = 0 # mm for multi-modal\n",
    "hamlet_to_HDurban['HD1mmF20'] = hamlet_to_HDurban['ha_HD1F20'] + hamlet_to_HDurban['NSnomax']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The geometry column is missing a comma for some reason and isn't reading as a GDF in Python or Arc. So let's quick make a shapefile version by extracting the points. Can use this as well for the non-flood travel times, which were saved as csv."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 20 columns):\n",
      " #   Column        Non-Null Count   Dtype  \n",
      "---  ------        --------------   -----  \n",
      " 0   Unnamed: 0    125886 non-null  int64  \n",
      " 1   Unnamed: 0.1  125886 non-null  int64  \n",
      " 2   Unnamed_ 0    125886 non-null  int64  \n",
      " 3   mgrs_code     125886 non-null  object \n",
      " 4   type          125886 non-null  object \n",
      " 5   GlobalID_x    125886 non-null  object \n",
      " 6   Shape_Leng    125886 non-null  float64\n",
      " 7   Shape_Area    125886 non-null  float64\n",
      " 8   NN            125886 non-null  int64  \n",
      " 9   NN_dist       125886 non-null  float64\n",
      " 10  ha_HD1F20     125883 non-null  float64\n",
      " 11  ha_HD2F20     125886 non-null  float64\n",
      " 12  ha_HD3F20     125886 non-null  float64\n",
      " 13  GlobalID_y    125886 non-null  object \n",
      " 14  HD1           125886 non-null  float64\n",
      " 15  NSnomax       125552 non-null  float64\n",
      " 16  HD1mm         125552 non-null  float64\n",
      " 17  HD1mmF20      125549 non-null  float64\n",
      " 18  X             125886 non-null  float64\n",
      " 19  Y             125886 non-null  float64\n",
      "dtypes: float64(12), int64(4), object(4)\n",
      "memory usage: 20.2+ MB\n"
     ]
    }
   ],
   "source": [
    "hamlet_to_HDurban[\"geometry\"] = hamlet_to_HDurban[\"geometry\"].astype('str')\n",
    "hamlet_to_HDurban[\"geometry\"]  = hamlet_to_HDurban[\"geometry\"] .str.strip('POINT ')\n",
    "hamlet_to_HDurban[\"geometry\"]  = hamlet_to_HDurban[\"geometry\"] .str.strip('()')\n",
    "XY = hamlet_to_HDurban[\"geometry\"] .str.split(\" \", expand=True)\n",
    "hamlet_to_HDurban[\"X\"] = XY[0]\n",
    "hamlet_to_HDurban[\"Y\"] = XY[1]\n",
    "hamlet_to_HDurban[\"X\"] = hamlet_to_HDurban[\"X\"].astype('float')\n",
    "hamlet_to_HDurban[\"Y\"] = hamlet_to_HDurban[\"Y\"].astype('float')\n",
    "hamlet_to_HDurban = hamlet_to_HDurban.drop(columns=['geometry'])\n",
    "hamlet_to_HDurban.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mgrs_code</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>HD1mm</th>\n",
       "      <th>HD1mmF20</th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>28PCU1265_01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7761872870</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.721473</td>\n",
       "      <td>12.348636</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28PCU1365_01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7761872869</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.716457</td>\n",
       "      <td>12.347887</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28PCU1365_02</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6442044321</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.713855</td>\n",
       "      <td>12.350881</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>28PCU1566_01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2142496418</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.701275</td>\n",
       "      <td>12.355585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28PCU1566_02</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2142496429</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.698774</td>\n",
       "      <td>12.356804</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125881</th>\n",
       "      <td>28QED6412_03</td>\n",
       "      <td>64.9628</td>\n",
       "      <td>8592243241</td>\n",
       "      <td>249.336709</td>\n",
       "      <td>256.288667</td>\n",
       "      <td>-14.397828</td>\n",
       "      <td>16.394143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125882</th>\n",
       "      <td>28QED6413_03</td>\n",
       "      <td>83.4117</td>\n",
       "      <td>6375187769</td>\n",
       "      <td>265.856177</td>\n",
       "      <td>272.808136</td>\n",
       "      <td>-14.397473</td>\n",
       "      <td>16.405677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125883</th>\n",
       "      <td>28QED6413_04</td>\n",
       "      <td>86.7114</td>\n",
       "      <td>8592243457</td>\n",
       "      <td>270.786551</td>\n",
       "      <td>277.738509</td>\n",
       "      <td>-14.400364</td>\n",
       "      <td>16.404303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125884</th>\n",
       "      <td>28QED6424_03</td>\n",
       "      <td>56.9831</td>\n",
       "      <td>3646207611</td>\n",
       "      <td>237.265011</td>\n",
       "      <td>244.233135</td>\n",
       "      <td>-14.396244</td>\n",
       "      <td>16.497151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125885</th>\n",
       "      <td>28QED6438_03</td>\n",
       "      <td>10.8968</td>\n",
       "      <td>7798350276</td>\n",
       "      <td>206.161026</td>\n",
       "      <td>214.778849</td>\n",
       "      <td>-14.395109</td>\n",
       "      <td>16.631548</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>125886 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           mgrs_code  NSnomax          NN       HD1mm    HD1mmF20          X  \\\n",
       "0       28PCU1265_01      NaN  7761872870         NaN         NaN -16.721473   \n",
       "1       28PCU1365_01      NaN  7761872869         NaN         NaN -16.716457   \n",
       "2       28PCU1365_02      NaN  6442044321         NaN         NaN -16.713855   \n",
       "3       28PCU1566_01      NaN  2142496418         NaN         NaN -16.701275   \n",
       "4       28PCU1566_02      NaN  2142496429         NaN         NaN -16.698774   \n",
       "...              ...      ...         ...         ...         ...        ...   \n",
       "125881  28QED6412_03  64.9628  8592243241  249.336709  256.288667 -14.397828   \n",
       "125882  28QED6413_03  83.4117  6375187769  265.856177  272.808136 -14.397473   \n",
       "125883  28QED6413_04  86.7114  8592243457  270.786551  277.738509 -14.400364   \n",
       "125884  28QED6424_03  56.9831  3646207611  237.265011  244.233135 -14.396244   \n",
       "125885  28QED6438_03  10.8968  7798350276  206.161026  214.778849 -14.395109   \n",
       "\n",
       "                Y  \n",
       "0       12.348636  \n",
       "1       12.347887  \n",
       "2       12.350881  \n",
       "3       12.355585  \n",
       "4       12.356804  \n",
       "...           ...  \n",
       "125881  16.394143  \n",
       "125882  16.405677  \n",
       "125883  16.404303  \n",
       "125884  16.497151  \n",
       "125885  16.631548  \n",
       "\n",
       "[125886 rows x 7 columns]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Remove unnecessary data. These data are still saved in the _to_HDurban files.\n",
    "hamlet_to_HDurban = hamlet_to_HDurban[['mgrs_code', 'NSnomax', 'NN', 'HD1mm','HD1mmF20', 'X', 'Y']]\n",
    "hamlet_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "26039\n",
      "337\n",
      "6541\n",
      "334\n"
     ]
    }
   ],
   "source": [
    "print(len(hamlet_to_HDurban.loc[hamlet_to_HDurban['HD1mmF20']>240]), end=\"\\n\") # Number of origins isolated by excessive travel time\n",
    "print(len(hamlet_to_HDurban.loc[hamlet_to_HDurban['HD1mmF20'].isnull()])) # Number of origins isolated by inability to access road\n",
    "\n",
    "# Second value should match the non-flooded results, as the cost-distance raster does not incorporate the flood penalties.\n",
    "print(len(hamlet_to_HDurban.loc[hamlet_to_HDurban['HD1mm']>240]), end=\"\\n\") # Number of origins isolated by excessive travel time\n",
    "print(len(hamlet_to_HDurban.loc[hamlet_to_HDurban['HD1mm'].isnull()]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mgrs_code</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>HD1mm</th>\n",
       "      <th>HD1mmF20</th>\n",
       "      <th>X</th>\n",
       "      <th>Y</th>\n",
       "      <th>geometry</th>\n",
       "      <th>HD1dif</th>\n",
       "      <th>HD1pc</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>28PCU1265_01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7761872870</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.721473</td>\n",
       "      <td>12.348636</td>\n",
       "      <td>POINT (-16.72147 12.34864)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28PCU1365_01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>7761872869</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.716457</td>\n",
       "      <td>12.347887</td>\n",
       "      <td>POINT (-16.71646 12.34789)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28PCU1365_02</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6442044321</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.713855</td>\n",
       "      <td>12.350881</td>\n",
       "      <td>POINT (-16.71386 12.35088)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>28PCU1566_01</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2142496418</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.701275</td>\n",
       "      <td>12.355585</td>\n",
       "      <td>POINT (-16.70128 12.35559)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28PCU1566_02</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2142496429</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>-16.698774</td>\n",
       "      <td>12.356804</td>\n",
       "      <td>POINT (-16.69877 12.35680)</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125881</th>\n",
       "      <td>28QED6412_03</td>\n",
       "      <td>64.9628</td>\n",
       "      <td>8592243241</td>\n",
       "      <td>249.336709</td>\n",
       "      <td>256.288667</td>\n",
       "      <td>-14.397828</td>\n",
       "      <td>16.394143</td>\n",
       "      <td>POINT (-14.39783 16.39414)</td>\n",
       "      <td>6.951959</td>\n",
       "      <td>2.788181</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125882</th>\n",
       "      <td>28QED6413_03</td>\n",
       "      <td>83.4117</td>\n",
       "      <td>6375187769</td>\n",
       "      <td>265.856177</td>\n",
       "      <td>272.808136</td>\n",
       "      <td>-14.397473</td>\n",
       "      <td>16.405677</td>\n",
       "      <td>POINT (-14.39747 16.40568)</td>\n",
       "      <td>6.951958</td>\n",
       "      <td>2.614932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125883</th>\n",
       "      <td>28QED6413_04</td>\n",
       "      <td>86.7114</td>\n",
       "      <td>8592243457</td>\n",
       "      <td>270.786551</td>\n",
       "      <td>277.738509</td>\n",
       "      <td>-14.400364</td>\n",
       "      <td>16.404303</td>\n",
       "      <td>POINT (-14.40036 16.40430)</td>\n",
       "      <td>6.951958</td>\n",
       "      <td>2.567320</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125884</th>\n",
       "      <td>28QED6424_03</td>\n",
       "      <td>56.9831</td>\n",
       "      <td>3646207611</td>\n",
       "      <td>237.265011</td>\n",
       "      <td>244.233135</td>\n",
       "      <td>-14.396244</td>\n",
       "      <td>16.497151</td>\n",
       "      <td>POINT (-14.39624 16.49715)</td>\n",
       "      <td>6.968124</td>\n",
       "      <td>2.936853</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>125885</th>\n",
       "      <td>28QED6438_03</td>\n",
       "      <td>10.8968</td>\n",
       "      <td>7798350276</td>\n",
       "      <td>206.161026</td>\n",
       "      <td>214.778849</td>\n",
       "      <td>-14.395109</td>\n",
       "      <td>16.631548</td>\n",
       "      <td>POINT (-14.39511 16.63155)</td>\n",
       "      <td>8.617823</td>\n",
       "      <td>4.180142</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>125886 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           mgrs_code  NSnomax          NN       HD1mm    HD1mmF20          X  \\\n",
       "0       28PCU1265_01      NaN  7761872870         NaN         NaN -16.721473   \n",
       "1       28PCU1365_01      NaN  7761872869         NaN         NaN -16.716457   \n",
       "2       28PCU1365_02      NaN  6442044321         NaN         NaN -16.713855   \n",
       "3       28PCU1566_01      NaN  2142496418         NaN         NaN -16.701275   \n",
       "4       28PCU1566_02      NaN  2142496429         NaN         NaN -16.698774   \n",
       "...              ...      ...         ...         ...         ...        ...   \n",
       "125881  28QED6412_03  64.9628  8592243241  249.336709  256.288667 -14.397828   \n",
       "125882  28QED6413_03  83.4117  6375187769  265.856177  272.808136 -14.397473   \n",
       "125883  28QED6413_04  86.7114  8592243457  270.786551  277.738509 -14.400364   \n",
       "125884  28QED6424_03  56.9831  3646207611  237.265011  244.233135 -14.396244   \n",
       "125885  28QED6438_03  10.8968  7798350276  206.161026  214.778849 -14.395109   \n",
       "\n",
       "                Y                    geometry    HD1dif     HD1pc  \n",
       "0       12.348636  POINT (-16.72147 12.34864)       NaN       NaN  \n",
       "1       12.347887  POINT (-16.71646 12.34789)       NaN       NaN  \n",
       "2       12.350881  POINT (-16.71386 12.35088)       NaN       NaN  \n",
       "3       12.355585  POINT (-16.70128 12.35559)       NaN       NaN  \n",
       "4       12.356804  POINT (-16.69877 12.35680)       NaN       NaN  \n",
       "...           ...                         ...       ...       ...  \n",
       "125881  16.394143  POINT (-14.39783 16.39414)  6.951959  2.788181  \n",
       "125882  16.405677  POINT (-14.39747 16.40568)  6.951958  2.614932  \n",
       "125883  16.404303  POINT (-14.40036 16.40430)  6.951958  2.567320  \n",
       "125884  16.497151  POINT (-14.39624 16.49715)  6.968124  2.936853  \n",
       "125885  16.631548  POINT (-14.39511 16.63155)  8.617823  4.180142  \n",
       "\n",
       "[125886 rows x 10 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hamlet_to_HDurban['HD1dif'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "hamlet_to_HDurban['HD1pc'] = 0 # pc for percent change\n",
    "hamlet_to_HDurban['HD1dif'] = hamlet_to_HDurban['HD1mmF20'] - hamlet_to_HDurban['HD1mm'] # Travel time is X minutes longer in flood conditions.\n",
    "hamlet_to_HDurban['HD1pc'] = hamlet_to_HDurban['HD1dif'] / hamlet_to_HDurban['HD1mm'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "hamlet_to_HDurban"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "display(hamlet_to_HDurban.describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Mon Dec 27 23:02:45 2021\n",
      "\n",
      "\n",
      "end: Mon Dec 27 23:03:28 2021\n",
      "\n",
      "--- processing complete\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "crs = \"EPSG:4326\"\n",
    "geometry = [Point(xy) for xy in zip(hamlet_to_HDurban.X, hamlet_to_HDurban.Y)]\n",
    "hamlet_to_HDurban = GeoDataFrame(hamlet_to_HDurban, crs=crs, geometry=geometry) \n",
    "hamlet_to_HDurban.to_file(driver='ESRI Shapefile', filename='R:/SEN/GEO/Team/Projects/Sen_TransportOV/hamlet_to_HDurban_mmF20.shp') \n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet_to_HDurban.to_csv(os.path.join(out_pth, 'hamlet_to_HDurban_mmF20.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add 1 in 50-year event to the base and flood table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# If reloading:\n",
    "agF20 = os.path.join(out_pth, \"ag_to_HDurban_mmF20.csv\")\n",
    "agF20 = pd.read_csv(agF20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 12 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID_ag       int64  \n",
      " 2   val         float64\n",
      " 3   x           float64\n",
      " 4   y           float64\n",
      " 5   NSnomax     float64\n",
      " 6   NN          int64  \n",
      " 7   HD1mmF20    float64\n",
      " 8   HD1mm       float64\n",
      " 9   HD1dif      float64\n",
      " 10  HD1pc       float64\n",
      " 11  geometry    object \n",
      "dtypes: float64(8), int64(3), object(1)\n",
      "memory usage: 1.2+ GB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 625851 entries, 0 to 625850\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   Unnamed: 0  625851 non-null  int64  \n",
      " 1   NN          625851 non-null  int64  \n",
      " 2   ag_HD1F50   625841 non-null  float64\n",
      " 3   ag_HD2F50   625851 non-null  float64\n",
      " 4   ag_HD3F50   625851 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 23.9 MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "agF50 = os.path.join(out_pth, \"ag_to_HDurban_flood50.csv\")\n",
    "agF50 = pd.read_csv(agF50)\n",
    "\n",
    "print(agF20.info())\n",
    "print(agF50.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "625851\n",
      "625851\n"
     ]
    }
   ],
   "source": [
    "# Make sure the ID is unique and fully matched\n",
    "list1 = list(agF50['NN'].unique())\n",
    "list2 = list(agF20['NN'].unique())\n",
    "list3 = list(agF10['NN'].unique())\n",
    "print(len(list1))\n",
    "print(len(list2))\n",
    "print(len(list3))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>ag_HD1F50</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3507831609</td>\n",
       "      <td>82.731533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3507831510</td>\n",
       "      <td>89.090029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6188134127</td>\n",
       "      <td>23.536822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8631201421</td>\n",
       "      <td>503.912228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8598305977</td>\n",
       "      <td>518.180056</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625846</th>\n",
       "      <td>3651042474</td>\n",
       "      <td>186216.753175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625847</th>\n",
       "      <td>3651042508</td>\n",
       "      <td>186208.602920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625848</th>\n",
       "      <td>3651042501</td>\n",
       "      <td>186214.098036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625849</th>\n",
       "      <td>3651042393</td>\n",
       "      <td>186200.567733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625850</th>\n",
       "      <td>6323434997</td>\n",
       "      <td>257953.468283</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625851 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                NN      ag_HD1F50\n",
       "0       3507831609      82.731533\n",
       "1       3507831510      89.090029\n",
       "2       6188134127      23.536822\n",
       "3       8631201421     503.912228\n",
       "4       8598305977     518.180056\n",
       "...            ...            ...\n",
       "625846  3651042474  186216.753175\n",
       "625847  3651042508  186208.602920\n",
       "625848  3651042501  186214.098036\n",
       "625849  3651042393  186200.567733\n",
       "625850  6323434997  257953.468283\n",
       "\n",
       "[625851 rows x 2 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agF50 = agF50[['NN', 'ag_HD1F50']]\n",
    "agF50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>HD1mmF20</th>\n",
       "      <th>HD1mm</th>\n",
       "      <th>HD1dif</th>\n",
       "      <th>HD1pc</th>\n",
       "      <th>geometry</th>\n",
       "      <th>ag_HD1F50</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>13941103</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>-16.458988</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831609</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.458988 12.208646)</td>\n",
       "      <td>82.731533</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>13941122</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>-16.375655</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.375655 12.208646)</td>\n",
       "      <td>89.090029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>13941017</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>-16.208989</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6188134127</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.208989 12.208646)</td>\n",
       "      <td>23.536822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>13941036</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>-16.208492</td>\n",
       "      <td>12.152815</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.208492 12.152815)</td>\n",
       "      <td>89.090029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>13941044</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>-16.294213</td>\n",
       "      <td>12.149487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.294213 12.149487)</td>\n",
       "      <td>89.090029</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941119</th>\n",
       "      <td>13941119</td>\n",
       "      <td>13941081</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>-15.291536</td>\n",
       "      <td>16.874047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042474</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.291536 16.874047)</td>\n",
       "      <td>186216.753175</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941120</th>\n",
       "      <td>13941120</td>\n",
       "      <td>13941093</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>-15.292326</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042508</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.292326 16.791961)</td>\n",
       "      <td>186208.602920</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941121</th>\n",
       "      <td>13941121</td>\n",
       "      <td>13941109</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>-15.374255</td>\n",
       "      <td>16.867753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042501</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.374255 16.867753)</td>\n",
       "      <td>186214.098036</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941122</th>\n",
       "      <td>13941122</td>\n",
       "      <td>13941119</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>-15.375659</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042393</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.375659 16.791961)</td>\n",
       "      <td>186200.567733</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941123</th>\n",
       "      <td>13941123</td>\n",
       "      <td>13941114</td>\n",
       "      <td>901.0</td>\n",
       "      <td>-14.875661</td>\n",
       "      <td>16.785873</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6323434997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-14.875661 16.785873)</td>\n",
       "      <td>257953.468283</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>13941124 rows × 13 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Unnamed: 0     ID_ag        val          x          y  NSnomax  \\\n",
       "0                  0  13941103  2008851.0 -16.458988  12.208646      NaN   \n",
       "1                  1  13941122    94605.0 -16.375655  12.208646      NaN   \n",
       "2                  2  13941017   708413.0 -16.208989  12.208646      NaN   \n",
       "3                  3  13941036   315719.0 -16.208492  12.152815      NaN   \n",
       "4                  4  13941044   375288.0 -16.294213  12.149487      NaN   \n",
       "...              ...       ...        ...        ...        ...      ...   \n",
       "13941119    13941119  13941081    87412.0 -15.291536  16.874047      NaN   \n",
       "13941120    13941120  13941093    21660.0 -15.292326  16.791961      NaN   \n",
       "13941121    13941121  13941109    70993.0 -15.374255  16.867753      NaN   \n",
       "13941122    13941122  13941119    54822.0 -15.375659  16.791961      NaN   \n",
       "13941123    13941123  13941114      901.0 -14.875661  16.785873      NaN   \n",
       "\n",
       "                  NN  HD1mmF20  HD1mm  HD1dif  HD1pc  \\\n",
       "0         3507831609       NaN    NaN     NaN    NaN   \n",
       "1         3507831510       NaN    NaN     NaN    NaN   \n",
       "2         6188134127       NaN    NaN     NaN    NaN   \n",
       "3         3507831510       NaN    NaN     NaN    NaN   \n",
       "4         3507831510       NaN    NaN     NaN    NaN   \n",
       "...              ...       ...    ...     ...    ...   \n",
       "13941119  3651042474       NaN    NaN     NaN    NaN   \n",
       "13941120  3651042508       NaN    NaN     NaN    NaN   \n",
       "13941121  3651042501       NaN    NaN     NaN    NaN   \n",
       "13941122  3651042393       NaN    NaN     NaN    NaN   \n",
       "13941123  6323434997       NaN    NaN     NaN    NaN   \n",
       "\n",
       "                              geometry      ag_HD1F50  \n",
       "0         POINT (-16.458988 12.208646)      82.731533  \n",
       "1         POINT (-16.375655 12.208646)      89.090029  \n",
       "2         POINT (-16.208989 12.208646)      23.536822  \n",
       "3         POINT (-16.208492 12.152815)      89.090029  \n",
       "4         POINT (-16.294213 12.149487)      89.090029  \n",
       "...                                ...            ...  \n",
       "13941119  POINT (-15.291536 16.874047)  186216.753175  \n",
       "13941120  POINT (-15.292326 16.791961)  186208.602920  \n",
       "13941121  POINT (-15.374255 16.867753)  186214.098036  \n",
       "13941122  POINT (-15.375659 16.791961)  186200.567733  \n",
       "13941123  POINT (-14.875661 16.785873)  257953.468283  \n",
       "\n",
       "[13941124 rows x 13 columns]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agF50 = pd.merge(agF20, agF50, on='NN',how='left')\n",
    "agF50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "agF50['HD1mmF50'] = 0 # mm for multi-modal\n",
    "agF50['HD1mmF50'] = agF50['ag_HD1F50'] + agF50['NSnomax']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 14 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID_ag       int64  \n",
      " 2   val         float64\n",
      " 3   x           float64\n",
      " 4   y           float64\n",
      " 5   NSnomax     float64\n",
      " 6   NN          int64  \n",
      " 7   F20         float64\n",
      " 8   base        float64\n",
      " 9   dif_20b     float64\n",
      " 10  pc_20b      float64\n",
      " 11  geometry    object \n",
      " 12  ag_HD1F50   float64\n",
      " 13  F50         float64\n",
      "dtypes: float64(10), int64(3), object(1)\n",
      "memory usage: 1.6+ GB\n"
     ]
    }
   ],
   "source": [
    "# Rename and remove fields for clarity\n",
    "agF50.rename(columns={'HD1mmF20':'F20', 'HD1mm':'base', 'HD1dif':'dif_20b', 'HD1pc':'pc_20b', 'HD1mmF50':'F50'}, inplace=True)\n",
    "agF50.info()\n",
    "# base: travel times before project intervention, no flood event.\n",
    "# F20: travel times before project intervention, during 1 in 20-year flood event.\n",
    "# F50: travel times before project intervention, during 1 in 50-year flood event.\n",
    "# dif_20b: change in travel time from 'base' travel to a 1 in 20-year flood event. Positive number = travel time increased.\n",
    "# pc_20b: percent change in travel time from 'base' travel to a 1 in 20-year flood event. Positive number = travel time increased."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 16 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID_ag       int64  \n",
      " 2   val         float64\n",
      " 3   x           float64\n",
      " 4   y           float64\n",
      " 5   NSnomax     float64\n",
      " 6   NN          int64  \n",
      " 7   F20         float64\n",
      " 8   base        float64\n",
      " 9   dif_20b     float64\n",
      " 10  pc_20b      float64\n",
      " 11  geometry    object \n",
      " 12  ag_HD1F50   float64\n",
      " 13  F50         float64\n",
      " 14  dif_50b     float64\n",
      " 15  pc_50b      float64\n",
      "dtypes: float64(12), int64(3), object(1)\n",
      "memory usage: 1.8+ GB\n"
     ]
    }
   ],
   "source": [
    "# Dif and pc\n",
    "agF50['dif_50b'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "agF50['pc_50b'] = 0 # pc for percent change\n",
    "agF50['dif_50b'] = agF50['F50'] - agF50['base'] # Travel time is X minutes longer in flood conditions.\n",
    "agF50['pc_50b'] = agF50['dif_50b'] / agF50['base'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "agF50.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>F20</th>\n",
       "      <th>base</th>\n",
       "      <th>dif_20b</th>\n",
       "      <th>pc_20b</th>\n",
       "      <th>ag_HD1F50</th>\n",
       "      <th>F50</th>\n",
       "      <th>dif_50b</th>\n",
       "      <th>pc_50b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.318626e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.394082e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>1.219702e+02</td>\n",
       "      <td>-1.558418e+01</td>\n",
       "      <td>1.432788e+01</td>\n",
       "      <td>2.209557e+01</td>\n",
       "      <td>5.121305e+09</td>\n",
       "      <td>5.217945e+02</td>\n",
       "      <td>7.234398e+01</td>\n",
       "      <td>4.494505e+02</td>\n",
       "      <td>3.929297e+02</td>\n",
       "      <td>2.455770e+03</td>\n",
       "      <td>1.177547e+03</td>\n",
       "      <td>1.105203e+03</td>\n",
       "      <td>1.143876e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>4.024456e+06</td>\n",
       "      <td>4.024456e+06</td>\n",
       "      <td>3.760062e+03</td>\n",
       "      <td>9.437674e-01</td>\n",
       "      <td>9.172257e-01</td>\n",
       "      <td>2.796064e+01</td>\n",
       "      <td>2.262019e+09</td>\n",
       "      <td>5.061590e+03</td>\n",
       "      <td>4.196269e+01</td>\n",
       "      <td>5.052823e+03</td>\n",
       "      <td>2.803781e+03</td>\n",
       "      <td>1.741179e+04</td>\n",
       "      <td>8.076292e+03</td>\n",
       "      <td>8.066331e+03</td>\n",
       "      <td>6.148201e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>1.563126e-01</td>\n",
       "      <td>-1.748885e+01</td>\n",
       "      <td>1.204963e+01</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>2.214129e+07</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>-1.136868e-13</td>\n",
       "      <td>-1.000430e-13</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>-1.705303e-13</td>\n",
       "      <td>-1.000430e-13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.485281e+06</td>\n",
       "      <td>3.485281e+06</td>\n",
       "      <td>4.293236e+01</td>\n",
       "      <td>-1.622061e+01</td>\n",
       "      <td>1.368601e+01</td>\n",
       "      <td>5.456861e+00</td>\n",
       "      <td>3.656862e+09</td>\n",
       "      <td>4.999677e+01</td>\n",
       "      <td>4.336144e+01</td>\n",
       "      <td>4.113212e-01</td>\n",
       "      <td>6.045770e-01</td>\n",
       "      <td>3.473214e+01</td>\n",
       "      <td>5.288514e+01</td>\n",
       "      <td>8.248076e-01</td>\n",
       "      <td>1.449902e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>7.008440e+01</td>\n",
       "      <td>-1.578565e+01</td>\n",
       "      <td>1.436783e+01</td>\n",
       "      <td>1.162371e+01</td>\n",
       "      <td>4.519739e+09</td>\n",
       "      <td>8.137713e+01</td>\n",
       "      <td>6.605466e+01</td>\n",
       "      <td>2.693749e+00</td>\n",
       "      <td>4.421227e+00</td>\n",
       "      <td>6.567086e+01</td>\n",
       "      <td>8.743365e+01</td>\n",
       "      <td>8.217781e+00</td>\n",
       "      <td>1.204507e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.045584e+07</td>\n",
       "      <td>1.045584e+07</td>\n",
       "      <td>1.013998e+02</td>\n",
       "      <td>-1.516850e+01</td>\n",
       "      <td>1.491868e+01</td>\n",
       "      <td>2.869398e+01</td>\n",
       "      <td>6.968809e+09</td>\n",
       "      <td>1.274517e+02</td>\n",
       "      <td>9.401759e+01</td>\n",
       "      <td>3.236852e+01</td>\n",
       "      <td>3.900011e+01</td>\n",
       "      <td>1.380510e+02</td>\n",
       "      <td>1.418556e+02</td>\n",
       "      <td>5.145242e+01</td>\n",
       "      <td>7.072402e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>6.939670e+06</td>\n",
       "      <td>-1.115399e+01</td>\n",
       "      <td>1.687405e+01</td>\n",
       "      <td>1.713278e+03</td>\n",
       "      <td>9.285641e+09</td>\n",
       "      <td>2.218282e+05</td>\n",
       "      <td>1.801212e+03</td>\n",
       "      <td>2.213788e+05</td>\n",
       "      <td>2.456662e+05</td>\n",
       "      <td>3.379222e+05</td>\n",
       "      <td>3.379440e+05</td>\n",
       "      <td>3.374946e+05</td>\n",
       "      <td>4.890921e+05</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0         ID_ag           val             x             y  \\\n",
       "count  1.394112e+07  1.394112e+07  1.394112e+07  1.394112e+07  1.394112e+07   \n",
       "mean   6.970562e+06  6.970562e+06  1.219702e+02 -1.558418e+01  1.432788e+01   \n",
       "std    4.024456e+06  4.024456e+06  3.760062e+03  9.437674e-01  9.172257e-01   \n",
       "min    0.000000e+00  0.000000e+00  1.563126e-01 -1.748885e+01  1.204963e+01   \n",
       "25%    3.485281e+06  3.485281e+06  4.293236e+01 -1.622061e+01  1.368601e+01   \n",
       "50%    6.970562e+06  6.970562e+06  7.008440e+01 -1.578565e+01  1.436783e+01   \n",
       "75%    1.045584e+07  1.045584e+07  1.013998e+02 -1.516850e+01  1.491868e+01   \n",
       "max    1.394112e+07  1.394112e+07  6.939670e+06 -1.115399e+01  1.687405e+01   \n",
       "\n",
       "            NSnomax            NN           F20          base       dif_20b  \\\n",
       "count  1.318626e+07  1.394112e+07  1.318596e+07  1.318596e+07  1.318596e+07   \n",
       "mean   2.209557e+01  5.121305e+09  5.217945e+02  7.234398e+01  4.494505e+02   \n",
       "std    2.796064e+01  2.262019e+09  5.061590e+03  4.196269e+01  5.052823e+03   \n",
       "min    0.000000e+00  2.214129e+07  0.000000e+00  0.000000e+00 -1.136868e-13   \n",
       "25%    5.456861e+00  3.656862e+09  4.999677e+01  4.336144e+01  4.113212e-01   \n",
       "50%    1.162371e+01  4.519739e+09  8.137713e+01  6.605466e+01  2.693749e+00   \n",
       "75%    2.869398e+01  6.968809e+09  1.274517e+02  9.401759e+01  3.236852e+01   \n",
       "max    1.713278e+03  9.285641e+09  2.218282e+05  1.801212e+03  2.213788e+05   \n",
       "\n",
       "             pc_20b     ag_HD1F50           F50       dif_50b        pc_50b  \n",
       "count  1.318596e+07  1.394082e+07  1.318596e+07  1.318596e+07  1.318596e+07  \n",
       "mean   3.929297e+02  2.455770e+03  1.177547e+03  1.105203e+03  1.143876e+03  \n",
       "std    2.803781e+03  1.741179e+04  8.076292e+03  8.066331e+03  6.148201e+03  \n",
       "min   -1.000430e-13  0.000000e+00  0.000000e+00 -1.705303e-13 -1.000430e-13  \n",
       "25%    6.045770e-01  3.473214e+01  5.288514e+01  8.248076e-01  1.449902e+00  \n",
       "50%    4.421227e+00  6.567086e+01  8.743365e+01  8.217781e+00  1.204507e+01  \n",
       "75%    3.900011e+01  1.380510e+02  1.418556e+02  5.145242e+01  7.072402e+01  \n",
       "max    2.456662e+05  3.379222e+05  3.379440e+05  3.374946e+05  4.890921e+05  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(agF50.describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [],
   "source": [
    "agF50.to_csv(os.path.join(out_pth, 'ag_to_HDurban_mmF50.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 625851 entries, 0 to 625850\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   Unnamed: 0  625851 non-null  int64  \n",
      " 1   NN          625851 non-null  int64  \n",
      " 2   ag_HD1F10   625841 non-null  float64\n",
      " 3   ag_HD2F10   625851 non-null  float64\n",
      " 4   ag_HD3F10   625851 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 23.9 MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# If reloading:\n",
    "agF10 = os.path.join(out_pth, \"ag_to_HDurban_flood10.csv\")\n",
    "agF10 = pd.read_csv(agF10)\n",
    "print(agF10.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>ag_HD1F10</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3507831609</td>\n",
       "      <td>82.054823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3507831510</td>\n",
       "      <td>88.413319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>6188134127</td>\n",
       "      <td>23.536822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>8631201421</td>\n",
       "      <td>94.721696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>8598305977</td>\n",
       "      <td>51.737958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625846</th>\n",
       "      <td>3651042474</td>\n",
       "      <td>95480.310462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625847</th>\n",
       "      <td>3651042508</td>\n",
       "      <td>95472.160207</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625848</th>\n",
       "      <td>3651042501</td>\n",
       "      <td>95477.655323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625849</th>\n",
       "      <td>3651042393</td>\n",
       "      <td>95464.125021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>625850</th>\n",
       "      <td>6323434997</td>\n",
       "      <td>95493.983780</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>625851 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                NN     ag_HD1F10\n",
       "0       3507831609     82.054823\n",
       "1       3507831510     88.413319\n",
       "2       6188134127     23.536822\n",
       "3       8631201421     94.721696\n",
       "4       8598305977     51.737958\n",
       "...            ...           ...\n",
       "625846  3651042474  95480.310462\n",
       "625847  3651042508  95472.160207\n",
       "625848  3651042501  95477.655323\n",
       "625849  3651042393  95464.125021\n",
       "625850  6323434997  95493.983780\n",
       "\n",
       "[625851 rows x 2 columns]"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agF10 = agF10[['NN', 'ag_HD1F10']]\n",
    "agF10"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 17 columns):\n",
      " #   Column        Dtype  \n",
      "---  ------        -----  \n",
      " 0   Unnamed: 0    int64  \n",
      " 1   Unnamed: 0.1  int64  \n",
      " 2   ID_ag         int64  \n",
      " 3   val           float64\n",
      " 4   x             float64\n",
      " 5   y             float64\n",
      " 6   NSnomax       float64\n",
      " 7   NN            int64  \n",
      " 8   F20           float64\n",
      " 9   base          float64\n",
      " 10  dif_20b       float64\n",
      " 11  pc_20b        float64\n",
      " 12  geometry      object \n",
      " 13  ag_HD1F50     float64\n",
      " 14  F50           float64\n",
      " 15  dif_50b       float64\n",
      " 16  pc_50b        float64\n",
      "dtypes: float64(12), int64(4), object(1)\n",
      "memory usage: 1.8+ GB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "agF50 = os.path.join(out_pth, \"ag_to_HDurban_mmF50.csv\")\n",
    "agF50 = pd.read_csv(agF50)\n",
    "print(agF50.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>val</th>\n",
       "      <th>x</th>\n",
       "      <th>y</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>NN</th>\n",
       "      <th>F20</th>\n",
       "      <th>base</th>\n",
       "      <th>dif_20b</th>\n",
       "      <th>pc_20b</th>\n",
       "      <th>geometry</th>\n",
       "      <th>ag_HD1F50</th>\n",
       "      <th>F50</th>\n",
       "      <th>dif_50b</th>\n",
       "      <th>pc_50b</th>\n",
       "      <th>ag_HD1F10</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13941103</td>\n",
       "      <td>2008851.0</td>\n",
       "      <td>-16.458988</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831609</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.458988 12.208646)</td>\n",
       "      <td>82.731533</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>82.054823</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>13941122</td>\n",
       "      <td>94605.0</td>\n",
       "      <td>-16.375655</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.375655 12.208646)</td>\n",
       "      <td>89.090029</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>88.413319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>13941017</td>\n",
       "      <td>708413.0</td>\n",
       "      <td>-16.208989</td>\n",
       "      <td>12.208646</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6188134127</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.208989 12.208646)</td>\n",
       "      <td>23.536822</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>23.536822</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>13941036</td>\n",
       "      <td>315719.0</td>\n",
       "      <td>-16.208492</td>\n",
       "      <td>12.152815</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.208492 12.152815)</td>\n",
       "      <td>89.090029</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>88.413319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>13941044</td>\n",
       "      <td>375288.0</td>\n",
       "      <td>-16.294213</td>\n",
       "      <td>12.149487</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3507831510</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-16.294213 12.149487)</td>\n",
       "      <td>89.090029</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>88.413319</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941119</th>\n",
       "      <td>13941119</td>\n",
       "      <td>13941119</td>\n",
       "      <td>13941081</td>\n",
       "      <td>87412.0</td>\n",
       "      <td>-15.291536</td>\n",
       "      <td>16.874047</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042474</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.291536 16.874047)</td>\n",
       "      <td>186216.753175</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95480.310462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941120</th>\n",
       "      <td>13941120</td>\n",
       "      <td>13941120</td>\n",
       "      <td>13941093</td>\n",
       "      <td>21660.0</td>\n",
       "      <td>-15.292326</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042508</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.292326 16.791961)</td>\n",
       "      <td>186208.602920</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95472.160207</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941121</th>\n",
       "      <td>13941121</td>\n",
       "      <td>13941121</td>\n",
       "      <td>13941109</td>\n",
       "      <td>70993.0</td>\n",
       "      <td>-15.374255</td>\n",
       "      <td>16.867753</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042501</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.374255 16.867753)</td>\n",
       "      <td>186214.098036</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95477.655323</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941122</th>\n",
       "      <td>13941122</td>\n",
       "      <td>13941122</td>\n",
       "      <td>13941119</td>\n",
       "      <td>54822.0</td>\n",
       "      <td>-15.375659</td>\n",
       "      <td>16.791961</td>\n",
       "      <td>NaN</td>\n",
       "      <td>3651042393</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-15.375659 16.791961)</td>\n",
       "      <td>186200.567733</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95464.125021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13941123</th>\n",
       "      <td>13941123</td>\n",
       "      <td>13941123</td>\n",
       "      <td>13941114</td>\n",
       "      <td>901.0</td>\n",
       "      <td>-14.875661</td>\n",
       "      <td>16.785873</td>\n",
       "      <td>NaN</td>\n",
       "      <td>6323434997</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>POINT (-14.875661 16.785873)</td>\n",
       "      <td>257953.468283</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>95493.983780</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>13941124 rows × 18 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          Unnamed: 0  Unnamed: 0.1     ID_ag        val          x          y  \\\n",
       "0                  0             0  13941103  2008851.0 -16.458988  12.208646   \n",
       "1                  1             1  13941122    94605.0 -16.375655  12.208646   \n",
       "2                  2             2  13941017   708413.0 -16.208989  12.208646   \n",
       "3                  3             3  13941036   315719.0 -16.208492  12.152815   \n",
       "4                  4             4  13941044   375288.0 -16.294213  12.149487   \n",
       "...              ...           ...       ...        ...        ...        ...   \n",
       "13941119    13941119      13941119  13941081    87412.0 -15.291536  16.874047   \n",
       "13941120    13941120      13941120  13941093    21660.0 -15.292326  16.791961   \n",
       "13941121    13941121      13941121  13941109    70993.0 -15.374255  16.867753   \n",
       "13941122    13941122      13941122  13941119    54822.0 -15.375659  16.791961   \n",
       "13941123    13941123      13941123  13941114      901.0 -14.875661  16.785873   \n",
       "\n",
       "          NSnomax          NN  F20  base  dif_20b  pc_20b  \\\n",
       "0             NaN  3507831609  NaN   NaN      NaN     NaN   \n",
       "1             NaN  3507831510  NaN   NaN      NaN     NaN   \n",
       "2             NaN  6188134127  NaN   NaN      NaN     NaN   \n",
       "3             NaN  3507831510  NaN   NaN      NaN     NaN   \n",
       "4             NaN  3507831510  NaN   NaN      NaN     NaN   \n",
       "...           ...         ...  ...   ...      ...     ...   \n",
       "13941119      NaN  3651042474  NaN   NaN      NaN     NaN   \n",
       "13941120      NaN  3651042508  NaN   NaN      NaN     NaN   \n",
       "13941121      NaN  3651042501  NaN   NaN      NaN     NaN   \n",
       "13941122      NaN  3651042393  NaN   NaN      NaN     NaN   \n",
       "13941123      NaN  6323434997  NaN   NaN      NaN     NaN   \n",
       "\n",
       "                              geometry      ag_HD1F50  F50  dif_50b  pc_50b  \\\n",
       "0         POINT (-16.458988 12.208646)      82.731533  NaN      NaN     NaN   \n",
       "1         POINT (-16.375655 12.208646)      89.090029  NaN      NaN     NaN   \n",
       "2         POINT (-16.208989 12.208646)      23.536822  NaN      NaN     NaN   \n",
       "3         POINT (-16.208492 12.152815)      89.090029  NaN      NaN     NaN   \n",
       "4         POINT (-16.294213 12.149487)      89.090029  NaN      NaN     NaN   \n",
       "...                                ...            ...  ...      ...     ...   \n",
       "13941119  POINT (-15.291536 16.874047)  186216.753175  NaN      NaN     NaN   \n",
       "13941120  POINT (-15.292326 16.791961)  186208.602920  NaN      NaN     NaN   \n",
       "13941121  POINT (-15.374255 16.867753)  186214.098036  NaN      NaN     NaN   \n",
       "13941122  POINT (-15.375659 16.791961)  186200.567733  NaN      NaN     NaN   \n",
       "13941123  POINT (-14.875661 16.785873)  257953.468283  NaN      NaN     NaN   \n",
       "\n",
       "             ag_HD1F10  \n",
       "0            82.054823  \n",
       "1            88.413319  \n",
       "2            23.536822  \n",
       "3            88.413319  \n",
       "4            88.413319  \n",
       "...                ...  \n",
       "13941119  95480.310462  \n",
       "13941120  95472.160207  \n",
       "13941121  95477.655323  \n",
       "13941122  95464.125021  \n",
       "13941123  95493.983780  \n",
       "\n",
       "[13941124 rows x 18 columns]"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "agF50 = pd.merge(agF50, agF10, on='NN',how='left')\n",
    "agF50"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "agF50['F10'] = 0 # mm for multi-modal\n",
    "agF50['F10'] = agF50['ag_HD1F10'] + agF50['NSnomax']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 21 columns):\n",
      " #   Column        Dtype  \n",
      "---  ------        -----  \n",
      " 0   Unnamed: 0    int64  \n",
      " 1   Unnamed: 0.1  int64  \n",
      " 2   ID_ag         int64  \n",
      " 3   val           float64\n",
      " 4   x             float64\n",
      " 5   y             float64\n",
      " 6   NSnomax       float64\n",
      " 7   NN            int64  \n",
      " 8   F20           float64\n",
      " 9   base          float64\n",
      " 10  dif_20b       float64\n",
      " 11  pc_20b        float64\n",
      " 12  geometry      object \n",
      " 13  ag_HD1F50     float64\n",
      " 14  F50           float64\n",
      " 15  dif_50b       float64\n",
      " 16  pc_50b        float64\n",
      " 17  ag_HD1F10     float64\n",
      " 18  F10           float64\n",
      " 19  dif_10b       float64\n",
      " 20  pc_10b        float64\n",
      "dtypes: float64(16), int64(4), object(1)\n",
      "memory usage: 2.3+ GB\n"
     ]
    }
   ],
   "source": [
    "# Dif and pc\n",
    "agF50['dif_10b'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "agF50['pc_10b'] = 0 # pc for percent change\n",
    "agF50['dif_10b'] = agF50['F10'] - agF50['base'] # Travel time is X minutes longer in flood conditions.\n",
    "agF50['pc_10b'] = agF50['dif_10b'] / agF50['base'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "agF50.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 16 columns):\n",
      " #   Column   Dtype  \n",
      "---  ------   -----  \n",
      " 0   ID_ag    int64  \n",
      " 1   val      float64\n",
      " 2   x        float64\n",
      " 3   y        float64\n",
      " 4   NSnomax  float64\n",
      " 5   NN       int64  \n",
      " 6   F20      float64\n",
      " 7   base     float64\n",
      " 8   dif_20b  float64\n",
      " 9   pc_20b   float64\n",
      " 10  F50      float64\n",
      " 11  dif_50b  float64\n",
      " 12  pc_50b   float64\n",
      " 13  F10      float64\n",
      " 14  dif_10b  float64\n",
      " 15  pc_10b   float64\n",
      "dtypes: float64(14), int64(2)\n",
      "memory usage: 1.8 GB\n"
     ]
    }
   ],
   "source": [
    "# Remove legacy variables.\n",
    "agF50 = agF50.drop(columns=['Unnamed: 0', 'Unnamed: 0.1', 'ag_HD1F50', 'geometry', 'ag_HD1F10'])\n",
    "agF50.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [],
   "source": [
    "agF50.to_csv(os.path.join(out_pth, 'ag_to_HDurban_mmF10.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Tue Dec 28 14:46:55 2021\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "crs = \"EPSG:4326\"\n",
    "geometry = [Point(xy) for xy in zip(agF50.x, agF50.y)]\n",
    "agF50 = GeoDataFrame(agF50, crs=crs, geometry=geometry) \n",
    "agF50.to_file(driver='ESRI Shapefile', filename='C:/Users/wb527163/GEO-Cdrive-Grace/SEN-Cdrive/ag_to_HDurban_base-F10-20-50.shp') \n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Hamlets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 125886 entries, 0 to 125885\n",
      "Data columns (total 5 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   GlobalID   125886 non-null  object \n",
      " 2   HD1        125886 non-null  float64\n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   HD1mm      125552 non-null  float64\n",
      "dtypes: float64(3), object(2)\n",
      "memory usage: 4.8+ MB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 125886 entries, 0 to 125885\n",
      "Data columns (total 10 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   Unnamed: 0  125886 non-null  int64  \n",
      " 1   Unnamed_ 0  125886 non-null  int64  \n",
      " 2   mgrs_code   125886 non-null  object \n",
      " 3   type        125886 non-null  object \n",
      " 4   GlobalID    125886 non-null  object \n",
      " 5   Shape_Leng  125886 non-null  float64\n",
      " 6   Shape_Area  125886 non-null  float64\n",
      " 7   geometry    125886 non-null  object \n",
      " 8   NN          125886 non-null  int64  \n",
      " 9   NN_dist     125886 non-null  float64\n",
      "dtypes: float64(3), int64(3), object(4)\n",
      "memory usage: 9.6+ MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "base_CD = os.path.join(out_pth, \"fromhamlet_drive-CD.csv\") # Contains base travel time, and the cost-distance travel from origin to road node.\n",
    "base_CD = pd.read_csv(base_CD)\n",
    "hamlet_snap = os.path.join(out_pth, \"hamlet_snap.csv\") \n",
    "hamlet_snap = pd.read_csv(hamlet_snap)\n",
    "\n",
    "print(base_CD.info())\n",
    "print(hamlet_snap.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 61230 entries, 0 to 61229\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   Unnamed: 0  61230 non-null  int64  \n",
      " 1   NN          61230 non-null  int64  \n",
      " 2   ha_HD1F10   61227 non-null  float64\n",
      " 3   ha_HD2F10   61230 non-null  float64\n",
      " 4   ha_HD3F10   61230 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 2.3 MB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 61230 entries, 0 to 61229\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   Unnamed: 0  61230 non-null  int64  \n",
      " 1   NN          61230 non-null  int64  \n",
      " 2   ha_HD1F20   61227 non-null  float64\n",
      " 3   ha_HD2F20   61230 non-null  float64\n",
      " 4   ha_HD3F20   61230 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 2.3 MB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 61230 entries, 0 to 61229\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count  Dtype  \n",
      "---  ------      --------------  -----  \n",
      " 0   Unnamed: 0  61230 non-null  int64  \n",
      " 1   NN          61230 non-null  int64  \n",
      " 2   ha_HD1F50   61227 non-null  float64\n",
      " 3   ha_HD2F50   61230 non-null  float64\n",
      " 4   ha_HD3F50   61230 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 2.3 MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# If reloading:\n",
    "hamletF10 = os.path.join(out_pth, \"hamlet_to_HDurban_flood10.csv\")\n",
    "hamletF10 = pd.read_csv(hamletF10)\n",
    "hamletF20 = os.path.join(out_pth, \"hamlet_to_HDurban_flood20.csv\")\n",
    "hamletF20 = pd.read_csv(hamletF20)\n",
    "hamletF50 = os.path.join(out_pth, \"hamlet_to_HDurban_flood50.csv\")\n",
    "hamletF50 = pd.read_csv(hamletF50)\n",
    "\n",
    "print(hamletF10.info())\n",
    "print(hamletF20.info())\n",
    "print(hamletF50.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "61230\n",
      "61230\n",
      "61230\n",
      "61230\n",
      "125886\n",
      "125886\n"
     ]
    }
   ],
   "source": [
    "# Make sure the ID is unique and fully matched\n",
    "list1 = list(hamletF50['NN'].unique())\n",
    "list2 = list(hamletF20['NN'].unique())\n",
    "list3 = list(hamletF10['NN'].unique())\n",
    "list4 = list(hamlet_snap['NN'].unique())\n",
    "print(len(list1))\n",
    "print(len(list2))\n",
    "print(len(list3))\n",
    "print(len(list4))\n",
    "\n",
    "list5 = list(hamlet_snap['mgrs_code'].unique())\n",
    "list6 = list(base_CD['mgrs_code'].unique())\n",
    "print(len(list5))\n",
    "print(len(list6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           NN  ha_HD1F10\n",
      "0  7761872870  62.891073\n",
      "1  7761872869  62.872513\n",
      "2  6442044321  61.862681\n",
      "3  2142496418  63.316203\n",
      "4  2142496429  63.698577\n",
      "\n",
      "           NN  ha_HD1F20\n",
      "0  7761872870  63.403221\n",
      "1  7761872869  63.384661\n",
      "2  6442044321  62.374829\n",
      "3  2142496418  63.828351\n",
      "4  2142496429  64.210725\n",
      "\n",
      "           NN  ha_HD1F50\n",
      "0  7761872870  65.474196\n",
      "1  7761872869  65.455636\n",
      "2  6442044321  64.445804\n",
      "3  2142496418  65.899326\n",
      "4  2142496429  66.281700\n",
      "\n",
      "      mgrs_code                                        geometry          NN\n",
      "0  28PCU1265_01  POINT (-16.721473282454415 12.348636090165247)  7761872870\n",
      "1  28PCU1365_01   POINT (-16.716456507935607 12.34788723564788)  7761872869\n",
      "2  28PCU1365_02  POINT (-16.713855008830738 12.350880992129111)  6442044321\n",
      "3  28PCU1566_01  POINT (-16.701275174874546 12.355585269999269)  2142496418\n",
      "4  28PCU1566_02  POINT (-16.698773736706396 12.356804484409668)  2142496429\n",
      "\n",
      "      mgrs_code  NSnomax  HD1mm\n",
      "0  28PCU1265_01      NaN    NaN\n",
      "1  28PCU1365_01      NaN    NaN\n",
      "2  28PCU1365_02      NaN    NaN\n",
      "3  28PCU1566_01      NaN    NaN\n",
      "4  28PCU1566_02      NaN    NaN\n",
      "\n"
     ]
    }
   ],
   "source": [
    "hamletF10 = hamletF10[['NN', 'ha_HD1F10']]\n",
    "hamletF20 = hamletF20[['NN', 'ha_HD1F20']]\n",
    "hamletF50 = hamletF50[['NN', 'ha_HD1F50']]\n",
    "hamlet_snap = hamlet_snap[['mgrs_code', 'geometry', 'NN']]\n",
    "base_CD = base_CD[['mgrs_code', 'NSnomax', 'HD1mm']]\n",
    "\n",
    "print(hamletF10.head(), end='\\n\\n')\n",
    "print(hamletF20.head(), end='\\n\\n')\n",
    "print(hamletF50.head(), end='\\n\\n')\n",
    "print(hamlet_snap.head(), end='\\n\\n')\n",
    "print(base_CD.head(), end='\\n\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 5 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   geometry   125886 non-null  object \n",
      " 2   NN         125886 non-null  int64  \n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   HD1mm      125552 non-null  float64\n",
      "dtypes: float64(2), int64(1), object(2)\n",
      "memory usage: 5.8+ MB\n",
      "None\n",
      "\n",
      "      mgrs_code                                        geometry          NN  \\\n",
      "0  28PCU1265_01  POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
      "1  28PCU1365_01   POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
      "2  28PCU1365_02  POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
      "3  28PCU1566_01  POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
      "4  28PCU1566_02  POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
      "\n",
      "   NSnomax  HD1mm  \n",
      "0      NaN    NaN  \n",
      "1      NaN    NaN  \n",
      "2      NaN    NaN  \n",
      "3      NaN    NaN  \n",
      "4      NaN    NaN  \n"
     ]
    }
   ],
   "source": [
    "hamlet_all = pd.merge(hamlet_snap, base_CD, on='mgrs_code',how='left')\n",
    "print(hamlet_all.info(), end='\\n\\n')\n",
    "print(hamlet_all.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 6 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   geometry   125886 non-null  object \n",
      " 2   NN         125886 non-null  int64  \n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   HD1mm      125552 non-null  float64\n",
      " 5   ha_HD1F10  125883 non-null  float64\n",
      "dtypes: float64(3), int64(1), object(2)\n",
      "memory usage: 6.7+ MB\n",
      "None\n",
      "\n",
      "      mgrs_code                                        geometry          NN  \\\n",
      "0  28PCU1265_01  POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
      "1  28PCU1365_01   POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
      "2  28PCU1365_02  POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
      "3  28PCU1566_01  POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
      "4  28PCU1566_02  POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
      "\n",
      "   NSnomax  HD1mm  ha_HD1F10  \n",
      "0      NaN    NaN  62.891073  \n",
      "1      NaN    NaN  62.872513  \n",
      "2      NaN    NaN  61.862681  \n",
      "3      NaN    NaN  63.316203  \n",
      "4      NaN    NaN  63.698577  \n"
     ]
    }
   ],
   "source": [
    "hamlet_all = pd.merge(hamlet_all, hamletF10, on='NN',how='left')\n",
    "print(hamlet_all.info(), end='\\n\\n')\n",
    "print(hamlet_all.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 7 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   geometry   125886 non-null  object \n",
      " 2   NN         125886 non-null  int64  \n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   HD1mm      125552 non-null  float64\n",
      " 5   ha_HD1F10  125883 non-null  float64\n",
      " 6   ha_HD1F20  125883 non-null  float64\n",
      "dtypes: float64(4), int64(1), object(2)\n",
      "memory usage: 7.7+ MB\n",
      "None\n",
      "\n",
      "      mgrs_code                                        geometry          NN  \\\n",
      "0  28PCU1265_01  POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
      "1  28PCU1365_01   POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
      "2  28PCU1365_02  POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
      "3  28PCU1566_01  POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
      "4  28PCU1566_02  POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
      "\n",
      "   NSnomax  HD1mm  ha_HD1F10  ha_HD1F20  \n",
      "0      NaN    NaN  62.891073  63.403221  \n",
      "1      NaN    NaN  62.872513  63.384661  \n",
      "2      NaN    NaN  61.862681  62.374829  \n",
      "3      NaN    NaN  63.316203  63.828351  \n",
      "4      NaN    NaN  63.698577  64.210725  \n"
     ]
    }
   ],
   "source": [
    "hamlet_all = pd.merge(hamlet_all, hamletF20, on='NN',how='left')\n",
    "print(hamlet_all.info(), end='\\n\\n')\n",
    "print(hamlet_all.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 8 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   geometry   125886 non-null  object \n",
      " 2   NN         125886 non-null  int64  \n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   HD1mm      125552 non-null  float64\n",
      " 5   ha_HD1F10  125883 non-null  float64\n",
      " 6   ha_HD1F20  125883 non-null  float64\n",
      " 7   ha_HD1F50  125883 non-null  float64\n",
      "dtypes: float64(5), int64(1), object(2)\n",
      "memory usage: 8.6+ MB\n",
      "None\n",
      "\n",
      "      mgrs_code                                        geometry          NN  \\\n",
      "0  28PCU1265_01  POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
      "1  28PCU1365_01   POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
      "2  28PCU1365_02  POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
      "3  28PCU1566_01  POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
      "4  28PCU1566_02  POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
      "\n",
      "   NSnomax  HD1mm  ha_HD1F10  ha_HD1F20  ha_HD1F50  \n",
      "0      NaN    NaN  62.891073  63.403221  65.474196  \n",
      "1      NaN    NaN  62.872513  63.384661  65.455636  \n",
      "2      NaN    NaN  61.862681  62.374829  64.445804  \n",
      "3      NaN    NaN  63.316203  63.828351  65.899326  \n",
      "4      NaN    NaN  63.698577  64.210725  66.281700  \n"
     ]
    }
   ],
   "source": [
    "hamlet_all = pd.merge(hamlet_all, hamletF50, on='NN',how='left')\n",
    "print(hamlet_all.info(), end='\\n\\n')\n",
    "print(hamlet_all.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mgrs_code</th>\n",
       "      <th>geometry</th>\n",
       "      <th>NN</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>base</th>\n",
       "      <th>ha_HD1F10</th>\n",
       "      <th>ha_HD1F20</th>\n",
       "      <th>ha_HD1F50</th>\n",
       "      <th>F10</th>\n",
       "      <th>F20</th>\n",
       "      <th>F50</th>\n",
       "      <th>dif_10b</th>\n",
       "      <th>pc_10b</th>\n",
       "      <th>dif_20b</th>\n",
       "      <th>pc_20b</th>\n",
       "      <th>dif_50b</th>\n",
       "      <th>pc_50b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>28PCU1265_01</td>\n",
       "      <td>POINT (-16.721473282454415 12.348636090165247)</td>\n",
       "      <td>7761872870</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>62.891073</td>\n",
       "      <td>63.403221</td>\n",
       "      <td>65.474196</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>28PCU1365_01</td>\n",
       "      <td>POINT (-16.716456507935607 12.34788723564788)</td>\n",
       "      <td>7761872869</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>62.872513</td>\n",
       "      <td>63.384661</td>\n",
       "      <td>65.455636</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28PCU1365_02</td>\n",
       "      <td>POINT (-16.713855008830738 12.350880992129111)</td>\n",
       "      <td>6442044321</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>61.862681</td>\n",
       "      <td>62.374829</td>\n",
       "      <td>64.445804</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>28PCU1566_01</td>\n",
       "      <td>POINT (-16.701275174874546 12.355585269999269)</td>\n",
       "      <td>2142496418</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>63.316203</td>\n",
       "      <td>63.828351</td>\n",
       "      <td>65.899326</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28PCU1566_02</td>\n",
       "      <td>POINT (-16.698773736706396 12.356804484409668)</td>\n",
       "      <td>2142496429</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>63.698577</td>\n",
       "      <td>64.210725</td>\n",
       "      <td>66.281700</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      mgrs_code                                        geometry          NN  \\\n",
       "0  28PCU1265_01  POINT (-16.721473282454415 12.348636090165247)  7761872870   \n",
       "1  28PCU1365_01   POINT (-16.716456507935607 12.34788723564788)  7761872869   \n",
       "2  28PCU1365_02  POINT (-16.713855008830738 12.350880992129111)  6442044321   \n",
       "3  28PCU1566_01  POINT (-16.701275174874546 12.355585269999269)  2142496418   \n",
       "4  28PCU1566_02  POINT (-16.698773736706396 12.356804484409668)  2142496429   \n",
       "\n",
       "   NSnomax  base  ha_HD1F10  ha_HD1F20  ha_HD1F50  F10  F20  F50  dif_10b  \\\n",
       "0      NaN   NaN  62.891073  63.403221  65.474196  NaN  NaN  NaN      NaN   \n",
       "1      NaN   NaN  62.872513  63.384661  65.455636  NaN  NaN  NaN      NaN   \n",
       "2      NaN   NaN  61.862681  62.374829  64.445804  NaN  NaN  NaN      NaN   \n",
       "3      NaN   NaN  63.316203  63.828351  65.899326  NaN  NaN  NaN      NaN   \n",
       "4      NaN   NaN  63.698577  64.210725  66.281700  NaN  NaN  NaN      NaN   \n",
       "\n",
       "   pc_10b  dif_20b  pc_20b  dif_50b  pc_50b  \n",
       "0     NaN      NaN     NaN      NaN     NaN  \n",
       "1     NaN      NaN     NaN      NaN     NaN  \n",
       "2     NaN      NaN     NaN      NaN     NaN  \n",
       "3     NaN      NaN     NaN      NaN     NaN  \n",
       "4     NaN      NaN     NaN      NaN     NaN  "
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hamlet_all.rename(columns={'HD1mm':'base'}, inplace=True)\n",
    "\n",
    "hamlet_all['F10'] = 0\n",
    "hamlet_all['F10'] = hamlet_all['ha_HD1F10'] + hamlet_all['NSnomax']\n",
    "hamlet_all['F20'] = 0\n",
    "hamlet_all['F20'] = hamlet_all['ha_HD1F20'] + hamlet_all['NSnomax']\n",
    "hamlet_all['F50'] = 0\n",
    "hamlet_all['F50'] = hamlet_all['ha_HD1F50'] + hamlet_all['NSnomax']\n",
    "\n",
    "hamlet_all['dif_10b'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "hamlet_all['pc_10b'] = 0 # pc for percent change\n",
    "hamlet_all['dif_10b'] = hamlet_all['F10'] - hamlet_all['base'] # Travel time is X minutes longer in flood conditions.\n",
    "hamlet_all['pc_10b'] = hamlet_all['dif_10b'] / hamlet_all['base'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "\n",
    "hamlet_all['dif_20b'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "hamlet_all['pc_20b'] = 0 # pc for percent change\n",
    "hamlet_all['dif_20b'] = hamlet_all['F20'] - hamlet_all['base'] # Travel time is X minutes longer in flood conditions.\n",
    "hamlet_all['pc_20b'] = hamlet_all['dif_20b'] / hamlet_all['base'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "\n",
    "hamlet_all['dif_50b'] = 0 # dif for difference between the two travel times (in minutes)\n",
    "hamlet_all['pc_50b'] = 0 # pc for percent change\n",
    "hamlet_all['dif_50b'] = hamlet_all['F50'] - hamlet_all['base'] # Travel time is X minutes longer in flood conditions.\n",
    "hamlet_all['pc_50b'] = hamlet_all['dif_50b'] / hamlet_all['base'] * 100 # Travel time is X percent longer in flood conditions.\n",
    "\n",
    "hamlet_all.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 17 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   geometry   125886 non-null  object \n",
      " 2   NN         125886 non-null  int64  \n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   base       125552 non-null  float64\n",
      " 5   ha_HD1F10  125883 non-null  float64\n",
      " 6   ha_HD1F20  125883 non-null  float64\n",
      " 7   ha_HD1F50  125883 non-null  float64\n",
      " 8   F10        125549 non-null  float64\n",
      " 9   F20        125549 non-null  float64\n",
      " 10  F50        125549 non-null  float64\n",
      " 11  dif_10b    125549 non-null  float64\n",
      " 12  pc_10b     125548 non-null  float64\n",
      " 13  dif_20b    125549 non-null  float64\n",
      " 14  pc_20b     125548 non-null  float64\n",
      " 15  dif_50b    125549 non-null  float64\n",
      " 16  pc_50b     125548 non-null  float64\n",
      "dtypes: float64(14), int64(1), object(2)\n",
      "memory usage: 17.3+ MB\n"
     ]
    }
   ],
   "source": [
    "hamlet_all.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NN</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>base</th>\n",
       "      <th>ha_HD1F10</th>\n",
       "      <th>ha_HD1F20</th>\n",
       "      <th>ha_HD1F50</th>\n",
       "      <th>F10</th>\n",
       "      <th>F20</th>\n",
       "      <th>F50</th>\n",
       "      <th>dif_10b</th>\n",
       "      <th>pc_10b</th>\n",
       "      <th>dif_20b</th>\n",
       "      <th>pc_20b</th>\n",
       "      <th>dif_50b</th>\n",
       "      <th>pc_50b</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.258860e+05</td>\n",
       "      <td>125552.000000</td>\n",
       "      <td>125552.000000</td>\n",
       "      <td>125883.000000</td>\n",
       "      <td>125883.000000</td>\n",
       "      <td>125883.000000</td>\n",
       "      <td>125549.000000</td>\n",
       "      <td>125549.000000</td>\n",
       "      <td>125549.000000</td>\n",
       "      <td>1.255490e+05</td>\n",
       "      <td>1.255480e+05</td>\n",
       "      <td>1.255490e+05</td>\n",
       "      <td>1.255480e+05</td>\n",
       "      <td>1.255490e+05</td>\n",
       "      <td>1.255480e+05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>5.308692e+09</td>\n",
       "      <td>23.883785</td>\n",
       "      <td>96.106479</td>\n",
       "      <td>514.512566</td>\n",
       "      <td>1038.752648</td>\n",
       "      <td>2090.903028</td>\n",
       "      <td>521.785551</td>\n",
       "      <td>1035.056472</td>\n",
       "      <td>2069.402565</td>\n",
       "      <td>4.256771e+02</td>\n",
       "      <td>2.509807e+02</td>\n",
       "      <td>9.389480e+02</td>\n",
       "      <td>5.792683e+02</td>\n",
       "      <td>1.973294e+03</td>\n",
       "      <td>1.288904e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>2.359009e+09</td>\n",
       "      <td>31.678803</td>\n",
       "      <td>72.802323</td>\n",
       "      <td>4787.457434</td>\n",
       "      <td>8334.005393</td>\n",
       "      <td>13427.302266</td>\n",
       "      <td>4656.940001</td>\n",
       "      <td>8104.217997</td>\n",
       "      <td>13120.371573</td>\n",
       "      <td>4.644338e+03</td>\n",
       "      <td>2.149234e+03</td>\n",
       "      <td>8.090129e+03</td>\n",
       "      <td>3.797200e+03</td>\n",
       "      <td>1.310162e+04</td>\n",
       "      <td>6.527796e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>2.238784e+07</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-1.136868e-13</td>\n",
       "      <td>-5.250238e-14</td>\n",
       "      <td>-5.684342e-14</td>\n",
       "      <td>-5.250238e-14</td>\n",
       "      <td>-5.684342e-14</td>\n",
       "      <td>-5.327503e-14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.670522e+09</td>\n",
       "      <td>5.330675</td>\n",
       "      <td>43.628459</td>\n",
       "      <td>29.890441</td>\n",
       "      <td>31.590684</td>\n",
       "      <td>34.098236</td>\n",
       "      <td>45.576487</td>\n",
       "      <td>48.903011</td>\n",
       "      <td>51.569585</td>\n",
       "      <td>1.812162e-01</td>\n",
       "      <td>3.350923e-01</td>\n",
       "      <td>4.285746e-01</td>\n",
       "      <td>6.610646e-01</td>\n",
       "      <td>8.361956e-01</td>\n",
       "      <td>1.479530e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>4.635455e+09</td>\n",
       "      <td>11.602750</td>\n",
       "      <td>75.401475</td>\n",
       "      <td>53.324630</td>\n",
       "      <td>63.959088</td>\n",
       "      <td>70.909068</td>\n",
       "      <td>83.663933</td>\n",
       "      <td>94.022616</td>\n",
       "      <td>104.567724</td>\n",
       "      <td>1.693662e+00</td>\n",
       "      <td>2.130879e+00</td>\n",
       "      <td>5.376619e+00</td>\n",
       "      <td>6.068798e+00</td>\n",
       "      <td>1.593465e+01</td>\n",
       "      <td>1.700048e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>7.798350e+09</td>\n",
       "      <td>30.418225</td>\n",
       "      <td>128.243527</td>\n",
       "      <td>143.970177</td>\n",
       "      <td>178.066406</td>\n",
       "      <td>244.660680</td>\n",
       "      <td>168.715592</td>\n",
       "      <td>203.875029</td>\n",
       "      <td>280.531199</td>\n",
       "      <td>1.481516e+01</td>\n",
       "      <td>1.350110e+01</td>\n",
       "      <td>5.395519e+01</td>\n",
       "      <td>4.627568e+01</td>\n",
       "      <td>1.069482e+02</td>\n",
       "      <td>9.895040e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>9.285336e+09</td>\n",
       "      <td>871.611000</td>\n",
       "      <td>934.181380</td>\n",
       "      <td>142262.045968</td>\n",
       "      <td>215643.754476</td>\n",
       "      <td>329565.199228</td>\n",
       "      <td>142296.201168</td>\n",
       "      <td>215677.909676</td>\n",
       "      <td>329599.354428</td>\n",
       "      <td>1.418084e+05</td>\n",
       "      <td>1.868351e+05</td>\n",
       "      <td>2.151901e+05</td>\n",
       "      <td>1.870044e+05</td>\n",
       "      <td>3.291116e+05</td>\n",
       "      <td>3.038653e+05</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 NN        NSnomax           base      ha_HD1F10  \\\n",
       "count  1.258860e+05  125552.000000  125552.000000  125883.000000   \n",
       "mean   5.308692e+09      23.883785      96.106479     514.512566   \n",
       "std    2.359009e+09      31.678803      72.802323    4787.457434   \n",
       "min    2.238784e+07       0.000000       0.000000       0.000000   \n",
       "25%    3.670522e+09       5.330675      43.628459      29.890441   \n",
       "50%    4.635455e+09      11.602750      75.401475      53.324630   \n",
       "75%    7.798350e+09      30.418225     128.243527     143.970177   \n",
       "max    9.285336e+09     871.611000     934.181380  142262.045968   \n",
       "\n",
       "           ha_HD1F20      ha_HD1F50            F10            F20  \\\n",
       "count  125883.000000  125883.000000  125549.000000  125549.000000   \n",
       "mean     1038.752648    2090.903028     521.785551    1035.056472   \n",
       "std      8334.005393   13427.302266    4656.940001    8104.217997   \n",
       "min         0.000000       0.000000       0.000000       0.000000   \n",
       "25%        31.590684      34.098236      45.576487      48.903011   \n",
       "50%        63.959088      70.909068      83.663933      94.022616   \n",
       "75%       178.066406     244.660680     168.715592     203.875029   \n",
       "max    215643.754476  329565.199228  142296.201168  215677.909676   \n",
       "\n",
       "                 F50       dif_10b        pc_10b       dif_20b        pc_20b  \\\n",
       "count  125549.000000  1.255490e+05  1.255480e+05  1.255490e+05  1.255480e+05   \n",
       "mean     2069.402565  4.256771e+02  2.509807e+02  9.389480e+02  5.792683e+02   \n",
       "std     13120.371573  4.644338e+03  2.149234e+03  8.090129e+03  3.797200e+03   \n",
       "min         0.000000 -1.136868e-13 -5.250238e-14 -5.684342e-14 -5.250238e-14   \n",
       "25%        51.569585  1.812162e-01  3.350923e-01  4.285746e-01  6.610646e-01   \n",
       "50%       104.567724  1.693662e+00  2.130879e+00  5.376619e+00  6.068798e+00   \n",
       "75%       280.531199  1.481516e+01  1.350110e+01  5.395519e+01  4.627568e+01   \n",
       "max    329599.354428  1.418084e+05  1.868351e+05  2.151901e+05  1.870044e+05   \n",
       "\n",
       "            dif_50b        pc_50b  \n",
       "count  1.255490e+05  1.255480e+05  \n",
       "mean   1.973294e+03  1.288904e+03  \n",
       "std    1.310162e+04  6.527796e+03  \n",
       "min   -5.684342e-14 -5.327503e-14  \n",
       "25%    8.361956e-01  1.479530e+00  \n",
       "50%    1.593465e+01  1.700048e+01  \n",
       "75%    1.069482e+02  9.895040e+01  \n",
       "max    3.291116e+05  3.038653e+05  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(hamlet_all.describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 14 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   geometry   125886 non-null  object \n",
      " 2   NN         125886 non-null  int64  \n",
      " 3   NSnomax    125552 non-null  float64\n",
      " 4   base       125552 non-null  float64\n",
      " 5   F10        125549 non-null  float64\n",
      " 6   F20        125549 non-null  float64\n",
      " 7   F50        125549 non-null  float64\n",
      " 8   dif_10b    125549 non-null  float64\n",
      " 9   pc_10b     125548 non-null  float64\n",
      " 10  dif_20b    125549 non-null  float64\n",
      " 11  pc_20b     125548 non-null  float64\n",
      " 12  dif_50b    125549 non-null  float64\n",
      " 13  pc_50b     125548 non-null  float64\n",
      "dtypes: float64(11), int64(1), object(2)\n",
      "memory usage: 14.4+ MB\n"
     ]
    }
   ],
   "source": [
    "# Remove legacy variables.\n",
    "hamlet_all = hamlet_all.drop(columns=['ha_HD1F10', 'ha_HD1F20', 'ha_HD1F50'])\n",
    "hamlet_all.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 125886 entries, 0 to 125885\n",
      "Data columns (total 15 columns):\n",
      " #   Column     Non-Null Count   Dtype  \n",
      "---  ------     --------------   -----  \n",
      " 0   mgrs_code  125886 non-null  object \n",
      " 1   NN         125886 non-null  int64  \n",
      " 2   NSnomax    125552 non-null  float64\n",
      " 3   base       125552 non-null  float64\n",
      " 4   F10        125549 non-null  float64\n",
      " 5   F20        125549 non-null  float64\n",
      " 6   F50        125549 non-null  float64\n",
      " 7   dif_10b    125549 non-null  float64\n",
      " 8   pc_10b     125548 non-null  float64\n",
      " 9   dif_20b    125549 non-null  float64\n",
      " 10  pc_20b     125548 non-null  float64\n",
      " 11  dif_50b    125549 non-null  float64\n",
      " 12  pc_50b     125548 non-null  float64\n",
      " 13  X          125886 non-null  float64\n",
      " 14  Y          125886 non-null  float64\n",
      "dtypes: float64(13), int64(1), object(1)\n",
      "memory usage: 15.4+ MB\n"
     ]
    }
   ],
   "source": [
    "hamlet_all[\"geometry\"] = hamlet_all[\"geometry\"].astype('str')\n",
    "hamlet_all[\"geometry\"]  = hamlet_all[\"geometry\"] .str.strip('POINT ')\n",
    "hamlet_all[\"geometry\"]  = hamlet_all[\"geometry\"] .str.strip('()')\n",
    "XY = hamlet_all[\"geometry\"] .str.split(\" \", expand=True)\n",
    "hamlet_all[\"X\"] = XY[0]\n",
    "hamlet_all[\"Y\"] = XY[1]\n",
    "hamlet_all[\"X\"] = hamlet_all[\"X\"].astype('float')\n",
    "hamlet_all[\"Y\"] = hamlet_all[\"Y\"].astype('float')\n",
    "hamlet_all = hamlet_all.drop(columns=['geometry'])\n",
    "hamlet_all.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "hamlet_all.to_csv(os.path.join(out_pth, 'hamlet_to_HDurban_base-F10-20-50.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "start: Mon Jan  3 14:14:59 2022\n",
      "\n",
      "\n",
      "end: Mon Jan  3 14:15:56 2022\n",
      "\n",
      "--- processing complete\n"
     ]
    }
   ],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "crs = \"EPSG:4326\"\n",
    "geometry = [Point(xy) for xy in zip(hamlet_all.X, hamlet_all.Y)]\n",
    "hamlet_all = GeoDataFrame(hamlet_all, crs=crs, geometry=geometry) \n",
    "hamlet_all.to_file(driver='ESRI Shapefile', filename='C:/Users/wb527163/GEO-Cdrive-Grace/SEN-Cdrive/hamlet_to_HDurban_base-F10-20-50.shp') \n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Agriculture"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "base_CD = os.path.join(out_pth, \"fromag_drive-CD.csv\") # Contains base travel time, and the cost-distance travel from origin to road node.\n",
    "base_CD = pd.read_csv(base_CD)\n",
    "ag_snap = os.path.join(out_pth, \"ag_snap.csv\") \n",
    "ag_snap = pd.read_csv(ag_snap)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 5 columns):\n",
      " #   Column    Dtype  \n",
      "---  ------    -----  \n",
      " 0   ID_ag     int64  \n",
      " 1   val       float64\n",
      " 2   NSnomax   float64\n",
      " 3   NN        int64  \n",
      " 4   ag_HD1mm  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 531.8 MB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 12 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID_ag       int64  \n",
      " 2   ID_spam     float64\n",
      " 3   grid_val    float64\n",
      " 4   ID_LC       float64\n",
      " 5   val         float64\n",
      " 6   x           float64\n",
      " 7   y           float64\n",
      " 8   NSnomax     float64\n",
      " 9   geometry    object \n",
      " 10  NN          int64  \n",
      " 11  NN_dist     float64\n",
      "dtypes: float64(8), int64(3), object(1)\n",
      "memory usage: 1.2+ GB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(base_CD.info())\n",
    "print(ag_snap.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 625851 entries, 0 to 625850\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   Unnamed: 0  625851 non-null  int64  \n",
      " 1   NN          625851 non-null  int64  \n",
      " 2   ag_HD1F10   625841 non-null  float64\n",
      " 3   ag_HD2F10   625851 non-null  float64\n",
      " 4   ag_HD3F10   625851 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 23.9 MB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 16 columns):\n",
      " #   Column        Dtype  \n",
      "---  ------        -----  \n",
      " 0   Unnamed: 0    int64  \n",
      " 1   Unnamed: 0.1  int64  \n",
      " 2   ID_ag         int64  \n",
      " 3   ID_spam       float64\n",
      " 4   grid_val      float64\n",
      " 5   ID_LC         float64\n",
      " 6   val           float64\n",
      " 7   x             float64\n",
      " 8   y             float64\n",
      " 9   NSnomax       float64\n",
      " 10  geometry      object \n",
      " 11  NN            int64  \n",
      " 12  NN_dist       float64\n",
      " 13  ag_HD1F20     float64\n",
      " 14  ag_HD2F20     float64\n",
      " 15  ag_HD3F20     float64\n",
      "dtypes: float64(11), int64(4), object(1)\n",
      "memory usage: 1.7+ GB\n",
      "None\n",
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 625851 entries, 0 to 625850\n",
      "Data columns (total 5 columns):\n",
      " #   Column      Non-Null Count   Dtype  \n",
      "---  ------      --------------   -----  \n",
      " 0   Unnamed: 0  625851 non-null  int64  \n",
      " 1   NN          625851 non-null  int64  \n",
      " 2   ag_HD1F50   625841 non-null  float64\n",
      " 3   ag_HD2F50   625851 non-null  float64\n",
      " 4   ag_HD3F50   625851 non-null  float64\n",
      "dtypes: float64(3), int64(2)\n",
      "memory usage: 23.9 MB\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "# If reloading:\n",
    "agF10 = os.path.join(out_pth, \"ag_to_HDurban_flood10.csv\")\n",
    "agF10 = pd.read_csv(agF10)\n",
    "agF20 = os.path.join(out_pth, \"ag_to_HDurban_flood20.csv\")\n",
    "agF20 = pd.read_csv(agF20)\n",
    "agF50 = os.path.join(out_pth, \"ag_to_HDurban_flood50.csv\")\n",
    "agF50 = pd.read_csv(agF50)\n",
    "\n",
    "print(agF10.info())\n",
    "print(agF20.info())\n",
    "print(agF50.info())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "625851\n",
      "625851\n",
      "625851\n",
      "625851\n",
      "13941124\n",
      "13941124\n"
     ]
    }
   ],
   "source": [
    "# Make sure the ID is unique and fully matched\n",
    "list1 = list(agF50['NN'].unique())\n",
    "list2 = list(agF20['NN'].unique())\n",
    "list3 = list(agF10['NN'].unique())\n",
    "list4 = list(ag_snap['NN'].unique())\n",
    "print(len(list1))\n",
    "print(len(list2))\n",
    "print(len(list3))\n",
    "print(len(list4))\n",
    "\n",
    "list5 = list(ag_snap['ID_ag'].unique())\n",
    "list6 = list(base_CD['ID_ag'].unique())\n",
    "print(len(list5))\n",
    "print(len(list6))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           NN  ag_HD1F10\n",
      "0  3507831609  82.054823\n",
      "1  3507831510  88.413319\n",
      "2  6188134127  23.536822\n",
      "3  8631201421  94.721696\n",
      "4  8598305977  51.737958\n",
      "\n",
      "           NN  ag_HD1F20\n",
      "0  3507831609  82.054823\n",
      "1  3507831510  88.413319\n",
      "2  6188134127  23.536822\n",
      "3  3507831510  88.413319\n",
      "4  3507831510  88.413319\n",
      "\n",
      "           NN   ag_HD1F50\n",
      "0  3507831609   82.731533\n",
      "1  3507831510   89.090029\n",
      "2  6188134127   23.536822\n",
      "3  8631201421  503.912228\n",
      "4  8598305977  518.180056\n",
      "\n",
      "      ID_ag                                        geometry          NN\n",
      "0  13941103  POINT (-16.458987999999977 12.208646000000044)  3507831609\n",
      "1  13941122  POINT (-16.375654999999938 12.208646000000044)  3507831510\n",
      "2  13941017  POINT (-16.208988999999974 12.208646000000044)  6188134127\n",
      "3  13941036   POINT (-16.20849199999998 12.152815000000032)  3507831510\n",
      "4  13941044  POINT (-16.294212999999957 12.149487000000022)  3507831510\n",
      "\n",
      "      ID_ag        val  NSnomax  ag_HD1mm\n",
      "0  13941103  2008851.0      NaN       NaN\n",
      "1  13941122    94605.0      NaN       NaN\n",
      "2  13941017   708413.0      NaN       NaN\n",
      "3  13941036   315719.0      NaN       NaN\n",
      "4  13941044   375288.0      NaN       NaN\n",
      "\n"
     ]
    }
   ],
   "source": [
    "agF10 = agF10[['NN', 'ag_HD1F10']]\n",
    "agF20 = agF20[['NN', 'ag_HD1F20']]\n",
    "agF50 = agF50[['NN', 'ag_HD1F50']]\n",
    "ag_snap = ag_snap[['ID_ag', 'geometry', 'NN']]\n",
    "base_CD = base_CD[['ID_ag', 'val', 'NSnomax', 'ag_HD1mm']]\n",
    "\n",
    "print(agF10.head(), end='\\n\\n')\n",
    "print(agF20.head(), end='\\n\\n')\n",
    "print(agF50.head(), end='\\n\\n')\n",
    "print(ag_snap.head(), end='\\n\\n')\n",
    "print(base_CD.head(), end='\\n\\n')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 6 columns):\n",
      " #   Column    Dtype  \n",
      "---  ------    -----  \n",
      " 0   ID_ag     int64  \n",
      " 1   geometry  object \n",
      " 2   NN        int64  \n",
      " 3   val       float64\n",
      " 4   NSnomax   float64\n",
      " 5   ag_HD1mm  float64\n",
      "dtypes: float64(3), int64(2), object(1)\n",
      "memory usage: 744.5+ MB\n",
      "None\n",
      "\n",
      "      ID_ag                                        geometry          NN  \\\n",
      "0  13941103  POINT (-16.458987999999977 12.208646000000044)  3507831609   \n",
      "1  13941122  POINT (-16.375654999999938 12.208646000000044)  3507831510   \n",
      "2  13941017  POINT (-16.208988999999974 12.208646000000044)  6188134127   \n",
      "3  13941036   POINT (-16.20849199999998 12.152815000000032)  3507831510   \n",
      "4  13941044  POINT (-16.294212999999957 12.149487000000022)  3507831510   \n",
      "\n",
      "         val  NSnomax  ag_HD1mm  \n",
      "0  2008851.0      NaN       NaN  \n",
      "1    94605.0      NaN       NaN  \n",
      "2   708413.0      NaN       NaN  \n",
      "3   315719.0      NaN       NaN  \n",
      "4   375288.0      NaN       NaN  \n"
     ]
    }
   ],
   "source": [
    "ag_all = pd.merge(ag_snap, base_CD, on='ID_ag',how='left')\n",
    "print(ag_all.info(), end='\\n\\n')\n",
    "print(ag_all.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "Int64Index: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 7 columns):\n",
      " #   Column     Dtype  \n",
      "---  ------     -----  \n",
      " 0   ID_ag      int64  \n",
      " 1   geometry   object \n",
      " 2   NN         int64  \n",
      " 3   val        float64\n",
      " 4   NSnomax    float64\n",
      " 5   ag_HD1mm   float64\n",
      " 6   ag_HD1F10  float64\n",
      "dtypes: float64(4), int64(2), object(1)\n",
      "memory usage: 850.9+ MB\n",
      "None\n",
      "\n",
      "      ID_ag                                        geometry          NN  \\\n",
      "0  13941103  POINT (-16.458987999999977 12.208646000000044)  3507831609   \n",
      "1  13941122  POINT (-16.375654999999938 12.208646000000044)  3507831510   \n",
      "2  13941017  POINT (-16.208988999999974 12.208646000000044)  6188134127   \n",
      "3  13941036   POINT (-16.20849199999998 12.152815000000032)  3507831510   \n",
      "4  13941044  POINT (-16.294212999999957 12.149487000000022)  3507831510   \n",
      "\n",
      "         val  NSnomax  ag_HD1mm  ag_HD1F10  \n",
      "0  2008851.0      NaN       NaN  82.054823  \n",
      "1    94605.0      NaN       NaN  88.413319  \n",
      "2   708413.0      NaN       NaN  23.536822  \n",
      "3   315719.0      NaN       NaN  88.413319  \n",
      "4   375288.0      NaN       NaN  88.413319  \n"
     ]
    }
   ],
   "source": [
    "agF10 = pd.merge(ag_all, agF10, on='NN',how='left')\n",
    "print(ag_all.info(), end='\\n\\n')\n",
    "print(ag_all.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save intermediate file, because it's so huge.\n",
    "agF10.to_csv(os.path.join(out_pth, 'ag_to_HDurban_base-F10.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Restart kernel from here if having memory errors.\n",
    "agF10 = os.path.join(out_pth, \"ag_to_HDurban_base-F10.csv\")\n",
    "agF10 = pd.read_csv(agF10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 9 columns):\n",
      " #   Column      Dtype  \n",
      "---  ------      -----  \n",
      " 0   Unnamed: 0  int64  \n",
      " 1   ID_ag       int64  \n",
      " 2   geometry    object \n",
      " 3   NN          int64  \n",
      " 4   val         float64\n",
      " 5   NSnomax     float64\n",
      " 6   base        float64\n",
      " 7   ag_HD1F10   float64\n",
      " 8   F10         float64\n",
      "dtypes: float64(5), int64(3), object(1)\n",
      "memory usage: 957.3+ MB\n"
     ]
    }
   ],
   "source": [
    "# Due to memory errors, need to create different data files for each flood.\n",
    "agF10.rename(columns={'ag_HD1mm':'base'}, inplace=True)\n",
    "agF10['F10'] = 0\n",
    "agF10['F10'] = agF10['ag_HD1F10'] + agF10['NSnomax']\n",
    "agF10.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>ID_ag</th>\n",
       "      <th>NN</th>\n",
       "      <th>val</th>\n",
       "      <th>NSnomax</th>\n",
       "      <th>base</th>\n",
       "      <th>ag_HD1F10</th>\n",
       "      <th>F10</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.318626e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "      <td>1.394082e+07</td>\n",
       "      <td>1.318596e+07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>5.121305e+09</td>\n",
       "      <td>1.219702e+02</td>\n",
       "      <td>2.209557e+01</td>\n",
       "      <td>7.234398e+01</td>\n",
       "      <td>8.373060e+02</td>\n",
       "      <td>2.889649e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>4.024456e+06</td>\n",
       "      <td>4.024456e+06</td>\n",
       "      <td>2.262019e+09</td>\n",
       "      <td>3.760062e+03</td>\n",
       "      <td>2.796064e+01</td>\n",
       "      <td>4.196269e+01</td>\n",
       "      <td>7.494902e+03</td>\n",
       "      <td>2.940325e+03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>2.214129e+07</td>\n",
       "      <td>1.563126e-01</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.485281e+06</td>\n",
       "      <td>3.485281e+06</td>\n",
       "      <td>3.656862e+09</td>\n",
       "      <td>4.293236e+01</td>\n",
       "      <td>5.456861e+00</td>\n",
       "      <td>4.336144e+01</td>\n",
       "      <td>2.983436e+01</td>\n",
       "      <td>4.542230e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>6.970562e+06</td>\n",
       "      <td>4.519739e+09</td>\n",
       "      <td>7.008440e+01</td>\n",
       "      <td>1.162371e+01</td>\n",
       "      <td>6.605466e+01</td>\n",
       "      <td>4.840991e+01</td>\n",
       "      <td>7.124638e+01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>1.045584e+07</td>\n",
       "      <td>1.045584e+07</td>\n",
       "      <td>6.968809e+09</td>\n",
       "      <td>1.013998e+02</td>\n",
       "      <td>2.869398e+01</td>\n",
       "      <td>9.401759e+01</td>\n",
       "      <td>9.097461e+01</td>\n",
       "      <td>1.084964e+02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>1.394112e+07</td>\n",
       "      <td>9.285641e+09</td>\n",
       "      <td>6.939670e+06</td>\n",
       "      <td>1.713278e+03</td>\n",
       "      <td>1.801212e+03</td>\n",
       "      <td>1.527663e+05</td>\n",
       "      <td>1.527882e+05</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         Unnamed: 0         ID_ag            NN           val       NSnomax  \\\n",
       "count  1.394112e+07  1.394112e+07  1.394112e+07  1.394112e+07  1.318626e+07   \n",
       "mean   6.970562e+06  6.970562e+06  5.121305e+09  1.219702e+02  2.209557e+01   \n",
       "std    4.024456e+06  4.024456e+06  2.262019e+09  3.760062e+03  2.796064e+01   \n",
       "min    0.000000e+00  0.000000e+00  2.214129e+07  1.563126e-01  0.000000e+00   \n",
       "25%    3.485281e+06  3.485281e+06  3.656862e+09  4.293236e+01  5.456861e+00   \n",
       "50%    6.970562e+06  6.970562e+06  4.519739e+09  7.008440e+01  1.162371e+01   \n",
       "75%    1.045584e+07  1.045584e+07  6.968809e+09  1.013998e+02  2.869398e+01   \n",
       "max    1.394112e+07  1.394112e+07  9.285641e+09  6.939670e+06  1.713278e+03   \n",
       "\n",
       "               base     ag_HD1F10           F10  \n",
       "count  1.318596e+07  1.394082e+07  1.318596e+07  \n",
       "mean   7.234398e+01  8.373060e+02  2.889649e+02  \n",
       "std    4.196269e+01  7.494902e+03  2.940325e+03  \n",
       "min    0.000000e+00  0.000000e+00  0.000000e+00  \n",
       "25%    4.336144e+01  2.983436e+01  4.542230e+01  \n",
       "50%    6.605466e+01  4.840991e+01  7.124638e+01  \n",
       "75%    9.401759e+01  9.097461e+01  1.084964e+02  \n",
       "max    1.801212e+03  1.527663e+05  1.527882e+05  "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(agF10.describe())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 7 columns):\n",
      " #   Column    Dtype  \n",
      "---  ------    -----  \n",
      " 0   ID_ag     int64  \n",
      " 1   geometry  object \n",
      " 2   NN        int64  \n",
      " 3   val       float64\n",
      " 4   NSnomax   float64\n",
      " 5   base      float64\n",
      " 6   F10       float64\n",
      "dtypes: float64(4), int64(2), object(1)\n",
      "memory usage: 744.5+ MB\n"
     ]
    }
   ],
   "source": [
    "# Remove legacy variables.\n",
    "agF10 = agF10.drop(columns=['Unnamed: 0','ag_HD1F10'])\n",
    "agF10.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Save to file\n",
    "agF10.to_csv(os.path.join(out_pth, 'ag_to_HDurban_base-F10.csv'))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1 in 20 year flood."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Restart kernel from here if having memory errors.\n",
    "agF10 = os.path.join(out_pth, \"ag_to_HDurban_base-F10.csv\")\n",
    "agF10 = pd.read_csv(agF10)\n",
    "agF10 = agF10.drop(columns=['Unnamed: 0', 'geometry', 'val', 'F10'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 4 columns):\n",
      " #   Column   Dtype  \n",
      "---  ------   -----  \n",
      " 0   ID_ag    int64  \n",
      " 1   NN       int64  \n",
      " 2   NSnomax  float64\n",
      " 3   base     float64\n",
      "dtypes: float64(2), int64(2)\n",
      "memory usage: 425.4 MB\n"
     ]
    }
   ],
   "source": [
    "agF10.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "ename": "MemoryError",
     "evalue": "Unable to allocate 107. GiB for an array with shape (14399955670,) and data type int64",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mMemoryError\u001b[0m                               Traceback (most recent call last)",
      "\u001b[1;32m~\\AppData\\Local\\Temp/ipykernel_1068/2823581609.py\u001b[0m in \u001b[0;36m<module>\u001b[1;34m\u001b[0m\n\u001b[0;32m      3\u001b[0m \u001b[0magF20\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0magF20\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;34m'NN'\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;34m'ag_HD1F20'\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      4\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m----> 5\u001b[1;33m \u001b[0magF20\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mpd\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mmerge\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0magF10\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0magF20\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mon\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'NN'\u001b[0m\u001b[1;33m,\u001b[0m\u001b[0mhow\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'left'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      6\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0magF20\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0minfo\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mend\u001b[0m\u001b[1;33m=\u001b[0m\u001b[1;34m'\\n\\n'\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m      7\u001b[0m \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0magF20\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhead\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\.conda\\envs\\geo\\lib\\site-packages\\pandas\\core\\reshape\\merge.py\u001b[0m in \u001b[0;36mmerge\u001b[1;34m(left, right, how, on, left_on, right_on, left_index, right_index, sort, suffixes, copy, indicator, validate)\u001b[0m\n\u001b[0;32m    119\u001b[0m         \u001b[0mvalidate\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mvalidate\u001b[0m\u001b[1;33m,\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    120\u001b[0m     )\n\u001b[1;32m--> 121\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0mop\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mget_result\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    122\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    123\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\.conda\\envs\\geo\\lib\\site-packages\\pandas\\core\\reshape\\merge.py\u001b[0m in \u001b[0;36mget_result\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    713\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mleft\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mright\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_indicator_pre_merge\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mleft\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mright\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    714\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 715\u001b[1;33m         \u001b[0mjoin_index\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mleft_indexer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mright_indexer\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_get_join_info\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    716\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    717\u001b[0m         llabels, rlabels = _items_overlap_with_suffix(\n",
      "\u001b[1;32m~\\.conda\\envs\\geo\\lib\\site-packages\\pandas\\core\\reshape\\merge.py\u001b[0m in \u001b[0;36m_get_join_info\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    964\u001b[0m             )\n\u001b[0;32m    965\u001b[0m         \u001b[1;32melse\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 966\u001b[1;33m             \u001b[1;33m(\u001b[0m\u001b[0mleft_indexer\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mright_indexer\u001b[0m\u001b[1;33m)\u001b[0m \u001b[1;33m=\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0m_get_join_indexers\u001b[0m\u001b[1;33m(\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m    967\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    968\u001b[0m             \u001b[1;32mif\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mright_index\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\.conda\\envs\\geo\\lib\\site-packages\\pandas\\core\\reshape\\merge.py\u001b[0m in \u001b[0;36m_get_join_indexers\u001b[1;34m(self)\u001b[0m\n\u001b[0;32m    938\u001b[0m         \u001b[1;34m\"\"\"return the join indexers\"\"\"\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    939\u001b[0m         \u001b[1;31m# Both returned ndarrays are np.intp\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m--> 940\u001b[1;33m         return get_join_indexers(\n\u001b[0m\u001b[0;32m    941\u001b[0m             \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mleft_join_keys\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mright_join_keys\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0msort\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0msort\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mhow\u001b[0m\u001b[1;33m=\u001b[0m\u001b[0mself\u001b[0m\u001b[1;33m.\u001b[0m\u001b[0mhow\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m    942\u001b[0m         )\n",
      "\u001b[1;32m~\\.conda\\envs\\geo\\lib\\site-packages\\pandas\\core\\reshape\\merge.py\u001b[0m in \u001b[0;36mget_join_indexers\u001b[1;34m(left_keys, right_keys, sort, how, **kwargs)\u001b[0m\n\u001b[0;32m   1504\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1505\u001b[0m     \u001b[1;31m# error: Cannot call function of unknown type\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[1;32m-> 1506\u001b[1;33m     \u001b[1;32mreturn\u001b[0m \u001b[0mjoin_func\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mlkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mrkey\u001b[0m\u001b[1;33m,\u001b[0m \u001b[0mcount\u001b[0m\u001b[1;33m,\u001b[0m \u001b[1;33m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[1;33m)\u001b[0m  \u001b[1;31m# type: ignore[operator]\u001b[0m\u001b[1;33m\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m   1507\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0;32m   1508\u001b[0m \u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;32m~\\.conda\\envs\\geo\\lib\\site-packages\\pandas\\_libs\\join.pyx\u001b[0m in \u001b[0;36mpandas._libs.join.left_outer_join\u001b[1;34m()\u001b[0m\n",
      "\u001b[1;31mMemoryError\u001b[0m: Unable to allocate 107. GiB for an array with shape (14399955670,) and data type int64"
     ]
    }
   ],
   "source": [
    "agF20 = os.path.join(out_pth, \"ag_to_HDurban_flood20.csv\")\n",
    "agF20 = pd.read_csv(agF20)\n",
    "agF20 = agF20[['NN', 'ag_HD1F20']]\n",
    "\n",
    "agF20 = pd.merge(agF10, agF20, on='NN',how='left')\n",
    "print(agF20.info(), end='\\n\\n')\n",
    "print(agF20.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 13941124 entries, 0 to 13941123\n",
      "Data columns (total 2 columns):\n",
      " #   Column     Dtype  \n",
      "---  ------     -----  \n",
      " 0   NN         int64  \n",
      " 1   ag_HD1F20  float64\n",
      "dtypes: float64(1), int64(1)\n",
      "memory usage: 212.7 MB\n"
     ]
    }
   ],
   "source": [
    "agF20.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "agF20.to_csv(os.path.join(out_pth, 'ag_to_HDurban_base-F20.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Due to memory errors, need to create different data files for each flood.\n",
    "agF20['F20'] = 0\n",
    "agF20['F20'] = agF20['ag_HD1F20'] + agF20['NSnomax']\n",
    "agF20.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Restart kernel from here if having memory errors.\n",
    "ag_all = os.path.join(out_pth, \"ag_to_HDurban_base-F10-20-50.csv\")\n",
    "ag_all = pd.read_csv(ag_all)\n",
    "agF50 = os.path.join(out_pth, \"ag_to_HDurban_flood50.csv\")\n",
    "agF50 = pd.read_csv(agF50)\n",
    "agF50 = agF20[['NN', 'ag_HD1F50']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ag_all = pd.merge(ag_all, agF50, on='NN',how='left')\n",
    "print(ag_all.info(), end='\\n\\n')\n",
    "print(ag_all.head())\n",
    "ag_all.to_csv(os.path.join(out_pth, 'ag_to_HDurban_base-F10-20-50.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ag_all['F20'] = 0\n",
    "ag_all['F20'] = ag_all['ha_HD1F20'] + ag_all['NSnomax']\n",
    "ag_all['F50'] = 0\n",
    "ag_all['F50'] = ag_all['ha_HD1F50'] + ag_all['NSnomax']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Remove legacy variables.\n",
    "ag_all = ag_all.drop(columns=['ha_HD1F10', 'ha_HD1F20', 'ha_HD1F50'])\n",
    "ag_all.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ag_all[\"geometry\"] = ag_all[\"geometry\"].astype('str')\n",
    "ag_all[\"geometry\"]  = ag_all[\"geometry\"] .str.strip('POINT ')\n",
    "ag_all[\"geometry\"]  = ag_all[\"geometry\"] .str.strip('()')\n",
    "XY = ag_all[\"geometry\"] .str.split(\" \", expand=True)\n",
    "ag_all[\"X\"] = XY[0]\n",
    "ag_all[\"Y\"] = XY[1]\n",
    "ag_all[\"X\"] = ag_all[\"X\"].astype('float')\n",
    "ag_all[\"Y\"] = ag_all[\"Y\"].astype('float')\n",
    "ag_all = ag_all.drop(columns=['geometry'])\n",
    "ag_all.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ag_all.to_csv(os.path.join(out_pth, 'ag_to_HDurban_base-F10-20-50.csv'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('start: %s\\n' % time.ctime())\n",
    "crs = \"EPSG:4326\"\n",
    "geometry = [Point(xy) for xy in zip(ag_all.X, ag_all.Y)]\n",
    "ag_all = GeoDataFrame(ag_all, crs=crs, geometry=geometry) \n",
    "ag_all.to_file(driver='ESRI Shapefile', filename='C:/Users/wb527163/GEO-Cdrive-Grace/SEN-Cdrive/ag_to_HDurban_base-F10-20-50.shp') \n",
    "print('\\nend: %s' % time.ctime())\n",
    "print('\\n--- processing complete')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}