NLP_C2_W4_lecture_nb_5_word_embeddings_step_by_step.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Word Embeddings: Ungraded Practice Notebook\n",
    "\n",
    "In this ungraded notebook, you'll try out all the individual techniques that you learned about in the lecture. Practicing on small examples will prepare you for the graded assignment, where you will combine the techniques in more advanced ways to create word embeddings from a real-life corpus.\n",
    "\n",
    "This notebook is made of two main parts: data preparation, and the continuous bag-of-words (CBOW) model.\n",
    "\n",
    "To get started, import and initialize all the libraries you will need."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "!{sys.executable} -m pip install emoji"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import re\n",
    "import nltk\n",
    "from nltk.tokenize import word_tokenize\n",
    "import emoji\n",
    "import numpy as np\n",
    "\n",
    "from utils2 import get_dict\n",
    "\n",
    "nltk.download('punkt')  # download pre-trained Punkt tokenizer for English"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the data preparation phase, starting with a corpus of text, you will:\n",
    "\n",
    "- Clean and tokenize the corpus.\n",
    "\n",
    "- Extract the pairs of context words and center word that will make up the training data set for the CBOW model. The context words are the features that will be fed into the model, and the center words are the target values that the model will learn to predict.\n",
    "\n",
    "- Create simple vector representations of the context words (features) and center words (targets) that can be used by the neural network of the CBOW model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleaning and tokenization\n",
    "\n",
    "To demonstrate the cleaning and tokenization process, consider a corpus that contains emojis and various punctuation signs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "corpus = 'Who ❤️ \"word embeddings\" in 2020? I do!!!'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, replace all interrupting punctuation signs — such as commas and exclamation marks — with periods."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'Corpus:  {corpus}')\n",
    "data = re.sub(r'[,!?;-]+', '.', corpus)\n",
    "print(f'After cleaning punctuation:  {data}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, use NLTK's tokenization engine to split the corpus into individual tokens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'Initial string:  {data}')\n",
    "data = nltk.word_tokenize(data)\n",
    "print(f'After tokenization:  {data}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, as you saw in the lecture, get rid of numbers and punctuation other than periods, and convert all the remaining tokens to lowercase."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'Initial list of tokens:  {data}')\n",
    "data = [ ch.lower() for ch in data\n",
    "         if ch.isalpha()\n",
    "         or ch == '.'\n",
    "         or emoji.get_emoji_regexp().search(ch)\n",
    "       ]\n",
    "print(f'After cleaning:  {data}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the heart emoji is considered as a token just like any normal word.\n",
    "\n",
    "Now let's streamline the cleaning and tokenization process by wrapping the previous steps in a function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def tokenize(corpus):\n",
    "    data = re.sub(r'[,!?;-]+', '.', corpus)\n",
    "    data = nltk.word_tokenize(data)  # tokenize string to words\n",
    "    data = [ ch.lower() for ch in data\n",
    "             if ch.isalpha()\n",
    "             or ch == '.'\n",
    "             or emoji.get_emoji_regexp().search(ch)\n",
    "           ]\n",
    "    return data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Apply this function to the corpus that you'll be working on in the rest of this notebook: \"I am happy because I am learning\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "corpus = 'I am happy because I am learning'\n",
    "print(f'Corpus:  {corpus}')\n",
    "words = tokenize(corpus)\n",
    "print(f'Words (tokens):  {words}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now try it out yourself with your own sentence.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tokenize(\"Now it's your turn: try with your own sentence!\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Sliding window of words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that you have transformed the corpus into a list of clean tokens, you can slide a window of words across this list. For each window you can extract a center word and the context words.\n",
    "\n",
    "The `get_windows` function in the next cell was introduced in the lecture."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_windows(words, C):\n",
    "    i = C\n",
    "    while i < len(words) - C:\n",
    "        center_word = words[i]\n",
    "        context_words = words[(i - C):i] + words[(i+1):(i+C+1)]\n",
    "        yield context_words, center_word\n",
    "        i += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first argument of this function is a list of words (or tokens). The second argument, `C`, is the context half-size. Recall that for a given center word, the context words are made of `C` words to the left and `C` words to the right of the center word.\n",
    "\n",
    "Here is how you can use this function to extract context words and center words from a list of tokens. These context and center words will make up the training set that you will use to train the CBOW model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for x, y in get_windows(\n",
    "            ['i', 'am', 'happy', 'because', 'i', 'am', 'learning'],\n",
    "            2\n",
    "        ):\n",
    "    print(f'{x}\\t{y}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first example of the training set is made of:\n",
    "\n",
    "- the context words \"i\", \"am\", \"because\", \"i\",\n",
    "\n",
    "- and the center word to be predicted: \"happy\".\n",
    "\n",
    "**Now try it out yourself. In the next cell, you can change both the sentence and the context half-size.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for x, y in get_windows(tokenize(\"Now it's your turn: try with your own sentence!\"), 1):\n",
    "    print(f'{x}\\t{y}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Transforming words into vectors for the training set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To finish preparing the training set, you need to transform the context words and center words into vectors.\n",
    "\n",
    "### Mapping words to indices and indices to words\n",
    "\n",
    "The center words will be represented as one-hot vectors, and the vectors that represent context words are also based on one-hot vectors.\n",
    "\n",
    "To create one-hot word vectors, you can start by mapping each unique word to a unique integer (or index). We have provided a helper function, `get_dict`, that creates a Python dictionary that maps words to integers and back."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "word2Ind, Ind2word = get_dict(words)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here's the dictionary that maps words to numeric indices."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "word2Ind"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use this dictionary to get the index of a word."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Index of the word 'i':  \",word2Ind['i'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And conversely, here's the dictionary that maps indices to words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Ind2word"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "print(\"Word which has index 2:  \",Ind2word[2] )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, get the length of either of these dictionaries to get the size of the vocabulary of your corpus, in other words the number of different words making up the corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "V = len(word2Ind)\n",
    "print(\"Size of vocabulary: \", V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting one-hot word vectors\n",
    "\n",
    "Recall from the lecture that you can easily convert an integer, $n$, into a one-hot vector.\n",
    "\n",
    "Consider the word \"happy\". First, retrieve its numeric index."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n = word2Ind['happy']\n",
    "n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now create a vector with the size of the vocabulary, and fill it with zeros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "center_word_vector = np.zeros(V)\n",
    "center_word_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can confirm that the vector has the right size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(center_word_vector) == V"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, replace the 0 of the $n$-th element with a 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "center_word_vector[n] = 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And you have your one-hot word vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "center_word_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**You can now group all of these steps in a convenient function, which takes as parameters: a word to be encoded, a dictionary that maps words to indices, and the size of the vocabulary.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def word_to_one_hot_vector(word, word2Ind, V):\n",
    "    one_hot_vector = np.zeros(V)\n",
    "    one_hot_vector[word2Ind[word]] = 1\n",
    "    \n",
    "    return one_hot_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check that it works as intended."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "word_to_one_hot_vector('happy', word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What is the word vector for \"learning\"?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "word_to_one_hot_vector('learning', word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([0., 0., 0., 0., 1.])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Getting context word vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To create the vectors that represent context words, you will calculate the average of the one-hot vectors representing the individual words.\n",
    "\n",
    "Let's start with a list of context words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "context_words = ['i', 'am', 'because', 'i']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using Python's list comprehension construct and the `word_to_one_hot_vector` function that you created in the previous section, you can create a list of one-hot vectors representing each of the context words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "context_words_vectors = [word_to_one_hot_vector(w, word2Ind, V) for w in context_words]\n",
    "context_words_vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And you can now simply get the average of these vectors using numpy's `mean` function, to get the vector representation of the context words."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.mean(context_words_vectors, axis=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note the `axis=0` parameter that tells `mean` to calculate the average of the rows (if you had wanted the average of the columns, you would have used `axis=1`).\n",
    "\n",
    "**Now create the `context_words_to_vector` function that takes in a list of context words, a word-to-index dictionary, and a vocabulary size, and outputs the vector representation of the context words.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def context_words_to_vector(context_words, word2Ind, V):\n",
    "    context_words_vectors = [word_to_one_hot_vector(w, word2Ind, V) for w in context_words]\n",
    "    context_words_vectors = np.mean(context_words_vectors, axis=0)\n",
    "    \n",
    "    return context_words_vectors"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And check that you obtain the same output as the manual approach above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "context_words_to_vector(['i', 'am', 'because', 'i'], word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**What is the vector representation of the context words \"am happy i am\"?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "context_words_to_vector(['am', 'happy', 'i', 'am'], word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([0.5 , 0.  , 0.25, 0.25, 0.  ])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Building the training set"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now combine the functions that you created in the previous sections, to build a training set for the CBOW model, starting from the following tokenized corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "words"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To do this you need to use the sliding window function (`get_windows`) to extract the context words and center words, and you then convert these sets of words into a basic vector representation using `word_to_one_hot_vector` and `context_words_to_vector`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for context_words, center_word in get_windows(words, 2):  # reminder: 2 is the context half-size\n",
    "    print(f'Context words:  {context_words} -> {context_words_to_vector(context_words, word2Ind, V)}')\n",
    "    print(f'Center word:  {center_word} -> {word_to_one_hot_vector(center_word, word2Ind, V)}')\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this practice notebook you'll be performing a single iteration of training using a single example, but in this week's assignment you'll train the CBOW model using several iterations and batches of example.\n",
    "Here is how you would use a Python generator function (remember the `yield` keyword from the lecture?) to make it easier to iterate over a set of examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_training_example(words, C, word2Ind, V):\n",
    "    for context_words, center_word in get_windows(words, C):\n",
    "        yield context_words_to_vector(context_words, word2Ind, V), word_to_one_hot_vector(center_word, word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The output of this function can be iterated on to get successive context word vectors and center word vectors, as demonstrated in the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for context_words_vector, center_word_vector in get_training_example(words, 2, word2Ind, V):\n",
    "    print(f'Context words vector:  {context_words_vector}')\n",
    "    print(f'Center word vector:  {center_word_vector}')\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Your training set is ready, you can now move on to the CBOW model itself."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# The continuous bag-of-words model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The CBOW model is based on a neural network, the architecture of which looks like the figure below, as you'll recall from the lecture.\n",
    "\n",
    "<div style=\"width:image width px; font-size:100%; text-align:center;\"><img src='./images/cbow_model_architecture.png?1' alt=\"alternate text\" width=\"width\" height=\"height\" style=\"width:917;height:337;\" /> Figure 1 </div>\n",
    "\n",
    "This part of the notebook will walk you through:\n",
    "\n",
    "- The two activation functions used in the neural network.\n",
    "\n",
    "- Forward propagation.\n",
    "\n",
    "- Cross-entropy loss.\n",
    "\n",
    "- Backpropagation.\n",
    "\n",
    "- Gradient descent.\n",
    "\n",
    "- Extracting the word embedding vectors from the weight matrices once the neural network has been trained."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Activation functions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by implementing the activation functions, ReLU and softmax."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ReLU"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "ReLU is used to calculate the values of the hidden layer, in the following formulas:\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{z_1} &= \\mathbf{W_1}\\mathbf{x} + \\mathbf{b_1}  \\tag{1} \\\\\n",
    " \\mathbf{h} &= \\mathrm{ReLU}(\\mathbf{z_1})  \\tag{2} \\\\\n",
    "\\end{align}\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's fix a value for $\\mathbf{z_1}$ as a working example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(10)\n",
    "z_1 = 10*np.random.rand(5, 1)-5\n",
    "z_1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To get the ReLU of this vector, you want all the negative values to become zeros.\n",
    "\n",
    "First create a copy of this vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = z_1.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now determine which of its values are negative."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "h < 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now simply set all of the values which are negative to 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "h[h < 0] = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And that's it: you have the ReLU of $\\mathbf{z_1}$!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now implement ReLU as a function.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def relu(z):\n",
    "    result = z.copy()\n",
    "    result[result < 0] = 0\n",
    "    \n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**And check that it's working.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = np.array([[-1.25459881], [ 4.50714306], [ 2.31993942], [ 0.98658484], [-3.4398136 ]])\n",
    "relu(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.        ],\n",
    "           [4.50714306],\n",
    "           [2.31993942],\n",
    "           [0.98658484],\n",
    "           [0.        ]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Softmax"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second activation function that you need is softmax. This function is used to calculate the values of the output layer of the neural network, using the following formulas:\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{z_2} &= \\mathbf{W_2}\\mathbf{h} + \\mathbf{b_2}   \\tag{3} \\\\\n",
    " \\mathbf{\\hat y} &= \\mathrm{softmax}(\\mathbf{z_2})   \\tag{4} \\\\\n",
    "\\end{align}\n",
    "\n",
    "To calculate softmax of a vector $\\mathbf{z}$, the $i$-th component of the resulting vector is given by:\n",
    "\n",
    "$$ \\textrm{softmax}(\\textbf{z})_i = \\frac{e^{z_i} }{\\sum\\limits_{j=1}^{V} e^{z_j} }  \\tag{5} $$\n",
    "\n",
    "Let's work through an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z = np.array([9, 8, 11, 10, 8.5])\n",
    "z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll need to calculate the exponentials of each element, both for the numerator and for the denominator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "e_z = np.exp(z)\n",
    "e_z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The denominator is equal to the sum of these exponentials."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sum_e_z = np.sum(e_z)\n",
    "sum_e_z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the value of the first element of $\\textrm{softmax}(\\textbf{z})$ is given by:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "e_z[0]/sum_e_z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is for one element. You can use numpy's vectorized operations to calculate the values of all the elements of the $\\textrm{softmax}(\\textbf{z})$ vector in one go.\n",
    "\n",
    "**Implement the softmax function.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def softmax(z):\n",
    "    e_z = np.exp(z)\n",
    "    sum_e_z = np.sum(e_z)\n",
    "    \n",
    "    return e_z / sum_e_z"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now check that it works.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "softmax([9, 8, 11, 10, 8.5])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([0.08276948, 0.03044919, 0.61158833, 0.22499077, 0.05020223])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dimensions: 1-D arrays vs 2-D column vectors\n",
    "\n",
    "Before moving on to implement forward propagation, backpropagation, and gradient descent, let's have a look at the dimensions of the vectors you've been handling until now.\n",
    "\n",
    "Create a vector of length $V$ filled with zeros."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "x_array = np.zeros(V)\n",
    "x_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a 1-dimensional array, as revealed by the `.shape` property of the array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_array.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To perform matrix multiplication in the next steps, you actually need your column vectors to be represented as a matrix with one column. In numpy, this matrix is represented as a 2-dimensional array.\n",
    "\n",
    "The easiest way to convert a 1D vector to a 2D column matrix is to set its `.shape` property to the number of rows and one column, as shown in the next cell."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_column_vector = x_array.copy()\n",
    "x_column_vector.shape = (V, 1)  # alternatively ... = (x_array.shape[0], 1)\n",
    "x_column_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The shape of the resulting \"vector\" is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_column_vector.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So you now have a 5x1 matrix that you can use to perform standard matrix multiplication."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Forward propagation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's dive into the neural network itself, which is shown below with all the dimensions and formulas you'll need.\n",
    "\n",
    "<div style=\"width:image width px; font-size:100%; text-align:center;\"><img src='./images/cbow_model_dimensions_single_input.png?2' alt=\"alternate text\" width=\"width\" height=\"height\" style=\"width:839;height:349;\" /> Figure 2 </div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Set $N$ equal to 3. Remember that $N$ is a hyperparameter of the CBOW model that represents the size of the word embedding vectors, as well as the size of the hidden layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "N = 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Initialization of the weights and biases"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before you start training the neural network, you need to initialize the weight matrices and bias vectors with random values.\n",
    "\n",
    "In the assignment you will implement a function to do this yourself using `numpy.random.rand`. In this notebook, we've pre-populated these matrices and vectors for you."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W1 = np.array([[ 0.41687358,  0.08854191, -0.23495225,  0.28320538,  0.41800106],\n",
    "               [ 0.32735501,  0.22795148, -0.23951958,  0.4117634 , -0.23924344],\n",
    "               [ 0.26637602, -0.23846886, -0.37770863, -0.11399446,  0.34008124]])\n",
    "\n",
    "W2 = np.array([[-0.22182064, -0.43008631,  0.13310965],\n",
    "               [ 0.08476603,  0.08123194,  0.1772054 ],\n",
    "               [ 0.1871551 , -0.06107263, -0.1790735 ],\n",
    "               [ 0.07055222, -0.02015138,  0.36107434],\n",
    "               [ 0.33480474, -0.39423389, -0.43959196]])\n",
    "\n",
    "b1 = np.array([[ 0.09688219],\n",
    "               [ 0.29239497],\n",
    "               [-0.27364426]])\n",
    "\n",
    "b2 = np.array([[ 0.0352008 ],\n",
    "               [-0.36393384],\n",
    "               [-0.12775555],\n",
    "               [-0.34802326],\n",
    "               [-0.07017815]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Check that the dimensions of these matrices match those shown in the figure above.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'V (vocabulary size): {V}')\n",
    "print(f'N (embedding size / size of the hidden layer): {N}')\n",
    "print(f'size of W1: {W1.shape} (NxV)')\n",
    "print(f'size of b1: {b1.shape} (Nx1)')\n",
    "print(f'size of W2: {W2.shape} (VxN)')\n",
    "print(f'size of b2: {b2.shape} (Vx1)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Training example"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Run the next cells to get the first training example, made of the vector representing the context words \"i am because i\", and the target which is the one-hot vector representing the center word \"happy\".\n",
    "\n",
    "> You don't need to worry about the Python syntax, but there are some explanations below if you want to know what's happening behind the scenes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "training_examples = get_training_example(words, 2, word2Ind, V)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `get_training_examples`, which uses the `yield` keyword, is known as a generator. When run, it builds an iterator, which is a special type of object that... you can iterate on (using a `for` loop for instance), to retrieve the successive values that the function generates.\n",
    ">\n",
    "> In this case `get_training_examples` `yield`s training examples, and iterating on `training_examples` will return the successive training examples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_array, y_array = next(training_examples)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `next` is another special keyword, which gets the next available value from an iterator. Here, you'll get the very first value, which is the first training example. If you run this cell again, you'll get the next value, and so on until the iterator runs out of values to return.\n",
    ">\n",
    "> In this notebook `next` is used because you will only be performing one iteration of training. In this week's assignment with the full training over several iterations you'll use regular `for` loops with the iterator that supplies the training examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The vector representing the context words, which will be fed into the neural network, is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The one-hot vector representing the center word to be predicted is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_array"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now convert these vectors into matrices (or 2D arrays) to be able to perform matrix multiplication on the right types of objects, as explained above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = x_array.copy()\n",
    "x.shape = (V, 1)\n",
    "print('x')\n",
    "print(x)\n",
    "print()\n",
    "\n",
    "y = y_array.copy()\n",
    "y.shape = (V, 1)\n",
    "print('y')\n",
    "print(y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Values of the hidden layer\n",
    "\n",
    "Now that you have initialized all the variables that you need for forward propagation, you can calculate the values of the hidden layer using the following formulas:\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{z_1} = \\mathbf{W_1}\\mathbf{x} + \\mathbf{b_1}  \\tag{1} \\\\\n",
    " \\mathbf{h} = \\mathrm{ReLU}(\\mathbf{z_1})  \\tag{2} \\\\\n",
    "\\end{align}\n",
    "\n",
    "First, you can calculate the value of $\\mathbf{z_1}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z1 = np.dot(W1, x) + b1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> `np.dot` is numpy's function for matrix multiplication.\n",
    "\n",
    "As expected you get an $N$ by 1 matrix, or column vector with $N$ elements, where $N$ is equal to the embedding size, which is 3 in this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can now take the ReLU of $\\mathbf{z_1}$ to get $\\mathbf{h}$, the vector with the values of the hidden layer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = relu(z1)\n",
    "h"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Applying ReLU means that the negative element of $\\mathbf{z_1}$ has been replaced with a zero."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Values of the output layer\n",
    "\n",
    "Here are the formulas you need to calculate the values of the output layer, represented by the vector $\\mathbf{\\hat y}$:\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{z_2} &= \\mathbf{W_2}\\mathbf{h} + \\mathbf{b_2}   \\tag{3} \\\\\n",
    " \\mathbf{\\hat y} &= \\mathrm{softmax}(\\mathbf{z_2})   \\tag{4} \\\\\n",
    "\\end{align}\n",
    "\n",
    "**First, calculate $\\mathbf{z_2}$.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "z2 = np.dot(W2, h) + b2\n",
    "\n",
    "z2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[-0.31973737],\n",
    "           [-0.28125477],\n",
    "           [-0.09838369],\n",
    "           [-0.33512159],\n",
    "           [-0.19919612]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is a $V$ by 1 matrix, where $V$ is the size of the vocabulary, which is 5 in this example."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now calculate the value of $\\mathbf{\\hat y}$.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_hat = softmax(z2)\n",
    "\n",
    "y_hat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.18519074],\n",
    "           [0.19245626],\n",
    "           [0.23107446],\n",
    "           [0.18236353],\n",
    "           [0.20891502]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you've performed the calculations with random matrices and vectors (apart from the input vector), the output of the neural network is essentially random at this point. The learning process will adjust the weights and biases to match the actual targets better.\n",
    "\n",
    "**That being said, what word did the neural network predict?**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Solution</b></font>\n",
    "</summary>\n",
    "<p>The neural network predicted the word \"happy\": the largest element of $\\mathbf{\\hat y}$ is the third one, and the third word of the vocabulary is \"happy\".</p>\n",
    "<p>Here's how you could implement this in Python:</p>\n",
    "<p><code>print(Ind2word[np.argmax(y_hat)])</code></p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Well done, you've completed the forward propagation phase!\n",
    "\n",
    "## Cross-entropy loss\n",
    "\n",
    "Now that you have the network's prediction, you can calculate the cross-entropy loss to determine how accurate the prediction was compared to the actual target.\n",
    "\n",
    "> Remember that you are working on a single training example, not on a batch of examples, which is why you are using *loss* and not *cost*, which is the generalized form of loss.\n",
    "\n",
    "First let's recall what the prediction was."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_hat"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And the actual target value is:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The formula for cross-entropy loss is:\n",
    "\n",
    "$$ J=-\\sum\\limits_{k=1}^{V}y_k\\log{\\hat{y}_k} \\tag{6}$$\n",
    "\n",
    "**Implement the cross-entropy loss function.**\n",
    "\n",
    "Here are a some hints if you're stuck."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Hint 1</b></font>\n",
    "</summary>\n",
    "    <p>To multiply two numpy matrices (such as <code>y</code> and <code>y_hat</code>) element-wise, you can simply use the <code>*</code> operator.</p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<details>    \n",
    "<summary>\n",
    "    <font size=\"3\" color=\"darkgreen\"><b>Hint 2</b></font>\n",
    "</summary>\n",
    "<p>Once you have a vector equal to the element-wise multiplication of <code>y</code> and <code>y_hat</code>, you can use <code>np.sum</code> to calculate the sum of the elements of this vector.</p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def cross_entropy_loss(y_predicted, y_actual):\n",
    "    loss = np.sum(-np.log(y_predicted)*y_actual)\n",
    "    \n",
    "    return loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now use this function to calculate the loss with the actual values of $\\mathbf{y}$ and $\\mathbf{\\hat y}$.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cross_entropy_loss(y_hat, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    1.4650152923611106"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This value is neither good nor bad, which is expected as the neural network hasn't learned anything yet.\n",
    "\n",
    "The actual learning will start during the next phase: backpropagation.\n",
    "\n",
    "## Backpropagation\n",
    "\n",
    "The formulas that you will implement for backpropagation are the following.\n",
    "\n",
    "\\begin{align}\n",
    " \\frac{\\partial J}{\\partial \\mathbf{W_1}} &= \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right )\\mathbf{x}^\\top \\tag{7}\\\\\n",
    " \\frac{\\partial J}{\\partial \\mathbf{W_2}} &= (\\mathbf{\\hat{y}} - \\mathbf{y})\\mathbf{h^\\top} \\tag{8}\\\\\n",
    " \\frac{\\partial J}{\\partial \\mathbf{b_1}} &= \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right ) \\tag{9}\\\\\n",
    " \\frac{\\partial J}{\\partial \\mathbf{b_2}} &= \\mathbf{\\hat{y}} - \\mathbf{y} \\tag{10}\n",
    "\\end{align}\n",
    "\n",
    "> Note: these formulas are slightly simplified compared to the ones in the lecture as you're working on a single training example, whereas the lecture provided the formulas for a batch of examples. In the assignment you'll be implementing the latter.\n",
    "\n",
    "Let's start with an easy one.\n",
    "\n",
    "**Calculate the partial derivative of the loss function with respect to $\\mathbf{b_2}$, and store the result in `grad_b2`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{b_2}} = \\mathbf{\\hat{y}} - \\mathbf{y} \\tag{10}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grad_b2 = y_hat - y\n",
    "\n",
    "grad_b2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[ 0.18519074],\n",
    "           [ 0.19245626],\n",
    "           [-0.76892554],\n",
    "           [ 0.18236353],\n",
    "           [ 0.20891502]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Next, calculate the partial derivative of the loss function with respect to $\\mathbf{W_2}$, and store the result in `grad_W2`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{W_2}} = (\\mathbf{\\hat{y}} - \\mathbf{y})\\mathbf{h^\\top} \\tag{8}$$\n",
    "\n",
    "> Hint: use `.T` to get a transposed matrix, e.g. `h.T` returns $\\mathbf{h^\\top}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grad_W2 = np.dot(y_hat - y, h.T)\n",
    "\n",
    "grad_W2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[ 0.06756476,  0.11798563,  0.        ],\n",
    "           [ 0.0702155 ,  0.12261452,  0.        ],\n",
    "           [-0.28053384, -0.48988499,  0.        ],\n",
    "           [ 0.06653328,  0.1161844 ,  0.        ],\n",
    "           [ 0.07622029,  0.13310045,  0.        ]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Now calculate the partial derivative with respect to $\\mathbf{b_1}$ and store the result in `grad_b1`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{b_1}} = \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right ) \\tag{9}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grad_b1 = relu(np.dot(W2.T, y_hat - y))\n",
    "\n",
    "grad_b1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.        ],\n",
    "           [0.        ],\n",
    "           [0.17045858]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Finally, calculate the partial derivative of the loss with respect to $\\mathbf{W_1}$, and store it in `grad_W1`.**\n",
    "\n",
    "$$\\frac{\\partial J}{\\partial \\mathbf{W_1}} = \\rm{ReLU}\\left ( \\mathbf{W_2^\\top} (\\mathbf{\\hat{y}} - \\mathbf{y})\\right )\\mathbf{x}^\\top \\tag{7}$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "grad_W1 = np.dot(relu(np.dot(W2.T, y_hat - y)), x.T)\n",
    "\n",
    "grad_W1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[0.        , 0.        , 0.        , 0.        , 0.        ],\n",
    "           [0.        , 0.        , 0.        , 0.        , 0.        ],\n",
    "           [0.04261464, 0.04261464, 0.        , 0.08522929, 0.        ]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before moving on to gradient descent, double-check that all the matrices have the expected dimensions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(f'V (vocabulary size): {V}')\n",
    "print(f'N (embedding size / size of the hidden layer): {N}')\n",
    "print(f'size of grad_W1: {grad_W1.shape} (NxV)')\n",
    "print(f'size of grad_b1: {grad_b1.shape} (Nx1)')\n",
    "print(f'size of grad_W2: {grad_W2.shape} (VxN)')\n",
    "print(f'size of grad_b2: {grad_b2.shape} (Vx1)')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gradient descent\n",
    "\n",
    "During the gradient descent phase, you will update the weights and biases by subtracting $\\alpha$ times the gradient from the original matrices and vectors, using the following formulas.\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{W_1} &:= \\mathbf{W_1} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{W_1}} \\tag{11}\\\\\n",
    " \\mathbf{W_2} &:= \\mathbf{W_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{W_2}} \\tag{12}\\\\\n",
    " \\mathbf{b_1} &:= \\mathbf{b_1} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_1}} \\tag{13}\\\\\n",
    " \\mathbf{b_2} &:= \\mathbf{b_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_2}} \\tag{14}\\\\\n",
    "\\end{align}\n",
    "\n",
    "First, let set a value for $\\alpha$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "alpha = 0.03"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The updated weight matrix $\\mathbf{W_1}$ will be:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W1_new = W1 - alpha * grad_W1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's compare the previous and new values of $\\mathbf{W_1}$:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('old value of W1:')\n",
    "print(W1)\n",
    "print()\n",
    "print('new value of W1:')\n",
    "print(W1_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The difference is very subtle (hint: take a closer look at the last row), which is why it takes a fair amount of iterations to train the neural network until it reaches optimal weights and biases starting from random values.\n",
    "\n",
    "**Now calculate the new values of $\\mathbf{W_2}$ (to be stored in `W2_new`), $\\mathbf{b_1}$ (in `b1_new`), and $\\mathbf{b_2}$ (in `b2_new`).**\n",
    "\n",
    "\\begin{align}\n",
    " \\mathbf{W_2} &:= \\mathbf{W_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{W_2}} \\tag{12}\\\\\n",
    " \\mathbf{b_1} &:= \\mathbf{b_1} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_1}} \\tag{13}\\\\\n",
    " \\mathbf{b_2} &:= \\mathbf{b_2} - \\alpha \\frac{\\partial J}{\\partial \\mathbf{b_2}} \\tag{14}\\\\\n",
    "\\end{align}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W2_new = W2 - alpha * grad_W2\n",
    "b1_new = b1 - alpha * grad_b1\n",
    "b2_new = b2 - alpha * grad_b2\n",
    "\n",
    "print('W2_new')\n",
    "print(W2_new)\n",
    "print()\n",
    "print('b1_new')\n",
    "print(b1_new)\n",
    "print()\n",
    "print('b2_new')\n",
    "print(b2_new)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    W2_new\n",
    "    [[-0.22384758 -0.43362588  0.13310965]\n",
    "     [ 0.08265956  0.0775535   0.1772054 ]\n",
    "     [ 0.19557112 -0.04637608 -0.1790735 ]\n",
    "     [ 0.06855622 -0.02363691  0.36107434]\n",
    "     [ 0.33251813 -0.3982269  -0.43959196]]\n",
    "\n",
    "    b1_new\n",
    "    [[ 0.09688219]\n",
    "     [ 0.29239497]\n",
    "     [-0.27875802]]\n",
    "\n",
    "    b2_new\n",
    "    [[ 0.02964508]\n",
    "     [-0.36970753]\n",
    "     [-0.10468778]\n",
    "     [-0.35349417]\n",
    "     [-0.0764456 ]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Congratulations, you have completed one iteration of training using one training example!\n",
    "\n",
    "You'll need many more iterations to fully train the neural network, and you can optimize the learning process by training on batches of examples, as described in the lecture. You will get to do this during this week's assignment.\n",
    "\n",
    "## Extracting word embedding vectors\n",
    "\n",
    "Once you have finished training the neural network, you have three options to get word embedding vectors for the words of your vocabulary, based on the weight matrices $\\mathbf{W_1}$ and/or $\\mathbf{W_2}$.\n",
    "\n",
    "### Option 1: extract embedding vectors from $\\mathbf{W_1}$\n",
    "\n",
    "The first option is to take the columns of $\\mathbf{W_1}$ as the embedding vectors of the words of the vocabulary, using the same order of the words as for the input and output vectors.\n",
    "\n",
    "> Note: in this practice notebook the values of the word embedding vectors are meaningless after a single iteration with just one training example, but here's how you would proceed after the training process is complete.\n",
    "\n",
    "For example $\\mathbf{W_1}$ is this matrix:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first column, which is a 3-element vector, is the embedding vector of the first word of your vocabulary. The second column is the word embedding vector for the second word, and so on.\n",
    "\n",
    "The first, second, etc. words are ordered as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for i in range(V):\n",
    "    print(Ind2word[i])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So the word embedding vectors corresponding to each word are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# loop through each word of the vocabulary\n",
    "for word in word2Ind:\n",
    "    # extract the column corresponding to the index of the word in the vocabulary\n",
    "    word_embedding_vector = W1[:, word2Ind[word]]\n",
    "    \n",
    "    print(f'{word}: {word_embedding_vector}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 2: extract embedding vectors from $\\mathbf{W_2}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second option is to take $\\mathbf{W_2}$ transposed, and take its columns as the word embedding vectors just like you did for $\\mathbf{W_1}$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W2.T"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# loop through each word of the vocabulary\n",
    "for word in word2Ind:\n",
    "    # extract the column corresponding to the index of the word in the vocabulary\n",
    "    word_embedding_vector = W2.T[:, word2Ind[word]]\n",
    "    \n",
    "    print(f'{word}: {word_embedding_vector}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Option 3: extract embedding vectors from $\\mathbf{W_1}$ and $\\mathbf{W_2}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The third option, which is the one you will use in this week's assignment, uses the average of $\\mathbf{W_1}$ and $\\mathbf{W_2^\\top}$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Calculate the average of $\\mathbf{W_1}$ and $\\mathbf{W_2^\\top}$, and store the result in `W3`.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "W3 = (W1+W2.T)/2\n",
    "\n",
    "W3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Expected output:\n",
    "\n",
    "    array([[ 0.09752647,  0.08665397, -0.02389858,  0.1768788 ,  0.3764029 ],\n",
    "           [-0.05136565,  0.15459171, -0.15029611,  0.19580601, -0.31673866],\n",
    "           [ 0.19974284, -0.03063173, -0.27839106,  0.12353994, -0.04975536]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Extracting the word embedding vectors works just like the two previous options, by taking the columns of the matrix you've just created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# loop through each word of the vocabulary\n",
    "for word in word2Ind:\n",
    "    # extract the column corresponding to the index of the word in the vocabulary\n",
    "    word_embedding_vector = W3[:, word2Ind[word]]\n",
    "    \n",
    "    print(f'{word}: {word_embedding_vector}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You're now ready to take on this week's assignment!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### How this practice relates to and differs from the upcoming graded assignment\n",
    "\n",
    "- In the assignment, for each iteration of training you will use batches of examples instead of a single example. The formulas for forward propagation and backpropagation will be modified accordingly, and you will use cross-entropy cost instead of cross-entropy loss.\n",
    "- You will also complete several iterations of training, until you reach an acceptably low cross-entropy cost, at which point you can extract good word embeddings from the weight matrices.\n",
    "- After extracting the word embedding vectors, you will use principal component analysis (PCA) to visualize the vectors, which will enable you to perform an intrinsic evaluation of the quality of the vectors, as explained in the lecture."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Congratulations on finishing all lecture notebooks for this week!** \n",
    "\n",
    "You're now ready to take on this week's assignment!\n",
    "\n",
    "**Keep it up!**"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}