6_Intro_to_NLP.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.7.6"
    },
    "colab": {
      "name": "Intro_to_NLP.ipynb",
      "provenance": [],
      "toc_visible": true,
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/SiP-AI-ML/LessonMaterials/blob/master/Intro_to_NLP.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wvdjN-w1gojl",
        "colab_type": "text"
      },
      "source": [
        "# 1. What is NLP?\n",
        "We edited these materials from the FastAI NLP Course, so full credit to them.\n",
        "Here is the repo: https://github.com/fastai/course-nlp\n",
        "We also referenced a blog from Algorithma: https://algorithmia.com/blog/introduction-natural-language-processing-nlp\n",
        "\n",
        "Natural language processing (NLP) is a field of artificial intelligence in which computers analyze, understand, and derive meaning from human language in a smart and useful way. By utilizing NLP, developers can organize and structure knowledge to perform tasks such as automatic summarization, translation, named entity recognition, relationship extraction, sentiment analysis, speech recognition, and topic segmentation.\n",
        "\n",
        "Some examples of NLP are with common products such as: \n",
        "- Amazon Alexa/Google Home\n",
        "- Spam detectors\n",
        "- Google Search\n",
        "- Lemmatization\n",
        "\n",
        "Besides common word processors, NLP considers the hierarchical structure of language.\n",
        "-  several words make a phrase\n",
        "- several phrases make a sentence\n",
        "- sentences convey ideas \n",
        "\n",
        "John Rehling, an NLP expert at Meltwater Group, said in How Natural Language Processing Helps Uncover Social Media Sentiment. “By analyzing language for its meaning, NLP systems have long filled useful roles, such as correcting grammar, converting speech to text and automatically translating between languages.”\n",
        "\n",
        "NLP is used to analyze text, allowing machines to understand how humans speak. This human-computer interaction enables real-world applications like automatic text summarization, sentiment analysis, topic extraction, named entity recognition, parts-of-speech tagging, relationship extraction, stemming, and more. NLP is commonly used for text mining, machine translation, and automated question answering.\n",
        "\n",
        "NLP is very difficult in computer science because human language is rarely precise or plainly spoken. To understand human language is to understand not only the words, but the concepts and how they’re linked together to create meaning. Despite language being one of the easiest things for the human mind to learn, the ambiguity of language is what makes natural language processing a difficult problem for computers to master."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RqVUx5Dzgojn",
        "colab_type": "text"
      },
      "source": [
        "## What can you do with NLP?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "3hDUJg_Pgojo",
        "colab_type": "text"
      },
      "source": [
        "NLP algorithms have a variety of uses. Basically, they allow developers to create a software that understands human language. Due to the complicated nature of human language, NLP can be difficult to learn and implement correctly. However, with the knowledge gained from this article, you will be better equipped to use NLP successfully. Some of the projects developers can use NLP algorithms for are:\n",
        "\n",
        "- Part-of-speech tagging: identify if each word is a noun, verb, adjective, etc.\n",
        "- Named entity recognition NER: identify person names, organizations, locations, medical codes, time expressions, quantities, monetary values, etc.\n",
        "- Question answering\n",
        "- Speech recognition\n",
        "- Text-to-speech and Speech-to-text\n",
        "- Topic modeling\n",
        "- Sentiment classification\n",
        "- Language modeling\n",
        "- Translation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CaYeqegogojp",
        "colab_type": "text"
      },
      "source": [
        "Many techniques from NLP are useful in a variety of places, for instance, you may have text within your tabular data.\n",
        "\n",
        "There are also interesting techniques that let you go between text and images:"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "jmuGvP_Vgojs",
        "colab_type": "text"
      },
      "source": [
        "## A changing field"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Y3HTqhmAgojt",
        "colab_type": "text"
      },
      "source": [
        "Historically, NLP originally relied on hard-coded rules about a language. In the 1990s, there was a change towards using statistical & machine learning approaches, but the complexity of natural language meant that simple statistical approaches were often not state-of-the-art. We are now currently in the midst of a major change in the move towards neural networks.  Because deep learning allows for much greater complexity, it is now achieving state-of-the-art for many things.\n",
        "\n",
        "This doesn't have to be binary: there is room to combine deep learning with rules-based approaches."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "hm7Eka2ygoj7",
        "colab_type": "text"
      },
      "source": [
        "## Here is a quick Intro Video\n",
        "https://www.youtube.com/watch?v=5ctbvkAMQO4\n",
        "Stop at 5:25"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Gzsi94zXgoj7",
        "colab_type": "text"
      },
      "source": [
        "## Resources"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Zm4H74aggoj8",
        "colab_type": "text"
      },
      "source": [
        "**Books**\n",
        "\n",
        "Here are a few helpful references if you want to get really in-depth: (Note, these are really advanced)\n",
        "\n",
        "- [**Speech and Language Processing**](https://web.stanford.edu/~jurafsky/slp3/), by Dan Jurafsky and James H. Martin (free PDF)\n",
        "\n",
        "- [**Introduction to Information Retrieval**](https://nlp.stanford.edu/IR-book/html/htmledition/irbook.html) by By Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze (free online)\n",
        "\n",
        "- [**Natural Language Processing with PyTorch**](https://learning.oreilly.com/library/view/natural-language-processing/9781491978221/) by Brian McMahan and Delip Rao (need to purchase or have O'Reilly Safari account) \n",
        "\n",
        "**Blogs**\n",
        "\n",
        "Good NLP-related blogs:\n",
        "- [Sebastian Ruder](http://ruder.io/)\n",
        "- [Joyce Xu](https://medium.com/@joycex99)\n",
        "- [Jay Alammar](https://jalammar.github.io/)\n",
        "- [Stephen Merity](https://smerity.com/articles/articles.html)\n",
        "- [Rachael Tatman](https://towardsdatascience.com/evaluating-text-output-in-nlp-bleu-at-your-own-risk-e8609665a213)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "8lSfftvKgoj9",
        "colab_type": "text"
      },
      "source": [
        "## NLP Tools"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "A6Ec1jSOgoj-",
        "colab_type": "text"
      },
      "source": [
        "- Regex (example: find all phone numbers: 123-456-7890, (123) 456-7890, etc.)\n",
        "- Tokenization: splitting your text into meaningful units (has a different meaning in security)\n",
        "- Lemmatization: grouping together the different conjugations of a word so they can be analyzed together \n",
        "- Word embeddings\n",
        "- Linear algebra/matrix decomposition\n",
        "- Neural nets\n",
        "- Hidden Markov Models\n",
        "- Parse trees"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qVJrYOEPgoj-",
        "colab_type": "text"
      },
      "source": [
        "## Python Libraries"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "kClVSvqMgoj_",
        "colab_type": "text"
      },
      "source": [
        "- [nltk](https://www.nltk.org/): first released in 2001, very broad NLP library\n",
        "- [spaCy](https://spacy.io/): creates parse trees, excellent tokenizer, opinionated\n",
        "- [gensim](https://radimrehurek.com/gensim/): topic modeling and similarity detection\n",
        "\n",
        "specialized tools:\n",
        "- [PyText](https://pytext-pytext.readthedocs-hosted.com/en/latest/)\n",
        "- [fastText](https://fasttext.cc/) has library of embeddings\n",
        "\n",
        "general ML/DL libraries with text features:\n",
        "- [sklearn](https://scikit-learn.org/stable/): general purpose Python ML library\n",
        "- [fastai](https://docs.fast.ai/): fast & accurate neural nets using modern best practices, on top of PyTorch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "collapsed": true,
        "id": "BaTTktK1goj_",
        "colab_type": "text"
      },
      "source": [
        "## Ethics issues"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "KlddDSoMgokA",
        "colab_type": "text"
      },
      "source": [
        "### Bias\n",
        "\n",
        "- [How Vector Space Mathematics Reveals the Hidden Sexism in Language](https://www.technologyreview.com/s/602025/how-vector-space-mathematics-reveals-the-hidden-sexism-in-language/)\n",
        "- [Semantics derived automatically from language corpora contain human-like biases](https://arxiv.org/abs/1608.07187)\n",
        "- [Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings But do not Remove Them](https://arxiv.org/abs/1903.03862)\n",
        "- [Word Embeddings, Bias in ML, Why You Don't Like Math, & Why AI Needs You](https://www.youtube.com/watch?v=25nC0n9ERq4&list=PLtmWHNX-gukLQlMvtRJ19s7-8MrnRV6h6&index=9)"
      ]
    }
  ]
}