00_overview.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5b023f58-24e2-4dae-9ce7-26c906a065dc",
   "metadata": {
    "tags": []
   },
   "source": [
    "<div align=\"center\">\n",
    "    <h1>Welcome to the Nebari tutorial!</h1>\n",
    "    <h2> PyData NYC, 2022</h2>\n",
    "    <hr>\n",
    "    <img src=\"./assets/nebari-banner.svg\" alt=\"Nebari logo mark with the tagline Your open source data science platform: Built for scale, designed for collaboration.\" width=\"70%\">\n",
    "    <hr>\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cbc3e05d-460c-4ee4-b26a-060c6e52b718",
   "metadata": {
    "tags": []
   },
   "source": [
    "## Tutorial structure\n",
    "\n",
    "### 💎 Key takeaways\n",
    "\n",
    "At the end of this tutorial, you will:\n",
    "\n",
    "- **Understand what Nebari is,**\n",
    "- **Be able to use some of it's core features, and**\n",
    "- **Know how to deploy your own Nebari cluster.**\n",
    "\n",
    "Ultimately, this only works if you're engaged and having fun, so ask questions and try things for yourself! 😊\n",
    "\n",
    "### Outline\n",
    "\n",
    "This tutorial will be 90 minutes long and will include plenty of opportunities to explore and play with a live Nebari instance. We will cover:\n",
    "\n",
    "0. Overview of Nebari (this notebook)\n",
    "1. How to log in and intro to the available interfaces\n",
    "2. Intro to conda-store and environment management\n",
    "3. Intro to Dask and Dask-Gateway and scaling your Python code\n",
    "4. Intro to ContainDS Dashboards and dashboard sharing\n",
    "5. How to install, initialize and deploy a Nebari cluster\n",
    "6. Conclusion and Q&A\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff26ac60-77be-4def-8f71-e19109019da0",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "slideshow": {
     "slide_type": "slide"
    },
    "tags": []
   },
   "source": [
    "## 📌 Housekeeping\n",
    "\n",
    "<details>\n",
    "    <summary>First time login instructions</summary>\n",
    "\n",
    "### Navigate to [demo.nebari.dev](https://demo.nebari.dev)\n",
    "\n",
    "1. Click on the `Sign in with Keycloak` button.\n",
    "\n",
    "2. Click `Register` at the bottom of the sign in prompt.\n",
    "\n",
    "3. Fill out the short form to register as a new user.\n",
    "\n",
    "> Your account will be active through the weekend for those who wish to play with the platform some more.\n",
    "\n",
    "> The email address will serve as your username. We are not storing your email address and it will never be shared with anyone. **You can also use a fake address here**.\n",
    "\n",
    "4. In the Hub page that opens automatically, click on the `Start My Server` button.\n",
    "\n",
    "5. For `Server Options`, select an instance type, then click `Start`.\n",
    "\n",
    "> If your server doesn't launch within 30 seconds, this means the cluster is in the process of auto-scaling to meet the new demand. Please be patient, this might take up to several minutes.\n",
    "\n",
    "6. From here, you can clone this repo in two ways, from the Jupyter terminal or from the JupyterLab git extention.\n",
    "\n",
    "> Clone this repo from the Jupyter terminal: `git clone https://github.com/nebari-dev/nebari-demo`\n",
    "\n",
    "> Or clone this repo using JupyterLab git extention.\n",
    "\n",
    "7. Finally, open the `00_overview.ipynb` notebook to follow along.\n",
    "\n",
    "</details>\n",
    "\n",
    "\n",
    "<details>\n",
    "    <summary>Demostrations and opportunities explore on your own</summary>\n",
    "    \n",
    "## Demostrations and opportunities explore on your own\n",
    "\n",
    "In several of the notebooks, you might see the following callouts.\n",
    "\n",
    "### 👀 Watch this:\n",
    "\n",
    "In these sections, the presenter will demonstrate a slightly more in-depth overview of a particular feature. During this time, there is nothing needed from you other than your attention.\n",
    "\n",
    "### 👉 Your turn:\n",
    "\n",
    "In these sections, attendees will be given several minutes to work through a set of basic exercises or explore a particular feature. Feel free to do so independently or with your neighbors.\n",
    "    \n",
    "</details>\n",
    "\n",
    "<details>\n",
    "    <summary>Asking questions and requesting assistance</summary>\n",
    "    \n",
    "## Asking questions and requesting assistance\n",
    "\n",
    "If you have a question about the topic being covered, please raise your hand. \n",
    "\n",
    "If you run into technical issues during any of the sections, please place a sticky note on the backside of your monitor and someone will come around to assist you. \n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6906a83a-e3cd-4395-bf6e-5eadb77cdee9",
   "metadata": {
    "tags": []
   },
   "source": [
    "## What is Nebari?\n",
    "\n",
    "**An opinionated, open source, and scalable all-in-one data science platform that allows your team to work efficiently and collaboratively.**\n",
    "\n",
    "Nebari was made for teams who need a ready-to-go data science platform, but who don't want to spend a whole bunch of time setting up, configuring, and managing the required infrastructure.\n",
    "<div align=\"center\">\n",
    "    <img src=\"./assets/nebari-why-choose.png\" alt=\"Image with reasons to deploy Nebari\" width=\"80%\"></img>\n",
    "</div>\n",
    "\n",
    "The term \"Nebari\" means:\n",
    "> visible spread of roots above the growing medium at the base of a bonsai\n",
    "> \n",
    "> ~ Source: [Bonsai aesthetics on Wikipedia](https://en.wikipedia.org/wiki/Bonsai_aesthetics#Nebari)\n",
    "\n",
    "<details>\n",
    "    <summary>Under the hood</summary>\n",
    "    \n",
    "### Under the hood\n",
    "\n",
    "At it's core, Nebari is a [JupyterHub](https://github.com/jupyterhub/jupyterhub) distribution.\n",
    "    \n",
    "Nebari is a declarative, infrastructure-as-code approach to deploying and managing [Zero-to-JupyterHub (Z2JH) for Kubernetes](https://z2jh.jupyter.org/en/stable/). The end-to-end deployment process is handled by [Terraform](https://www.terraform.io/) which provides a configurable way of provisioning all of the necessary resources and whose inputs are all centralized in a single configuration file, the `nebari-config.yaml`. \n",
    "\n",
    "By leveraging Terraform, a Nebari cluster can be deployed from scratch on [AWS](https://nebari-docs.netlify.app/docs/how-tos/nebari-aws), [Azure](https://nebari-docs.netlify.app/docs/how-tos/nebari-azure), [Digital Ocean](https://nebari-docs.netlify.app/docs/how-tos/nebari-do) or [GCP](https://nebari-docs.netlify.app/docs/how-tos/nebari-gcp), or one an existing Kubernetes cluster on most other major cloud providers. Using Terraform also provides a relatively simple and declarative approach to adding new features and services to the platform.\n",
    "\n",
    "The details of how to install, initialize and deploy a Nebari cluster will be covered in [05_setup.ipynb](./05_setup.ipynb).\n",
    "\n",
    "This deployment approach taken by Nebari allows those with little-to-no DevOps experience to deploy and manage a production-ready data science platform that would otherwise require users with expertise in JupyterHub, Kubernetes, cloud infrastructure and other DevOps technologies. \n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01194176-cd95-475e-9da8-f992958d52b3",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Features and integrations\n",
    "\n",
    "<div align=\"center\">\n",
    "    <img src=\"./assets/nebari-integrations.png\" alt=\"List of Nebari integrations\" width=\"80%\"></img>\n",
    "</div>\n",
    "\n",
    "The core features and integrations covered in this tutorial include:\n",
    "- Access to a **shared filesystem** to collaborate with your team.\n",
    "- **VS Code** in the browser with [code-server](https://github.com/coder/code-server).\n",
    "- [**JupyterHub-SSH**](https://github.com/yuvipanda/jupyterhub-ssh) to securely connect to the cluster remotely.\n",
    "- Access to a **scalable+distributed compute clusters** with [Dask](https://dask.org) and [Dask-Gateway](https://gateway.dask.org/).\n",
    "- **Reproducible, yet flexible environments** with [conda-store](https://conda-store.readthedocs.io/en/latest/index.html).\n",
    "- **Sharable dashboards** with [ContainDS Dashboards](https://cdsdashboards.readthedocs.io/en/stable/).\n",
    "\n",
    "Other services and features include:\n",
    "- [**Prometheus**](https://prometheus.io/) + [**Grafana**](https://grafana.com/) as the cluster monitoring solution.\n",
    "- [**Argo-Workflows**](https://argoproj.github.io/workflows/) as the workflow management platform (Kubernetes).\n",
    "- [**kbatch**](https://github.com/kbatch-dev/kbatch) to submit batch jobs/cronjobs from JupyterLab.\n",
    "- [**Jupyter-VideoChat (Jitsi)**](https://github.com/jupyterlab-contrib/jupyter-videochat) as an embedded video-chat service.\n",
    "- [**Prefect**](https://prefect.io) as another workflow management platform (Python). *(not default)*\n",
    "- [**ClearML**](https://clear.ml) as a machine learning experiment tracking and orchestration platform. *(not default)*\n",
    "\n",
    "\n",
    "Along with all of the features listed above, Nebari is platform that takes a **DevOps for non-DevOps people approach**.\n",
    "\n",
    "<div align=\"center\">\n",
    "    <img src=\"./assets/nebari-deploy-anywhere.png\" alt=\"Deploy Nebari easily on the cloud or your local machine\" width=\"70%\"></img>\n",
    "</div>\n",
    "\n",
    "- Nebari **can be deployed anywhere** - local machine<sup>*</sup>, cloud providers (AWS, GCP, Azure, DigitalOcean), or HPC clusters.\n",
    "- Includes team-wide **identity and access management** with [Keycloak](https://www.keycloak.org/)\n",
    "- Built with **security and infrastructure best practices**.\n",
    "- Uses free and open source tools wherever possible for **complete customization**.\n",
    "- **Quick set up** in about 30 mins!\n",
    "\n",
    "We'll look into all of these in more detail in this tutorial!\n",
    "\n",
    "> <sup>*</sup> *local deployments* on Mac OS and Windows OS are currently a work-in-progress."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18625ba3-435a-405f-8c19-f5ee7523af39",
   "metadata": {},
   "source": [
    "_For more information about the project, how to get started, and how to contribute, \n",
    "check out the documentation at [**nebari.dev**](https://www.nebari.dev)._"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff86eee8-5ea8-476a-b812-adc66fe47281",
   "metadata": {},
   "source": [
    "---\n",
    "## 👏 Next:\n",
    "* [01_interface](./01_interface.ipynb)\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d22b232-9a00-4cde-9449-b659ec39f830",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "global-global-nebari-demo",
   "language": "python",
   "name": "conda-env-global-global-nebari-demo-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}