started lec 38 on plotting

s24/Louis_Lecture_Notes/38_Plotting3/Lec38_Plotting3_Template_Oliphant.ipynb

+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 0: Imports"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sqlite3\n",
+    "import pandas as pd\n",
+    "from pandas import DataFrame, Series\n",
+    "import matplotlib\n",
+    "from matplotlib import pyplot as plt\n",
+    "matplotlib.rcParams[\"font.size\"] = 16"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 1a: Save all the data from the `\"piazza\"` table to `piazza_df`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 1b: Set the index of `piazza_df` to be `student_id`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 1c: Add a column `\"total\"` to `piazza_df`.\n",
+    "\n",
+    "This should be the sum of the number of `posts`, `answers`, `edits`, `followups`, and `replies_to_followups`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 2a: Create a new dataframe `contributors_df` which contains those that had more than 0 total contributions.\n",
+    "\n",
+    "Sort by this value from **highest** to **lowest**. Break **ties** by `name` in **alphabetical** order."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 2b: Answer the same question using SQL"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 3a: Of those that contributed, what was their average number of contributions?\n",
+    "\n",
+    "Do your analysis by `role` (e.g. by `\"ta\"`, `\"instructor\"`, and `\"student\"`)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 3b: Answer the same question using SQL"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 4: What is the correlation between all of the numeric columns?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Warmup 5: Close the connection"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Plotting Applications\n",
+    "\n",
+    "**Learning Objectives**\n",
+    "\n",
+    "- Make a line plot on a series or on a DataFrame\n",
+    "- Apply features of line plots and bar plots to visualize results of data investigations\n",
+    "- Clean Series data by dropping NaN values and by converting to int\n",
+    "- Make a stacked bar plot"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Line plots\n",
+    "- `SERIES.plot.line()`\n",
+    "- `DATAFRAME.plot.line()`    each column in the data frame becomes a line in the plot\n",
+    "\n",
+    "https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.plot.line.html"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### When you make a Series from a list, the default indices are: 0, 1, 2, ..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "s = Series([1758, 2002, 2408, 2898, 3814, 4803, 5713, 6661, 7618, 8391, 8764]) # y values\n",
+    "s.plot.line()\n",
+    "s"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### You can make a series from a list and add indices"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "s = Series([1758, 2002, 2408, 2898, 3814, 4803, 5713, 6661, 7618, 8391, 8764], \\\n",
+    "           index=[2010, 2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020])\n",
+    "s.plot.line()\n",
+    "s"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### We can save the AxesSubplot and \"beautify\" it like the other plots..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ax = s.plot.line()\n",
+    "ax.set_title(\"Number of Craft Breweries in the USA\")\n",
+    "ax.set_xlabel(\"Year\")\n",
+    "ax.set_ylabel(\"# Craft Breweries\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Be careful! If the indices are out of order you get a MESS!\n",
+    "\n",
+    "Pandas plots each `(index, value)` in the order given"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "s = Series([1758, 2408, 2898, 3814, 4803, 5713, 6661, 7618, 8391, 8764, 2002], \\\n",
+    "           index=[2010, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2011])\n",
+    "s.plot.line()\n",
+    "s"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### You can fix this by calling `sort_index`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "s.sort_index().plot.line()\n",
+    "s.sort_index()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Plotting lines from a DataFrame"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### City of Madison normal high and low (degrees F) by month"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "temp_df = DataFrame( \n",
+    "    {\n",
+    "    \"high\": [26, 31, 43, 57, 68, 78, 82, 79, 72, 59, 44, 30],\n",
+    "    \"low\": [11, 15, 25, 36, 46, 56, 61, 59, 50, 39, 28, 16]     }\n",
+    ")\n",
+    "temp_df"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### A Line Plot made from a DataFrame automatically plots all columns\n",
+    "\n",
+    "The same is true for bar plots; we'll see this later."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ax = temp_df.plot.line(figsize=(12, 4))\n",
+    "ax.set_title(\"Average Temperatures in Madison, WI\")\n",
+    "ax.set_xlabel(\"Month\")\n",
+    "ax.set_ylabel(\"Temp (Fahrenheit)\")\n",
+    "ax.set_xticks(range(12))   # makes a range from 0 to 11\n",
+    "ax.set_xticklabels([\"Jan\", \"Feb\", \"Mar\", \"Apr\", \"May\", \"Jun\",\n",
+    "                   \"Jul\", \"Aug\", \"Sep\", \"Oct\", \"Nov\", \"Dec\"])\n",
+    "\n",
+    "None # This gets rid of the weird output"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### We can also explicitly pass the `x` and `y` parameters..."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "temp_df_with_month = DataFrame( \n",
+    "    {\n",
+    "    \"month\": [\"Jan\", \"Feb\", \"Mar\", \"Apr\", \"May\", \"Jun\",\n",
+    "                   \"Jul\", \"Aug\", \"Sep\", \"Oct\", \"Nov\", \"Dec\"],\n",
+    "    \"high\": [26, 31, 43, 57, 68, 78, 82, 79, 72, 59, 44, 30],\n",
+    "    \"low\": [11, 15, 25, 36, 46, 56, 61, 59, 50, 39, 28, 16]     }\n",
+    ")\n",
+    "\n",
+    "ax = temp_df_with_month.plot.line(x=\"month\", y=[\"high\", \"low\"], figsize=(12, 4))\n",
+    "ax.set_title(\"Average Temperatures in Madison, WI\")\n",
+    "ax.set_xlabel(\"Month\")\n",
+    "ax.set_ylabel(\"Temp (Fahrenheit)\")"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### We can perform a calculation on an entire DataFrame\n",
+    "Let's change the entire DataFrame to Celcius"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def f_to_c(f):\n",
+    "    return (f - 32)*5/9\n",
+    "\n",
+    "celcius_df = f_to_c(temp_df)\n",
+    "celcius_df"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### We can also do that using a lambda function"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "celcius_df = (lambda f: (f - 32)*5/9)(temp_df)\n",
+    "celcius_df"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### Here is one way to add a horizontal line to our line plots"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "celcius_df[\"freezing\"] = 0\n",
+    "celcius_df"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### This plots each column as lines with rotation for the tick labels"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ax = celcius_df.plot.line(y=[\"high\", \"low\", \"freezing\"], figsize = (12,4))\n",
+    "ax.set_xlabel(\"Month\")\n",
+    "ax.set_ylabel(\"Temp (Celcius)\")\n",
+    "ax.set_xticks(range(12))\n",
+    "ax.set_xticklabels([\"Jan\", \"Feb\", \"Mar\", \"Apr\", \"May\", \"Jun\",\n",
+    "                    \"Jul\", \"Aug\", \"Sep\", \"Oct\", \"Nov\", \"Dec\"], rotation=45)\n",
+    "None"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Bar Plot Example w/ Fire Hydrants\n",
+    "\n",
+    "- General review of Pandas\n",
+    "- Some new Bar Plot options\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "hdf = pd.read_csv(\"Fire_Hydrants.csv\")\n",
+    "hdf.tail()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exercise 1: Find just the column names of `hdf`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Let's create a *bar plot* to visualize *colors* of fire hydrants."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exercise 2: Make a series called `counts_series` which stores the value counts of the `\"nozzle_color\"`"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "#### We can see that the data needs to be cleaned, since colors seem to be repeating with different cases..."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exercise 3: Clean the `\"nozzle_color\"` data by changing the case to lower case before finding the value counts."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exercise 4: Make a horizontal bar plot of counts of colors and have the colors match"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Let's create a *bar plot* to visualize *style* of fire hydrants."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exercise 5: Repeat the same as above with `\"Style\"` instead\n",
+    "\n",
+    "Make your own decisions on how to present the data, and \"beautify\" it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Exercise 6: Plot the decade manufactured for the Pacer Style as opposed to other styles\n",
+    "\n",
+    "1. Get the year of manufacture of all `\"Pacer\"` style hydrants.\n",
+    "2. Get the year of manufacture of all other hydrants.\n",
+    "3. Get the decade of manufacture of all `\"Pacer\"` style hydrants as well as other hydrants.\n",
+    "4. Drop missing data, and find the value counts of the decade of manufacture for both `\"Pacer\"` style hydrants as well as other hydrants.\n",
+    "5. Create a DataFrame with all the data, and make a stacked bar plot. Beautify it."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# write your code here"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.5"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}