updated lecture to include announcment of enrichment activities

e4a0714f · LOUIS TYRRELL OLIPHANT · 04aa65b0 · e4a0714f
Commit e4a0714f authored 1 month ago by LOUIS TYRRELL OLIPHANT
--- a/s25/Louis_Lecture_Notes/15_CSVs/Lec_15_CSVs.ipynb
+++ b/s25/Louis_Lecture_Notes/15_CSVs/Lec_15_CSVs.ipynb
 {
 "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Announcement\n",
+    "\n",
+    "### CS 220 Enrichment Activities\n",
+    "Students interested in working on a real-world data set and learning about the full data processing pipeline?\n",
+    "\n",
+    "Voluntary working groups will learn about data management, data wrangling/processing, modeling, and reporting/communication skills.\n",
+    "\n",
+    "**When: Thursday, March 6th @ 4pm**\n",
+    "\n",
+    "**Where: Computer Science Room 1325**"
+   ]
+  },
  {
   "cell_type": "markdown",
   "metadata": {},

 %% Cell type:markdown id: tags:

+## Announcement
+
+### CS 220 Enrichment Activities
+Students interested in working on a real-world data set and learning about the full data processing pipeline?
+
+Voluntary working groups will learn about data management, data wrangling/processing, modeling, and reporting/communication skills.
+
+**When: Thursday, March 6th @ 4pm**
+
+**Where: Computer Science Room 1325**
+
+%% Cell type:markdown id: tags:
+
 ## Warmup

 %% Cell type:code id: tags:

 ``` python
 # Warmup 1: Mutating a list

 my_groceries = []

 #use .append() to add an item to the end of a list

 print(my_groceries)

 #use .extend() to add *ALL* items from one list to another list

 print(my_groceries)

 #use .pop() to remove an item at a specific index

 print(my_groceries)

 #use .remove() by take out a value

 print(my_groceries)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Warmup #2: Take a look at these list methods
 # https://www.w3schools.com/python/python_ref_list.asp
 dairy = ["milk", "ice cream", "cheese", "yogurt" ]

 #use the .index() method to get the index of "ice cream"
 ```

 %% Cell type:code id: tags:

 ``` python
 # Warmup #3:  Because a list is a sequence, we can use the 'in' operator
 food_shelf = ["peanut butter", "milk", "bread", "cheese", "YOGURT"]
 for item in food_shelf:
    if ???:  # see if item.lower() is in the dairy list
        print(item, "is dairy")
    else:
        print(item, "is not dairy")
 ```

 %% Cell type:markdown id: tags:

 # Comma-Separated Values (CSV) Files

 ## Readings

 - [Sweigart Ch 16 (through "Reading Data from Reader Objects in a for Loop")](https://automatetheboringstuff.com/2e/chapter16/)

 ## Learning Objectives

 After this lecture you will be able to...

 - Open an Spreadsheet file and export it to a Comma Separated Value file.
 - View a CSV file in Jupyter Lab's *CSV Viewer*.
 - Open a CSV file in TextEditor/Jupyter and connect the elements of the CSV file to the rows and columns in the spreadsheet.

 - Use pre-written Python code to read a CSV file into a list of lists.

 - Write Python statements with double list indexing to access any element of a CSV file via a list of lists.

 - Write code that answers questions about CSV data by writing for loops on lists of lists.

 %% Cell type:markdown id: tags:

 ## CSV File Format

 CSV file format is a non-propriatary format for sharing tables of data.  The data in the CSV file is contained in rows with the values in the rows separated by commas.  It is very common (though not required) to have the top row of a CSV file contain the the header names for the columns of data:

 ```
 Animal,Country,Population
 panda, China,1864
 lemur,Madagascar,2500
 macaque,India,100000
 ```

 You can open CSV files using a spreadsheet program like Microsoft Excel or Apple Numbers.  With Jupyter Lab you can open a CSV file for viewing by doubule-clicking on the file name within Jupyter.  If you want to open it for editing then you need to right-click on the file name and select *Open With -> Editor* in the context menu which will open the file in a text editor.

 %% Cell type:markdown id: tags:

 ## Reading a CSV in Python

 Python has a module for working with CSV files called `csv`.  One common approach for reading in all the data from a CSV file is to load the data as a list of lists.  Look at the function defined in the cell below which takes a file name, loads the data from the file using methods from the csv module and returns a list of lists.

 %% Cell type:code id: tags:

 ``` python
 # Now lets store the contents of the CSV file into Python lists

 # copied from https://automatetheboringstuff.com/chapter14/
 import csv

 def process_csv(filename):
    # open the file, its a text file utf-8
    exampleFile = open(filename, encoding="utf-8")

    # prepare it for reading as a CSV object
    exampleReader = csv.reader(exampleFile)

    # use the built-in list function to convert this into a list of lists
    exampleData = list(exampleReader)

    # close the file to tidy up our workspace
    exampleFile.close()

    # return the list of lists
    return exampleData
 ```

 %% Cell type:markdown id: tags:

 Finish the code in the cell below by useing the `process_csv()` function to load the `colors.csv` file as a list of lists and store the data in the `colors` variable then print the loaded data.  Open the colors.csv file in a separate tab to see data in a spreadsheet type layout and compare what you see in Jupyter Lab with the content on that tab.

 %% Cell type:code id: tags:

 ``` python
 ##TODO load the 'colors.csv' data and store and print the data

 colors = ...

 print(colors)
 ```

 %% Cell type:markdown id: tags:

 You can use double indexing to access specific elements from the loaded data.  Recall that the first indexing will access the row and the second indexing will access the column within that row:

 ```python
 colors[3][0]
 ```
 ```
 Aqua
 ```
 ```python
 colors[3][2]
 ```
 ```
 rgb(0,100,100)
 ```

 On web pages you can describe colors using the color's name, the color's hex value, or the color's rgb value.  The function defined in the cell below accepts any of these three ways of describing a color and displays a tiny swatch of that color.

 %% Cell type:code id: tags:

 ``` python
 from IPython.display import HTML

 def show_color(color):
    color=color.replace(' ','')
    color_html = '<div style="background-color: {}; width: 50px; height: 20px;"></div>'.format(color)
    display(HTML(color_html))
 ```

 %% Cell type:markdown id: tags:

 ### Using Double Indexing
 Finish the code in the cell below by calling `show_color()` with specific values from the `colors` variable.

 %% Cell type:code id: tags:

 ``` python
 ## Display Blanched Almond (index 9) using its name (index 0)
 print(colors[9][0])
 show_color(colors[9][0])

 ## Display Blue Violet (index 11) using the HEX value (index 1)
 print(colors[11][1])
 show_color(colors[11][1])

 ## Display Khaki (index 61) using the rgb value (index 2)
 print(...)
 show_color(...)

 ## You pick a color and display the color swatch using the name, HEX value, and rgb value
 ## also print the name, HEX value, and rgb value

 ```

 %% Cell type:markdown id: tags:

 Now let's work with our survey data.  Finish the code in the cell below to load the data using the `process_csv()` function.  Then  divide the header row off from the data portion, and printer the header and the top 2 rows of the data.

 %% Cell type:code id: tags:

 ``` python
 # Call the process_csv function and store the list of lists in cs220_csv
 cs220_csv = process_csv('cs220_survey_data.csv')
 cs220_csv
 cs220_header = ...
 cs220_data = ...

 print(cs220_header)
 print(...)
 ```

 %% Cell type:markdown id: tags:

 ## CSVs as a List of Lists

 Finish the cells below to access and print the desired item.  Since the csv_data is a list of lists, you will need to use double indexing.

 %% Cell type:code id: tags:

 ``` python
 # Print out the lecture number of the 4th student...by hardcoding its row and column....
 cs220_data[???][???] # [row][col]
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out the sleeping habit for the 2nd student...by hardcoding its row and column....
 cs220_data[???][???]
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out how many students completed the survey.
 len(???)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out every student's sleep habits and major
 for i in range(len(cs220_data)):
    current_sleep_habit = ???
    current_major = ???
    print(current_sleep_habit + '\t\t' + current_major)
 ```

 %% Cell type:code id: tags:

 ``` python
 # FIX: Print out every students' age in 10 years.
 for i in range(???):
    current_age = cs220_data[i][2]
    print(current_age + 10)
 ```

 %% Cell type:markdown id: tags:

 ## It would be nice to have a helper function!
 Let's introduce `cell`

 %% Cell type:code id: tags:

 ``` python
 # Remember creating cs220_header?
 cs220_header
 ```

 %% Cell type:code id: tags:

 ``` python
 # Get the column index of "Pizza Topping"
 cs220_header.index(???)
 ```

 %% Cell type:code id: tags:

 ``` python
 # We want to invoke something like...
 # cell(24, "Cats or Dogs")
 # cell(63, "Zip Code")
 def cell(row_idx, col_name):
    col_idx = ??? # get the index of col_name
    val = ???     # get the value of cs220_data at the specified cell
    return val
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out the lecture number of the 4th student... using the cell function
 cell(???, ???)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out every student's sleep habits and major using the cell function
 for i in range(len(cs220_data)):
    current_sleep_habit = ???
    current_major = ???
    print(current_sleep_habit + '\t\t' + current_major)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out every students' age in 10 years using the cell function
 # ... that didn't really help us here!
 for i in range(len(cs220_data)):
    current_age = cell(i, "Age")
    if current_age != None:
        print(current_age + 10)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Improve the cell function so it returns the appropriate type.
 # If there is nothing in the cell, return None
 def cell(row_idx, col_name):
    col_idx = cs220_header.index(col_name)
    val = cs220_data[row_idx][col_idx]
    if ???:
        return None
    elif ???:
        return int(val)
    else:
        return val
 ```

 %% Cell type:code id: tags:

 ``` python
 # Print out every students' age in 10 years using the cell function
 # ... much better!
 for i in range(len(cs220_data)):
    current_age = cell(i, "Age")
    if current_age != None:
        print(current_age + 10)
 ```

 %% Cell type:code id: tags:

 ``` python
 # Get the average age of each lecture...
 ```

 %% Cell type:markdown id: tags:

 ## You try!

 %% Cell type:markdown id: tags:

 Complete the challenges below. First try completing the problem directly using the list of lists (e.g. double indexing \[\]\[\]), then try using the `cell` function!

 %% Cell type:code id: tags:

 ``` python
 # Of all runners, how many are procrastinators?
 ```

 %% Cell type:code id: tags:

 ``` python
 # What percentage of 18-year-olds have their major declared as "Other"?
 ```

 %% Cell type:code id: tags:

 ``` python
 # Does the oldest basil/spinach-loving Business major prefer cats, dogs, or neither?
 ```

 %% Cell type:markdown id: tags:

 ## Summary

 The `csv` module can be used to load CSV files and store them as a list of lists.  Use double-indexing and helper functions to work with data in this format.