diff --git a/Labs/Lab1/lab1.md b/Labs/Lab1/lab1.md new file mode 100644 index 0000000000000000000000000000000000000000..807ce6888c8e3f3a1ac471a02166ff80b7cc9f03 --- /dev/null +++ b/Labs/Lab1/lab1.md @@ -0,0 +1,11 @@ +# Lab 2: VM Setup + +1. Find your group. You can find your group info in your [Canvas Inbox](https://canvas.wisc.edu/conversations#filter=type=inbox). You can find your group under Canvas - People - Study Group if you want another way to access it. Ask your TA/peer mentor if you need help finding your group members. + +2. Get to know your group members, asking each other the following: + +- Name + major +- What is your hometown, and what is your favorite place there? +- If you could create a silly law, what would it be? + +3. Setup your [virtual machine](./vm) diff --git a/Labs/vm/README.md b/Labs/vm/README.md new file mode 100644 index 0000000000000000000000000000000000000000..76e16ae2e94ba5eac5e200bf9e9e6b88c5a40a26 --- /dev/null +++ b/Labs/vm/README.md @@ -0,0 +1,36 @@ +# Creating Your Virtual Machine + +The lectures, instructions, and projects for this course are designed +for the Linux operating system (*Ubuntu 22.04 LTS Linux*, to be +precise -- there are many flavors/versions!). + +Rather than replace Windows or Mac OS X on your computer, it will be +easier to rent a virtual machine (VM) in the cloud, which you will +then connect to remotely. Knowing how to create virtual machines is +an important data-science skill because it makes your analysis more +*reproducible* -- if your code works in your virtual machine, and +other people know how to reconstruct a similar virtual machine for +themselves (with the same operating system and programs installed), +they're more likely to be able to reproduce the same results by +running your code. + +At the low-end, renting a VM costs about $10-20/month. Fortunately, +the major cloud providers often provide free credit for students and +new users, so you'll likely pay little or (hopefully) nothing this +semester. + +We provide directions for two major cloud providers: GCP (Google's +cloud) and Azure (Microsoft's cloud). If you want to find another +way/place to use Ubuntu 22.04 LTS and install Jupyter, that's fine +too. But we will only support the first +two options during office hours. Even if you are planning to use +your own Ubuntu machine, we recommend you to set up the virtual machine. + +Follow the [GCP Directions](gcp/README.md) for the rest of this lab to set up your +virtual machine. Google gave me educational +credits ($50/student) to use for CS 320, which should cover you. + +Backup option (future directions if you even run out of GCP credits - not needed for now): +[Azure Directions](azure/README.md). At the time these directions +were written, Microsoft offers students $100/year of credit per year. +These directions might be a bit stale. diff --git a/Labs/vm/azure/README.md b/Labs/vm/azure/README.md new file mode 100644 index 0000000000000000000000000000000000000000..5835780a70b0b00a99aaacfebe2a01d02fcf2001 --- /dev/null +++ b/Labs/vm/azure/README.md @@ -0,0 +1,195 @@ +# Azure Option for Virtual Machine + +## Creating an Account + +Go here: https://azure.microsoft.com/en-us/free/students/ + +<img src="img/1-activate.png" width=600> + +Click "Activate now". Click @wisc.edu account if it shows up. +Otherwise, click "use another account" and type your @wisc.edu email. + +<img src="img/2-profile.png" width=600> + +Fill your profile details and click "Next". Check "I agree" and "Sign up". + +You can view the credits remaining on your Azure Student account at https://www.microsoftazuresponsorships.com/. + +## New Virtual Machine + +Search for "Virtual machines" and click it: + +<img src="img/3-search-vms.png" width=600> + +Click the "+Add" button to create a new virtual machine: + +<img src="img/4-add-vm.png" width=600> + +Give your machine a name and choose a region near you ("Central US" +will probably be fastest for remote access for those working from +Wisconsin). Make sure the image is "Ubuntu Server 20.04 LTS - Gen1" +(probably the default already, but double check). + +**Very Important:** change the "Size" to "Standard_B1s", which costs + <$10/month. Larger sizes will quickly burn through any free credit + you might have. + +<img src="img/5-create-vm.png" width=600> + +Other important options: + + - **Authentication type**: "SSH public key" (choose your **username**) + - **SSH public key source**: "Generate new key pair" + - **Key pair name**: "cs320" + - **Select inbound ports**: check all options (80, 443, 22) + +**Important:** write down the **username** you chose + +There are a bunch of other pages that you could get to with buttons +like "Next: Disks" and "Next: Networking". We'll take the defaults, +so feel free to skip these with the "Review + create" button. + +Click "Create", then click "Download private key and create +resource".save the .pem file somewhere safe). Click "Go to resource". + +**Important:** remember where you saved the .pem file. Without it, + you won't be able to access your VM later. + +You'll see a page like the following: + +<img src="img/6-view-vm.png" width=600> + +**Important:** note down the "Public IP address" -- you'll be using it later. + +## Firewall + +What is a firewall? First, a little review from 220. + +Remember from CS 220/301 that a computer (typically) has an *IP address* +(something like 123.234.210.001), and other computers knowing that IP +address can communicate with programs running on that computer. + +If there are multiple running programs, they are differentiated by +*port numbers*. For example, sending a message to 123.234.210.001:220 +sends a message to the program on port 220 running on a computer with +address 123.234.210.001 (note the `:` between the IP address and port +number). Sending a message to 123.234.210.001:320 might send a +message to a different program on that same machine. Analogy: an IP +address is like the address of an apartment building (the computer), +and a port number is like the apartment number of a specific unit; +programs are the residents living in the units. + +A Firewall (among other things) can block access to certain ports for +outsiders, to provide security. We will unblock all ports for our +virtual machine. This is not the most secure option, but it is +convenient, and fine for the purposes of this course. + +Click "Networking" on the page for your VM: + +<img src="img/8-networking.png" width=600> + +Click "Add inbound port rule". The following will pop up on the right: + +<img src="img/9-add-ports.png" width=600> + +Add "*" to the destination port ranges. This is the most convenient +option but not the most secure. It's fine for a class, but not +advisable in other scenarios (e.g., if you were hosting a web +application on the VM). + +## SSH + +A "shell" is a program that runs in the terminal that lets you +navigate through files/directories and run programs. "SSH" stands for +"secure shell", and it is a program that lets you remotely access +another computer (for example, your virtual machine). + +For these directions, make sure you have this information handy (which +we asked to write down earlier): + +1. your username +2. location of the .pem file you downloaded +3. your VM's IP address + +Open the terminal (on Mac, it is called terminal; on Windows, open +PowerShell). Use `cd` to navigate to the location where you have your +.pem file. Run `ls` to make sure you can see the ".pem" file. + +On some systems, you need to change the permissions on the .pem file +before you can use it. Try running `chmod 400 cs320.pem` (and don't +worry too much if it doesn't work). + +Now run this: +``` +ssh -i cs320.pem USERNAME@YOUR_IP_ADDRESS +``` + +<img src="img/7-ssh.png" width=600> + +This starts an *ssh session*. This means that any command you type +don't run on your computer. They run on your virtual machine. For +example, if you do a "pip3 install ..." here, you'll be installing a +python package to your virtual machine, not your laptop. + +## Jupyter + +You might already know `pip` (well, on Ubuntu you'll run `pip3` +instead) can install Python packages. `apt` is similar program on +Ubuntu that is more general purpose -- it's useful for all programs, +not just Python things. + +The first thing you should do is install the latest updates on your virtual machine, using `apt`: + +``` +sudo apt update +sudo apt upgrade +``` + +Putting `sudo` in front of a command means Ubuntu should "do" the +command as the "super user" (the admin). Only the admin can install +updates and install software. + +We'll now use `apt` to install `pip3`, and then we'll use `pip3` to +install Jupyter. + +``` +sudo apt install python3-pip +pip3 install jupyter +``` + +When you start Jupyter notebook remotely, you'll want to set a +password for connecting to it. Make it a good one, or anybody will be +able to take over your VM! (Whenever you need to enter something, like +a password, in the terminal, don't worry if nothing is appearing as +you're typing. Your keystrokes are still registering; the terminal +just isn't displaying them!) Run the following: + +``` +mkdir -p ~/.jupyter +python3 -m notebook password +``` + +Now let's start Jupyter. Run the following: + +`nohup python3 -m notebook --no-browser --ip=0.0.0.0 --port=2020` + +Putting "nohup" in front of a background means the program (in this +case `jupyter`) should keep running in the background, even if you +close the terminal. "nohup" stands for "no hangup". + +Now, open up a new browser window, and type `YOUR_IP_ADDRESS:2020` for +your VM's URL (IP should be the External IP of the virtual machine). +You can enter the same password that you set for Jupyter earlier. + +After you login, make sure the setup works (e.g., you can create a +notebook and run code). + +Good work on getting Jupyter running on your virtual machine! We +suggest you bookmark the login page so you can come back to it later. + +## Stopping your VM + +Once done, you may want to stop your VM to release resources and ensure you aren't billed. +You can do so via the dashboard: + +<img src="img/10-stop-vm.png" width=600> \ No newline at end of file diff --git a/Labs/vm/azure/img/1-activate.png b/Labs/vm/azure/img/1-activate.png new file mode 100644 index 0000000000000000000000000000000000000000..30baa599d77c9e485aef53baadff596b7c759e7e Binary files /dev/null and b/Labs/vm/azure/img/1-activate.png differ diff --git a/Labs/vm/azure/img/10-stop-vm.png b/Labs/vm/azure/img/10-stop-vm.png new file mode 100644 index 0000000000000000000000000000000000000000..9b1c3eeb2c1b2fd8406c963d30794e449f59670c Binary files /dev/null and b/Labs/vm/azure/img/10-stop-vm.png differ diff --git a/Labs/vm/azure/img/2-profile.png b/Labs/vm/azure/img/2-profile.png new file mode 100644 index 0000000000000000000000000000000000000000..1595ba107869e900afb4b8dd49f1aae08ec5c322 Binary files /dev/null and b/Labs/vm/azure/img/2-profile.png differ diff --git a/Labs/vm/azure/img/3-search-vms.png b/Labs/vm/azure/img/3-search-vms.png new file mode 100644 index 0000000000000000000000000000000000000000..abb3d36a45ed88e12a43d38f7f548a646a914b81 Binary files /dev/null and b/Labs/vm/azure/img/3-search-vms.png differ diff --git a/Labs/vm/azure/img/4-add-vm.png b/Labs/vm/azure/img/4-add-vm.png new file mode 100644 index 0000000000000000000000000000000000000000..a3a8f6daa73246cf1076a27930e5ac3d31e8e6f6 Binary files /dev/null and b/Labs/vm/azure/img/4-add-vm.png differ diff --git a/Labs/vm/azure/img/5-create-vm.png b/Labs/vm/azure/img/5-create-vm.png new file mode 100644 index 0000000000000000000000000000000000000000..572a0842374c89ff99a1ff109b610985cee54c99 Binary files /dev/null and b/Labs/vm/azure/img/5-create-vm.png differ diff --git a/Labs/vm/azure/img/6-view-vm.png b/Labs/vm/azure/img/6-view-vm.png new file mode 100644 index 0000000000000000000000000000000000000000..51c86d3d94e0bb6db62e53a4f513ea704c13a753 Binary files /dev/null and b/Labs/vm/azure/img/6-view-vm.png differ diff --git a/Labs/vm/azure/img/7-ssh.png b/Labs/vm/azure/img/7-ssh.png new file mode 100644 index 0000000000000000000000000000000000000000..aa4fe9dfb317de9930f89a9127f34bb718db37bb Binary files /dev/null and b/Labs/vm/azure/img/7-ssh.png differ diff --git a/Labs/vm/azure/img/8-networking.png b/Labs/vm/azure/img/8-networking.png new file mode 100644 index 0000000000000000000000000000000000000000..a42a34a224e229e77c413efbd8d8808915d599df Binary files /dev/null and b/Labs/vm/azure/img/8-networking.png differ diff --git a/Labs/vm/azure/img/9-add-ports.png b/Labs/vm/azure/img/9-add-ports.png new file mode 100644 index 0000000000000000000000000000000000000000..fc89ad12fde8e75575108b43dbc27a53ea1a2141 Binary files /dev/null and b/Labs/vm/azure/img/9-add-ports.png differ diff --git a/Labs/vm/gcp/README.md b/Labs/vm/gcp/README.md new file mode 100644 index 0000000000000000000000000000000000000000..17cb49d4d961627aac3763a5261298a9535e35e4 --- /dev/null +++ b/Labs/vm/gcp/README.md @@ -0,0 +1,50 @@ +# Google Option for Virtual Machine + +## Step 1: Create Account and Redeem Credits + +See details about how to get the $50 credit: https://canvas.wisc.edu/courses/374279/discussion_topics/1640217. + +Then follow [these instructions](gcp.md) to create an account and redeem the credit. + +## Step 2: Configuring the Firewall + +Follow [these instructions](firewall.md) to configure the firewall to unblock all +ports. + +## Step 3 [Recommended]: Creating an SSH Key + +You already have an idea of what a shell is: you've been either using +PowerShell (Windows) or bash (default on Mac) in CS 220/301. *ssh* +stands for "secure shell", and it's a special shell that lets you +remotely connect to other computers over the network and run commands +there. An *ssh key* is like a randomly generated password that ssh +automatically uses to access other machines (you won't generally type +it). + +Macs and recent Windows computers (Windows 10 with recent updates) +should have a program OpenSSH installed, which will be the most +convenient way to access your virtual machine. + +Try [these directions](ssh.md) to create an SSH key and connect it +with your cloud account. If it doesn't work (e.g., because you don't +have OpenSSH), don't worry too much -- there are workarounds later. + +## Step 4: Launching your Virtual Machine + +Now it's time to actually create your virtual machine! Follow [these steps](launch.md). + +## Step 5a: Connecting with SSH from your terminal + +If you were able to configure the SSH key properly in Step 3, follow +[these directions](connect.md). + +## Step 5b [Alternative]: Connecting without SSH + +Please only consider following [these +directions](connect-alt.md) if you couldn't configure the SSH key. Using this option may +cause problem in the long run. + +## Step 6: Setting up Jupyter + +Now lets get Jupyter and some other software installed. Follow [these +directions](jupyter.md). diff --git a/Labs/vm/gcp/connect-alt.md b/Labs/vm/gcp/connect-alt.md new file mode 100644 index 0000000000000000000000000000000000000000..11dd95f23bcd7d5ae3ea7077d426259b43d3b32d --- /dev/null +++ b/Labs/vm/gcp/connect-alt.md @@ -0,0 +1,15 @@ +# Connecting via Cloud Console + +In the table of VM instances, click the arrow by "SSH" in the row for your virtual machine, and click "Open in browser window". + +<img src="img/23.png" width=600> + +When the window finally opens, run `lsb_release -a`. If all is well, +it should show you what version of Ubuntu you're running (hopefully +22.04). It should look something like this: + +<img src="img/24.png" width=600> + +Congrats, you've created your first virtual machine with Linux +installed! This is no small feat, and being able to do so in the +future in an invaluable career skill. diff --git a/Labs/vm/gcp/connect.md b/Labs/vm/gcp/connect.md new file mode 100644 index 0000000000000000000000000000000000000000..ee6fc0e2c726774ac725e227de7ecbc3acfbf652 --- /dev/null +++ b/Labs/vm/gcp/connect.md @@ -0,0 +1,29 @@ +# Connecting with SSH from the Terminal + +In a terminal Window, run something like `ssh USER@IP`. You should +replace `USER` with the user name you got in the last step of the +directions for creating the SSH Key ([here](ssh.md)). You should +replace "IP" with the "External IP" you saw in the table of VM +instances after you launched your virtual machine. + +You might see something like "The authenticity of host +'104.154.180.165 (104.154.180.165)' can't be established. ECDSA key +fingerprint is SHA256:.... Are you sure you want to continue +connecting (yes/no/[fingerprint])?" You can type "yes". + +If the ssh connection works, run `lsb_release -a`. If all is well, it +should show you what version of Ubuntu you're running (hopefully +22.04). It should look something like this: + +<img src="img/25.png" width=600> + +Congrats, you've created your first virtual machine with Linux +installed! This is no small feat, and being able to do so in the +future in an invaluable career skill. + +**Note 1**: you can type "exit" or `CTRL-D` to exit the SSH connection and +run commands on your own computer again. + +**Note 2**: if you're connecting to SSH from Windows, CTRL-C and + CTRL-V don't work as copy/paste in the terminal. Right clicking can + do both copy or paste (if something has already been copied). diff --git a/Labs/vm/gcp/firewall.md b/Labs/vm/gcp/firewall.md new file mode 100644 index 0000000000000000000000000000000000000000..73f4df226028061d1aa14777daf130af827058fc --- /dev/null +++ b/Labs/vm/gcp/firewall.md @@ -0,0 +1,48 @@ +# Firewall Setup + +What is a firewall? First, a little background. + +Remember from CS 220/301 that a computer (typically) has an *IP address* +(something like 123.234.210.001), and other computers knowing that IP +address can communicate with programs running on that computer. + +If there are multiple running programs, they are differentiated by +*port numbers*. For example, sending a message to 123.234.210.001:220 +sends a message to the program on port 220 running on a computer with +address 123.234.210.001 (note the `:` between the IP address and port +number). Sending a message to 123.234.210.001:320 might send a +message to a different program on that same machine. Analogy: an IP +address is like the address of an apartment building (the computer), +and a port number is like the apartment number of a specific unit; +programs are the residents living in the units. + +A Firewall (among other things) can block access to certain ports for +outsiders, to provide security. We will unblock all ports for our +virtual machine. This is not the most secure option, but it is +convenient, and fine for the purposes of this course. + +1. Go to the Google Cloud Console at https://console.cloud.google.com/. Then sign into your account. Be sure your @wisc.edu user is selected (top right of blue bar) and that the "cs320-fa23" project is also selected (to the left of the search box in the blue bar)! + +<img src="img/7.png" width=600> + +2. In the menu on the left, open the "VPC network" menu and select "Firewall". + +<img src="img/Firewall_part2_update.png" width=300> + +3. After clicking "Firewall", you might recieve the "Data could not be loaded" error, just click "cotinue". Then enable the "Compute Engine API". This may take a couple minutes, and you probably need to re-click "Firewall" on the left after it finishes. + +4. Click the "CREATE FIREWALL RULE" button + +<img src="img/Firewall_part3_update.png" width=600> + +5. For the name and description, enter "cs320" + +<img src="img/10.png" width=450> + +6. Under Targets, chose "All instances in the network". Set "Source filter" to "IPv4 ranges" and "Source IP ranges" to "0.0.0.0/0". Choose "Allow all" under "Protocols and ports". Click "Create". + +<img src="img/11.png" width=450> + +7. You should see the new "cs320" rule in the table: + +<img src="img/12.png" width=600> diff --git a/Labs/vm/gcp/gcp.md b/Labs/vm/gcp/gcp.md new file mode 100644 index 0000000000000000000000000000000000000000..78ee78d3fd1591af6dee3d000a4d23cbc4e07c91 --- /dev/null +++ b/Labs/vm/gcp/gcp.md @@ -0,0 +1,42 @@ +# Google Cloud Signup + +## Account and Billing + +1. Go to https://cloud.google.com. Click "Get started". (If prompted to type in any Credit Card information just press skip.) + +2. Use your "@wisc.edu" email address + +3. Continue through any remaining steps to create an account. Sometimes you'll be prompted to enter a credit card -- if that happens, you should be able to move on to the next steps where you redeem the coupon code (it seems enough of the account is created prior to the credit card prompt for you to redeem the coupon and proceed). + +4. Look for the Canvas announcement about how to get a free $50 coupon code + +5. Go to https://console.cloud.google.com/education + +6. Double check that you're signed in as your "@wisc.edu" user (if you use Gmail, it may automatically default to your "@gmail.com" account). Check this in general whenever using the Google Cloud Console: + +<img src="img/1.png" width=600> + +7. Go to "Billing": https://console.cloud.google.com/billing + +8. Look for "Billing Account for Education" (or similar) and click it + +9. Verify that you have $50 remaining + +## Project + +Any **resources** we create (like virtual machines), are grouped into +**projects**. Each project is associated with a billing account. + +1. Go to the "Manage Resources" page in the console. It is under the drop-down menu in the top-left: "IAM & Admin" > "Manage Resources". Or, here is a link for quick access: https://console.cloud.google.com/cloud-resource-manager + +2. Click "CREATE PROJECT" + +3. Call it "cs320-fa23" and associate it with your account that has the free credits. Sometimes an option will appear to select an organization in which to nest your project. If this happens, select "wisc.edu". + +<img src="img/2.png" width=600> + +4. After it is created, click "SELECT PROJECT" + +5. In general, when working in the console, make sure your project is selected when given the option + +<img src="img/3.png" width=500> diff --git a/Labs/vm/gcp/img/1.png b/Labs/vm/gcp/img/1.png new file mode 100644 index 0000000000000000000000000000000000000000..401e7d77b74c6adb559c348879f213c579f5080a Binary files /dev/null and b/Labs/vm/gcp/img/1.png differ diff --git a/Labs/vm/gcp/img/10.png b/Labs/vm/gcp/img/10.png new file mode 100644 index 0000000000000000000000000000000000000000..5b77653a2f5500d31b57ec2e2132bcfb9bbc4661 Binary files /dev/null and b/Labs/vm/gcp/img/10.png differ diff --git a/Labs/vm/gcp/img/11.png b/Labs/vm/gcp/img/11.png new file mode 100644 index 0000000000000000000000000000000000000000..ee00d35227d8a9b9bfe5c2796f6a5579970a8ae4 Binary files /dev/null and b/Labs/vm/gcp/img/11.png differ diff --git a/Labs/vm/gcp/img/12.png b/Labs/vm/gcp/img/12.png new file mode 100644 index 0000000000000000000000000000000000000000..c23770d52d10da5e93c6322aaecb4bdc7a10b228 Binary files /dev/null and b/Labs/vm/gcp/img/12.png differ diff --git a/Labs/vm/gcp/img/13.png b/Labs/vm/gcp/img/13.png new file mode 100644 index 0000000000000000000000000000000000000000..3a43a65c912355b2340e98366b620d08b95b34ad Binary files /dev/null and b/Labs/vm/gcp/img/13.png differ diff --git a/Labs/vm/gcp/img/14.png b/Labs/vm/gcp/img/14.png new file mode 100644 index 0000000000000000000000000000000000000000..e4f6324d0adbe4fe655aa252893be4f4df16898f Binary files /dev/null and b/Labs/vm/gcp/img/14.png differ diff --git a/Labs/vm/gcp/img/15.png b/Labs/vm/gcp/img/15.png new file mode 100644 index 0000000000000000000000000000000000000000..7cdaa6bd10a186dad6a7fc288f20db5ff77c7d06 Binary files /dev/null and b/Labs/vm/gcp/img/15.png differ diff --git a/Labs/vm/gcp/img/16.png b/Labs/vm/gcp/img/16.png new file mode 100644 index 0000000000000000000000000000000000000000..722ad9bce2ef2949be0b9fd947fa91f625590912 Binary files /dev/null and b/Labs/vm/gcp/img/16.png differ diff --git a/Labs/vm/gcp/img/17.png b/Labs/vm/gcp/img/17.png new file mode 100644 index 0000000000000000000000000000000000000000..d34d338b06085b5d2108a87d2597b475badbd793 Binary files /dev/null and b/Labs/vm/gcp/img/17.png differ diff --git a/Labs/vm/gcp/img/18.png b/Labs/vm/gcp/img/18.png new file mode 100644 index 0000000000000000000000000000000000000000..d072236e72d1a4a4a1aad2466b040287d715ba9e Binary files /dev/null and b/Labs/vm/gcp/img/18.png differ diff --git a/Labs/vm/gcp/img/19.png b/Labs/vm/gcp/img/19.png new file mode 100644 index 0000000000000000000000000000000000000000..e8f3bdec910bbb2fa7ae4d68eea01280c74b81f1 Binary files /dev/null and b/Labs/vm/gcp/img/19.png differ diff --git a/Labs/vm/gcp/img/2.png b/Labs/vm/gcp/img/2.png new file mode 100644 index 0000000000000000000000000000000000000000..bf5efb91d9550c622e6b799157aa9797ccf70315 Binary files /dev/null and b/Labs/vm/gcp/img/2.png differ diff --git a/Labs/vm/gcp/img/20.png b/Labs/vm/gcp/img/20.png new file mode 100644 index 0000000000000000000000000000000000000000..de794c69b70792b7c4aab6460bec42323c4494be Binary files /dev/null and b/Labs/vm/gcp/img/20.png differ diff --git a/Labs/vm/gcp/img/21.png b/Labs/vm/gcp/img/21.png new file mode 100644 index 0000000000000000000000000000000000000000..77ba3a1d96ed56711c1c865ea5104f32b5ce84fe Binary files /dev/null and b/Labs/vm/gcp/img/21.png differ diff --git a/Labs/vm/gcp/img/22.png b/Labs/vm/gcp/img/22.png new file mode 100644 index 0000000000000000000000000000000000000000..d6e4bcac2ce19c6704c7fc0d1a6a6a9274917521 Binary files /dev/null and b/Labs/vm/gcp/img/22.png differ diff --git a/Labs/vm/gcp/img/23.png b/Labs/vm/gcp/img/23.png new file mode 100644 index 0000000000000000000000000000000000000000..0e3f6e8e5299d85a1c0e10f3783728b21606a1ac Binary files /dev/null and b/Labs/vm/gcp/img/23.png differ diff --git a/Labs/vm/gcp/img/24.png b/Labs/vm/gcp/img/24.png new file mode 100644 index 0000000000000000000000000000000000000000..eaaa490ac0a96e379fbfbb82be985e6d145bcbb5 Binary files /dev/null and b/Labs/vm/gcp/img/24.png differ diff --git a/Labs/vm/gcp/img/25.png b/Labs/vm/gcp/img/25.png new file mode 100644 index 0000000000000000000000000000000000000000..629dbdd44a69ec630ba130338f0df8891c0eadfd Binary files /dev/null and b/Labs/vm/gcp/img/25.png differ diff --git a/Labs/vm/gcp/img/26.png b/Labs/vm/gcp/img/26.png new file mode 100644 index 0000000000000000000000000000000000000000..40bc432697665b137e829299c43c34f994873afc Binary files /dev/null and b/Labs/vm/gcp/img/26.png differ diff --git a/Labs/vm/gcp/img/27.png b/Labs/vm/gcp/img/27.png new file mode 100644 index 0000000000000000000000000000000000000000..7406eb8ca2a8112877f869f9347016ea2998074c Binary files /dev/null and b/Labs/vm/gcp/img/27.png differ diff --git a/Labs/vm/gcp/img/28.png b/Labs/vm/gcp/img/28.png new file mode 100644 index 0000000000000000000000000000000000000000..deb47c42df963a8c55731d5a26da737ba6dd887f Binary files /dev/null and b/Labs/vm/gcp/img/28.png differ diff --git a/Labs/vm/gcp/img/29.png b/Labs/vm/gcp/img/29.png new file mode 100644 index 0000000000000000000000000000000000000000..b2ac146cb2a30e97b80057313bd0d0374809231b Binary files /dev/null and b/Labs/vm/gcp/img/29.png differ diff --git a/Labs/vm/gcp/img/3.png b/Labs/vm/gcp/img/3.png new file mode 100644 index 0000000000000000000000000000000000000000..f72756798af9ff0de0b5ed5fcffc23af438305ae Binary files /dev/null and b/Labs/vm/gcp/img/3.png differ diff --git a/Labs/vm/gcp/img/30.png b/Labs/vm/gcp/img/30.png new file mode 100644 index 0000000000000000000000000000000000000000..8c4f7e82528e1b9ed30b512aff415c0c4e6efef9 Binary files /dev/null and b/Labs/vm/gcp/img/30.png differ diff --git a/Labs/vm/gcp/img/31.png b/Labs/vm/gcp/img/31.png new file mode 100644 index 0000000000000000000000000000000000000000..478d9b31af962392dacc46c2b825dc0ffac88937 Binary files /dev/null and b/Labs/vm/gcp/img/31.png differ diff --git a/Labs/vm/gcp/img/7.png b/Labs/vm/gcp/img/7.png new file mode 100644 index 0000000000000000000000000000000000000000..811876e917f51ca1c914685259c64d0c995deafa Binary files /dev/null and b/Labs/vm/gcp/img/7.png differ diff --git a/Labs/vm/gcp/img/8.png b/Labs/vm/gcp/img/8.png new file mode 100644 index 0000000000000000000000000000000000000000..5c45dacaeece619b865fb8c12d87e7af447f8ad6 Binary files /dev/null and b/Labs/vm/gcp/img/8.png differ diff --git a/Labs/vm/gcp/img/9.png b/Labs/vm/gcp/img/9.png new file mode 100644 index 0000000000000000000000000000000000000000..646a7eeebbee2e1803ae3a9fcffca304b1b2253a Binary files /dev/null and b/Labs/vm/gcp/img/9.png differ diff --git a/Labs/vm/gcp/img/Firewall_part2_update.png b/Labs/vm/gcp/img/Firewall_part2_update.png new file mode 100644 index 0000000000000000000000000000000000000000..ad26c2744b8eb9a28f6152f105827466d23c0b46 Binary files /dev/null and b/Labs/vm/gcp/img/Firewall_part2_update.png differ diff --git a/Labs/vm/gcp/img/Firewall_part3_update.png b/Labs/vm/gcp/img/Firewall_part3_update.png new file mode 100644 index 0000000000000000000000000000000000000000..0b381bf43293cd32fc40159bb2d947d14f0a4956 Binary files /dev/null and b/Labs/vm/gcp/img/Firewall_part3_update.png differ diff --git a/Labs/vm/gcp/jupyter.md b/Labs/vm/gcp/jupyter.md new file mode 100644 index 0000000000000000000000000000000000000000..8c4677e577005a726fda239703962d6a033f2187 --- /dev/null +++ b/Labs/vm/gcp/jupyter.md @@ -0,0 +1,82 @@ +# Jupyter + +1. Connect via SSH (or the Google Cloud Console) to your virtual machine. + +2. Before we install Jupyter, let's get pip. Run the following: + +Please run the commands one at a time so if an error occurs, you are able to catch it. + +- `sudo apt update` + +- `sudo apt upgrade` + +- `sudo apt install python3-pip` + +If prompted, enter "Y" (for yes) when prompted. If you're using an +international keyboard, be careful about what mode you're typing in -- +we've heard that sometimes a character that looks like a "Y" to a +human isn't recognized by the installer. If prompted about services (as below), just hit the ENTER key to accept the recommendation: + +<img src="img/28.png" width=400> + +The `apt` program lets you install software on an Ubuntu system; think +of it like `pip`, for more general (you can install stuff not related +to Python). Putting `sudo` in front of the command means "do this as +a super user". You're signed in as a regular user, without permission +to install software by default, so you'll use `sudo` often for +installing tools and other tasks. + +You might be prompted with a "Restarting services" menu after running `sudo apt upgrade`. If so, enter "9". + +<img src="img/31.png" width=700> + +3. Now let's use pip3 to install Jupyter (don't use sudo for this one): + +``` +pip3 install jupyterlab==3.4.5 MarkupSafe==2.0.1 +``` + +4. When you start Jupyter notebook remotely, you'll want to set a +password for connecting to it. Make it a good one, or anybody will be +able to take over your VM! (Whenever you need to enter something, like a password, +in the terminal, don't worry if nothing is appearing as you're typing. Your keystrokes +are still registering; the terminal just isn't displaying them!) +Run the following: + +``` +python3 -m jupyterlab password +``` + +**Important!** Choose a strong password. Anybody on the Internet can + guess your password an unlimited number of times. Most semesters at + least one student loses their VM to malicious actors. + +5. Now let's start Jupyter. Run the following: + +``` +nohup python3 -m jupyterlab --no-browser --ip=0.0.0.0 --port=2020 & +``` + +You can now close the terminal window. + +6. Now, open up a new browser window, and type `IP:2020` for the URL +(IP should be the External IP of the virtual machine). You can enter +the same password that you set in step 4: + +<img src="img/26.png" width=600> + +7. After you login, make sure the setup works (e.g., you can create a +notebook and run code). + +Good work on getting Jupyter running on your virtual machine! We +suggest you bookmark the login page so you can come back to it later. + +# git + +Install git by running this command: `sudo apt install git`. You should now be able to +try out the git commands that were covered as part of "Reproducibility 3" lecture. + +# nano + +Install nano by running this command: `sudo apt install nano`. You can install other command +line text editors like `vim` or `emacs` if you would like to learn and use those. diff --git a/Labs/vm/gcp/launch.md b/Labs/vm/gcp/launch.md new file mode 100644 index 0000000000000000000000000000000000000000..66463cee2e8146c621681c8ef5db99ee863612be --- /dev/null +++ b/Labs/vm/gcp/launch.md @@ -0,0 +1,41 @@ +# Creating a Virtual Machine + +1. in the menu on the left, expand the "Compute Engine" menu under +"COMPUTE" and click "VM Instances". You should see the following, and you should click "CREATE INSTANCE". + +<img src="img/18.png" width=600> + +2. under "Machine type" choose "e2-small". Billing should be about +$13-14 a month, which will comes close to your $50 credit over the +course of the semester. + +<img src="img/19.png" width=600> + +3. Scroll down to "Boot disk" and click "Change" + +4. Select "Ubuntu" for the operating system. Ubuntu is a popular +version of Linux (choosing a popular one is good because it's easier +to search for useful online suggestions when things go amiss). Choose +"Ubuntu 22.04 LTS" for the version (there are two options that say +this -- choose the one that says "x86/64" as a subtitle). "LTS" +stands for "Long Term Support". LTS versions aren't generally the +most recent version, but they're generally more reliable. Update your +boot disk to use 15 GB. + +<img src="img/21.png" width=600> + +6. Further down, click to allow HTTP and HTTPS traffic, then click +"Create". If you get an error (e.g., `e2 instances do not support +onHostMaintenance=TERMINATE unless they are preemptible`, deleting the +VM and retrying the creation sometimes works the second time) + +7. You should see one instance here. After the semester, or if you +ever want to start fresh with a new virtual machine, you may want to +delete the old virtual machines on this page. Remember that you pay +per virtual machine (e.g., about $14/month for each small instance), +so it's a waste of money (or hopefully credits) to have more running +than necessary. + +8. Make a note of the "External IP" -- you'll be using it again. + +<img src="img/29.png" width=600> \ No newline at end of file diff --git a/Labs/vm/gcp/ssh.md b/Labs/vm/gcp/ssh.md new file mode 100644 index 0000000000000000000000000000000000000000..e93c78e057df6830e5ace1fe00b1571c3b0a5a85 --- /dev/null +++ b/Labs/vm/gcp/ssh.md @@ -0,0 +1,46 @@ +# SSH + +1. In the menu on the left, open the "Compute Engine" menu, and find "Metadata" under "Settings". + +<img src="img/13.png" width=300> + +2. Go to "SSH Keys" and click "Add SSH keys". There will be a box +where we need to paste a key, which we haven't generated yet. + +<img src="img/14.png" width=600> + +3. Open your terminal(MacOS) or PowerShell(Windows), and run the +command `ssh-keygen`. You should be able to just use the defaults by +hitting enter a few times (I don't recommend setting a password for +this or a passphrase). It looks like the following on a Mac, but +should work the same on Windows. If `ssh-keygen` isn't found, then +you should skip the rest of this section and go back to the next steps +on the [main page](README.md) for now (feel free to get help more +specific to your setup during office hours). + +Note that the `ssh-keygen` creates a "private key" on the computer +where you run it that will let you connect to your VM without a +password via SSH this semester. If you switch computers at some +point, you'll need to revisit this part of the lab. + +Sometimes `ssh-keygen` creates SSH keys which usernames only consist of numbers. +This is an invalid SSH key for GCP. You can generate a new SSH key specifiying a +username by `ssh-keygen -C [USERNAME]` + +<img src="img/15.png" width=600> + +4. run `cat ~/.ssh/id_rsa.pub`. Then copy the output (the part +highlighted in the following screenshot). (If your using windows, run the command using windows powershell) + +<img src="img/16.png" width=600> + +5. go back to where you were adding an SSH key in the Google cloud +console and paste your key (as copied in step 4). Then click "Save". + +<img src="img/17.png" width=600> + +6. Make a note of the +username that appears to the left of the box (in this case "msyamkumar") -- +you'll need it later. + +<img src="img/30.png" width=700>