From 0fede8297122be1a6794e9ad1cf5cef4ba383441 Mon Sep 17 00:00:00 2001 From: TYLER CARAZA-HARTER <tharter@cs544-tharter.cs.wisc.edu> Date: Fri, 21 Feb 2025 21:32:40 -0600 Subject: [PATCH] update README --- p3/README.md | 49 +++++++++++++++++++++++++++++++++++++------------ 1 file changed, 37 insertions(+), 12 deletions(-) diff --git a/p3/README.md b/p3/README.md index 14b1757..245f670 100644 --- a/p3/README.md +++ b/p3/README.md @@ -24,7 +24,7 @@ Before starting, please review the [general project directions](../projects.md). * none yet -## Part 1: Communication +## Part 1: Communication (gRPC) In this project, three client programs (upload.py, csvsum.py, and parquetsum.py) will communicate with a server, server.py, via gRPC. @@ -46,19 +46,22 @@ servicer code for your server. All field types will be strings, except `total` and `csv_data`, which should be `int64` and `bytes` respectively. +### Local Testing (Recommended) + Now build the .proto on your VM. Install the tools like this: ```sh python3 -m venv venv source venv/bin/activate -pip3 install grpcio==1.66.1 grpcio-tools==1.66.1 numpy==2.1.1 protobuf==5.27.2 pyarrow==17.0.0 setuptools==75.1.0 +pip install grpcio-tools==1.70.0 grpcio==1.70.0 protobuf==5.29.3 ``` Then use `grpc_tools.protoc` to build your .proto file. -In your server, override the two RPC methods for the generated -servicer. For now, you can just return messages with the error field -set to "TODO", leaving any other field unspecified. +Create a server.py program that uses the generated gRPC code. Your +server should override the two RPC methods for the servicer. +For now, you can just return messages with the error field set to +"TODO", leaving any other field unspecified. If communication is working correctly so far, you should be able to start a server and used a client to get back a "TODO" error message @@ -70,7 +73,9 @@ python3 upload.py simple.csv # should see "TODO" ``` -Create a Dockerfile to build an image that will also let you run your +### Docker Deployment (Required) + +Create a Dockerfile to build an image that will let you run your server in a container. It should be possible to build and run your server like this: @@ -81,12 +86,32 @@ docker run -d -m 512m -p 127.0.0.1:5440:5440 p3 The client programs should then be able to communicate with the Dockerized programs the same way they communicated with the server -outside of a container. +outside of a container (in the "Local Testing" example above). + +Note that we won't install any Python packages (such as the gRPC +tools) on our test VM (like we asked you to do for manual testing), so +it is important that compiling your .proto file is one of the steps +that happens during Docker build. + +Your Dockerfile should also copy in the client programs we provide +into your Docker image, and your image should include whatever these +clients need to run. When we test your code, we will run the clients +in the same container as the server, like this: + +``` +docker run --name=server -d -m 512m -p 127.0.0.1:5440:5440 -v ./inputs:/inputs p3 # server +docker exec server python3 upload.py /inputs/test1.csv # client +``` + +Note that we are using a volume mount to share make input data visible +inside your container so that we can `exec` clients inside your +container with that data. -If you want to make code changes without rebuilding the image each -time, consider using a volume mount to make the latest version of -server.py on your VM replace the server.py in the file (if the server -is not at /server.py inside the container, modify accordingly): +You may also wish to use volume mounts to speed up your development +process. For example, if you volume mount your server code (on your +VM) over your server.py file inside your container, you will be able +to re-run your container with newer server.py code without rebuilding +first. Here's an example: ``` docker run --rm -m 512m -p 127.0.0.1:5440:5440 -v ./server.py:/server.py p3 @@ -190,4 +215,4 @@ your code must NOT hold the lock when doing file I/O. ## Tester -Not released yet. Details coming soon! \ No newline at end of file +Not released yet. Details coming soon! -- GitLab