Skip to content
GitLab
Explore
Sign in
Primary navigation
Search or go to…
Project
M
main
Manage
Activity
Members
Labels
Plan
Issues
Issue boards
Milestones
Wiki
Requirements
Code
Merge requests
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Snippets
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Package Registry
Container Registry
Model registry
Operate
Environments
Terraform modules
Monitor
Incidents
Service Desk
Analyze
Value stream analytics
Contributor analytics
CI/CD analytics
Repository analytics
Code review analytics
Issue analytics
Insights
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
JIHONG MIN
main
Commits
0fede829
Commit
0fede829
authored
2 weeks ago
by
TYLER CARAZA-HARTER
Browse files
Options
Downloads
Patches
Plain Diff
update README
parent
7ecabf8a
No related branches found
Branches containing commit
No related tags found
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
p3/README.md
+37
-12
37 additions, 12 deletions
p3/README.md
with
37 additions
and
12 deletions
p3/README.md
+
37
−
12
View file @
0fede829
...
...
@@ -24,7 +24,7 @@ Before starting, please review the [general project directions](../projects.md).
*
none yet
## Part 1: Communication
## Part 1: Communication
(gRPC)
In this project, three client programs (upload.py, csvsum.py, and
parquetsum.py) will communicate with a server, server.py, via gRPC.
...
...
@@ -46,19 +46,22 @@ servicer code for your server. All field types will be strings,
except
`total`
and
`csv_data`
, which should be
`int64`
and
`bytes`
respectively.
### Local Testing (Recommended)
Now build the .proto on your VM. Install the tools like this:
```
sh
python3
-m
venv venv
source
venv/bin/activate
pip
3
install
grpcio
==
1.66.1 grpcio-tools
==
1.66.1
numpy
==
2.1.1
protobuf
==
5.27.2
pyarrow
==
17.0.0
setuptools
==
75.1.0
pip
install
grpcio
-tools
==
1.70.0
grpcio
==
1.70.0
protobuf
==
5.29.3
```
Then use
`grpc_tools.protoc`
to build your .proto file.
In your server, override the two RPC methods for the generated
servicer. For now, you can just return messages with the error field
set to "TODO", leaving any other field unspecified.
Create a server.py program that uses the generated gRPC code. Your
server should override the two RPC methods for the servicer.
For now, you can just return messages with the error field set to
"TODO", leaving any other field unspecified.
If communication is working correctly so far, you should be able to
start a server and used a client to get back a "TODO" error message
...
...
@@ -70,7 +73,9 @@ python3 upload.py simple.csv
# should see "TODO"
```
Create a Dockerfile to build an image that will also let you run your
### Docker Deployment (Required)
Create a Dockerfile to build an image that will let you run your
server in a container. It should be possible to build and run your
server like this:
...
...
@@ -81,12 +86,32 @@ docker run -d -m 512m -p 127.0.0.1:5440:5440 p3
The client programs should then be able to communicate with the
Dockerized programs the same way they communicated with the server
outside of a container.
outside of a container (in the "Local Testing" example above).
Note that we won't install any Python packages (such as the gRPC
tools) on our test VM (like we asked you to do for manual testing), so
it is important that compiling your .proto file is one of the steps
that happens during Docker build.
Your Dockerfile should also copy in the client programs we provide
into your Docker image, and your image should include whatever these
clients need to run. When we test your code, we will run the clients
in the same container as the server, like this:
```
docker run --name=server -d -m 512m -p 127.0.0.1:5440:5440 -v ./inputs:/inputs p3 # server
docker exec server python3 upload.py /inputs/test1.csv # client
```
Note that we are using a volume mount to share make input data visible
inside your container so that we can
`exec`
clients inside your
container with that data.
If you want to make code changes without rebuilding the image each
time, consider using a volume mount to make the latest version of
server.py on your VM replace the server.py in the file (if the server
is not at /server.py inside the container, modify accordingly):
You may also wish to use volume mounts to speed up your development
process. For example, if you volume mount your server code (on your
VM) over your server.py file inside your container, you will be able
to re-run your container with newer server.py code without rebuilding
first. Here's an example:
```
docker run --rm -m 512m -p 127.0.0.1:5440:5440 -v ./server.py:/server.py p3
...
...
@@ -190,4 +215,4 @@ your code must NOT hold the lock when doing file I/O.
## Tester
Not released yet. Details coming soon!
\ No newline at end of file
Not released yet. Details coming soon!
This diff is collapsed.
Click to expand it.
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment