Merge branch 'training-supplementation' into 'master'

Supplements Training Documentation with Additional Information

See merge request !44

README.md

@@ -44,7 +44,7 @@ Use [template.md](./template.md) to create new content.
 * [Logging in](./docs/logging-in.md)
 * [Tutorials](./docs/tutorials.md)
 * [How To Guides](./docs/howto.md)
-* [Training](./docs/training.md)
+* [Training](./docs/training/training.md)
 * [Best Practices And Recommendations](./docs/best-practices.md)
 * [Shared Orgs and Sub Orgs](./docs/shared-org-vs-sub-org.md)
-* [Get Help](./get-help.md)
\ No newline at end of file
+* [Get Help](./get-help.md)

docs/best-practices/secure-agent.md

@@ -17,7 +17,7 @@ The Secure Agent program is upgraded automatically by Informatica, but you are r
 
 The DoIT Integration Platform team has experience managing secure agents.
 We run our secure agent in a Docker container on Linux, hosted by Amazon Web Services (AWS).
-[The Secure Agent Docker image is available here](https://git.doit.wisc.edu/interop/iics/iics_secure_agent) if you want to build and try it yourself.
+[Instructions on setting up and running a Secure Agent in a similar manner are available here](../training/docker-secure-agent.md) if you want to build and try it yourself.
 We use AWS Elastic Container Service (ECS) to manage the secure agent deployment environment.
 By allowing ECS to run the Secure Agent container, we can make sure that the Secure Agent is always running because ECS would bring up a new instance if the current instance crashed.
 

docs/on-boarding-to-iics.md

@@ -18,5 +18,5 @@ Access is controlled through Manifest.
 See [this document](./howto.md#how-to-create-a-manifest-group-for-your-team-for-access-control) for more information.
 We will also work on setting up any additional connectors, sub-orgs, or secure agent licenses, if applicable to your use case.
 
-After getting access to IICS, you can start [training](./training.md), follow [tutorials](./tutorials.md), and start implementing your integration(s).
+After getting access to IICS, you can start [training](./training/training.md), follow [tutorials](./tutorials.md), and start implementing your integration(s).
 Informatica also provides documentation to get started (e.g. [getting started guide for Cloud Application Integration (CAI)](https://knowledge.informatica.com/s/article/DOC-17653?language=en_US)).

docs/training/docker-secure-agent.md

+# Basic Requirements
+
+To set up a secure agent on Linux, running in a Docker container, you will need several things:
+
+- Access to a workstation with an Intel-based processor. This is needed to run the Secure Agent. This could be a work computer, or, in the case of someone using a Macbook with an M1 processor, this could be an Amazon WorkSpaces desktop.
+- Git. This is needed to pull the necessary Secure Agent files. [Official instructions on installation can be found here](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git).
+- Docker. This is used to set up the container in which the Secure Agent will run.
+- [Docker Compose](https://docs.docker.com/compose/). This is an additional command used to build the image which the Secure Agent runs. I[nstructions to install this and Docker on an AWS Workspace can be found here](https://www.cyberciti.biz/faq/how-to-install-docker-on-amazon-linux-2/)
+- Your own IICS trial account. If you are already using the shared test or prod organizations, this tutorial is unnecessary. Use ei.secureagent.doit.wisc.edu instead. If you are using a different organization with a pre-existing Secure Agent, follow the instructions given to you when your access was provisioned.
+
+# Setup
+
+0. Open up the command line, in whatever directory you would like to be the parent directory of your Secure Agent folder.
+1. Clone the Secure Agent repository from GitLab. Change the current working directory of your command line to the newly cloned folder. This will create a new folder in whatever directory you are working in from the command line:
+```
+    git clone https://git.doit.wisc.edu/interop/iics/iics_secure_agent
+    cd iics_secure_agent
+```
+2. Switch to the ```production-override``` branch of the repository, and confirm you are on this branch:
+```
+    git checkout production-override
+    git status
+```
+3. Copy the Environment Variable file, and the infaagent file:
+```
+    cp .env-example .env
+    cp conf/infaagent-example.ini conf/infaagent.ini
+```
+- After doing this, open the Enviornment Variable file (.env), and add these lines:
+```
+    INFORMATICA_USER=[Your Informatica username]
+    INFORMATICA_PASSWORD=[Your Informatica trial account password]
+    HOSTNAME=your-hostname
+    POD=usw5
+    REGION=dm-us
+```
+- The user name and password are the username and password of the IICS trial account you are doing the training on.
+- The hostname is the name of the secure agent. You can choose any name here, but note that if you are doing this on AWS WorkSpaces, it will be overridden in IICS.
+- The pod and region are determined by looking at the URL of your IICS account. For example, if the URL was https<area>://usw5.dm-us.informaticacloud.com/…, the pod would be usw5 and the region would be dm-us.
+4. Create shared volume and logs volume, for transferring files from the docker container to your workstation and vice-versa:
+```
+    mkdir -p volumes/log
+    mkdir -p volumes/shared
+    touch volumes/log/agentcore.log
+    touch conf/odbc.ini
+```
+- Afterwards, change ownership permissions to allow the secure agent to write to the volumes (note that these commands specifically must be entered one at a time):
+```
+    sudo chown -R 1000:1000 volumes/shared/
+    sudo chown -R 1000:1000 conf/*
+    sudo chown -R 1000:1000 volumes/log/
+```
+5. Build the container:
+```
+    docker-compose build
+```
+# Running the Secure Agent
+1. Run the secure agent, and find the Container ID:
+```
+    docker-compose up -d
+    docker ps
+```
+- After finding the Container ID, use it to follow the logs:
+```
+    docker logs --follow <docker name or number>
+```
+
+2. Check your IICS account, in the Runtime Enviornment tab of the Administrator service. It could take some time for the Secure Agent to enable all services. An audit log is available to check.
+3. To shut down the Secure Agent:
+```
+    docker stop <container ID>
+```
+- alternatively:
+```
+    docker kill <container ID>
+```
+# Accessing Secure Agent Filesystem
+Type in this command:
+```
+    docker exec -ti <container ID> bash
+```
+This allows you to view the files in the Docker container through a command-line interface. This can be useful for checking issues with the various volumes that are shared between the Docker container and the desktop.
+# Deleting Secure Agent for Fresh Install
+1. Purge all Docker containers and images:
+```
+    docker kill <container ID>
+    docker system prune -a  
+```
+- You will have to type 'y' after doing this to have the system prune go through.
+2. Delete the iics_secure_agent folder.
+3. Delete the Secure Agent on your IICS trial account. Note that you may first have to delete all connections and tasks using the group for this Secure Agent as a runtime environment.
+# Notes
+- These commands are designed to be run on a Linux machine, specifically Amazon Linux 2 on Amazon WorkSpaces desktops. Other Linux environments or operating systems may produce different results.
+- As of August 2022, this is the only viable method to get a Secure Agent running on a Mac with an M1 chip. Stable virtualization and emulation options which support running a Secure Agent on macOS (e.g., Parallels, VMware Fusion) are not yet available for machines with Apple silicon processors.
+- The Docker container can also be run by typing 'docker-compose up', but this will have it take up the command line with log tracking. Exiting will shut down the secure agent if run in this method, whereas using the '-d' flag will not.
+- The Container ID of the Secure Agent will usually be either 'iics_secure_agent_iics-secure-agent-1' or 'iics_secure_agent_iics-secure-agent_1'.

docs/training/training-supplements.md

+IICS Training provides clear instructions on how to complete each component of the tutorials, but if you are unable to follow these instructions exactly as given, you may have trouble finding viable workarounds. Supplemental materials based on experiments and tests by various parties who have done IICS training__ are documented on this page.
+
+#### Secure Agent on Non-Windows Machines
+
+Assuming one is using their own trial organization while going through the tutorials, they will need to set up their own secure agent. All instructions given in IICS Training assume that you are running a secure agent from the desktop of your x86 or x64 windows machine. If one is using a different operating system this can prove troublesome, even more so if one is using a Macbook with an M1 card (or some other alternative chip format). One well-tested solution to this issue that will most likely work in any circumstance is to create a Docker Container inside of an AWS Workspace; [Instructions on this process can be found here](docker-secure-agent.md).
+
+Do note that when using a machine with an M1 chip, you will be unable to successfully run the secure agent outside of AWS, even inside of a virtual machine. This has been tested with a variety of methods and is most likely a product of the nature of the program, something that cannot be worked around.
+
+Assuming one is not using their own trial organization, but rather the shared test org, then they may simply use ei.secureagent.doit.wisc.edu instead. Do note that loading flat files into this secure agent may prove difficult due to its remote nature, and one might want to use an alternative for storing files (such as an S3 bucket).
+
+#### Databases Without Oracle
+
+Myriad labs in IICS request that you use an Oracle database as a source for data. Setting these up can be significantly time consuming. A PostgreSQL database will work fine for this purpose if the proper data is loaded in, however.
+
+The University hosts a test database for IICS purposes. [Information on this database can be found here](https://git.doit.wisc.edu/interop/iics/training-resources/-/tree/main). Several extant tables exist that are equivalents of what would be in an Oracle database. Do note that t you can use a database as a target in IICS, although this never comes up in training.
+
+If you are setting up the database yourself, or wish to create your own tables in the test database, do note that the given database commands are written for Oracle specifically: they will not work for Postgres, and will have to be converted to the other format.
+
+Do note that when using these instructions, Lab 4-1 in CAI may not work properly, which may necessitate skipping it.
+
+#### Understanding Expressions Beyond the Basics
+
+Most labs have you use an expression transformation, and provide you with a block of text to paste somewhere to transform data from one state to another. The labs fail to explain what this text means and how to make your own expressions, however. [A reference to the functions available for use in expressions can be found here](https://docs.informatica.com/integration-cloud/cloud-data-integration/current-version/function-reference/preface.html).
+
+For those with experience in using programming language, an alternative can be found in the [Java Transformation](https://docs.informatica.com/integration-cloud/cloud-data-integration/current-version/transformations/java-transformation.html). Usage of the Expression Transformation is reccomended over this for most use cases however, as it has less overhead and is easier to read.

docs/training.md → docs/training/training.md

@@ -50,6 +50,9 @@ Some areas of the Success Portal link out to [Informatica Network](https://netwo
 ## Secure Agents for Training
 
 During some trainings, there might be mentions of setting up your own secure agent or accessing a secure agent directly.
-For training purposes, the shared test organization can be used to create secure agents.
 
-Please make sure to destroy any secure agents used for training purposes when the training is completed.
+If one is using their own trial organization for training purposes, secure agents can be created using [These Instructions](./docker-secure-agent.md).
+
+## Supplemental Training Information
+
+For additional information regarding training, look at this [Supplemental Training Documentation](./training-supplements.md).

get-help.md

@@ -2,7 +2,7 @@
 
 If you weren't able to answer your question using the documentation in this repository, here are some options that are available to get help.
 
-- Informatica provides documentation on their website. If you're looking for IICS training, please see our [training documentation](./docs/training.md).
+- Informatica provides documentation on their website. If you're looking for IICS training, please see our [training documentation](./docs/training/training.md).
     - [Cloud Data Integration](https://onlinehelp.informatica.com/IICS/prod/CDI/en/index.htm)
     - [Cloud Application Integration ](https://onlinehelp.informatica.com/IICS/prod/CAI/en/index.htm)
 - Frequently asked questions are [documented in the Informatica KB](https://search.informatica.com/).

scripts/link-checker-config.json

 {
     "ignorePatterns": [
-        {
-            "pattern": "interop.it.wisc.edu"
-        }
+            "interop.it.wisc.edu",
+            "https://www.cyberciti.biz/faq/how-to-install-docker-on-amazon-linux-2/"
     ]
 }