Merge branch 'volume' into 'master'

Externalize Secure Agent configurations so that those can be persisted and re-use

See merge request interop/iics_secure_agent!6

Dockerfile

@@ -52,14 +52,6 @@ chmod +x /tmp/agent64_install.bin && \
 rm -rf /tmp/agent64_install.bin
 
 WORKDIR $WORK_DIR
-## Define Volumes for Shared Data Staging area
-VOLUME [ "/data" ]
-VOLUME [ "/exports" ]
-## Ports used by the agent that might be used for external Connections
-# 7080 Process Engine Shutdown Port
-# 7443 Process Engine https port
-# 5432 Process Engine Postgres DB
-EXPOSE 7080 7443 5432
 
 COPY run_agent.sh .
 

README.md

 # IICS Secure Agent Docker Image
 
 ## Description
-
 This project was forked from [jbrazda/ic-sagent-docker|https://github.com/jbrazda/ic-sagent-docker] with the intention of containerizing the IICS Secure Agent to run in Amazon Web Services.  As of early January 2019, there is no official Docker image for IICS Secure Agent.
 
 ## Requirements
-
-* Docker
+* Docker.
 * An IICS user account and password for your organization that has appropriate rights to create and manage secure agents.  This user will be used to login to the Informatica APIs to register the agent.
 * Your Informatica POD and REGION, which can be inferred from https://${POD}.${REGION}.informaticacloud.com/.  Defaults are set to POD=usw3 and REGION=dm-us, but can be overriden with Docker environment variables.
 
-## The image
-
+## The Image
 The image is based on Ubuntu 18.04.  It downloads and installs necessary dependencies, and then downloads the installer file from the correct Informatica URL based on your POD and REGION.
 
-1. Clone this repository
-2. `docker build . -t secure-agent`  Optionally pass --build-arg parameters for POD and REGION if the defaults are not correct.  POD and REGION are also set as environment variables, because containers will use these values for communicating with the Informatica API.
+1. Clone this repository.
+2. `docker build . -t iics_secure_agent:<tag>` - optionally pass --build-arg parameters for POD and REGION if the defaults are not correct.  POD and REGION are also set as environment variables, because containers will use these values for communicating with the Informatica API.
 
 ## Containers
-
 The container executes a bash script called run_agent.sh upon start.  It needs INFORMATICA_USER and INFORMATICA_PASSWORD environment variables set during runtime in order to communicate with the Informatica API to check and register the Secure Agent.
 
-### Environment variables
-
+### Environment Variables
 * INFORMATICA_USER (required) - User that can run the agent and access Informatica APIs
 * INFORMATICA_PASSWORD (required) - Password for above credential.  Used for API access only.
 * JSON_LOG (optional) - If this is set, then the log output will be in JSON format.
 
-### Volumes
+### Externalized Configurations
+Secure Agent's configurations can be externalized using Docker [volumes](https://docs.docker.com/storage/volumes/). Following 
+Secure Agent's directories and files can be externalized for containers. Note that after initial 
+container startup, these configurations can be used to start subsequent containers.  
+
+* `agentcore.log` - Secure Agent default runtime log file. 
+* `infaagent.log` - Secure Agent start up log file.
+* `logs` - contains Secure Agent runtime log files and lock files.
+* `data` - contains the configuration data for mapping and integration components.
+* `conf` - contains Secure Agent runtime configuration files.
+   1. `infaagent.ini` - contains the registration information for the secure agent(this will be populated by Secure Agent during start up).
+   2. `proxy.ini` - proxy configurations for Secure Agent, if running behind a [proxy](https://kb.informatica.com/howto/6/pages/20/512361.aspx) server.  
 
-* Log files - tomcat log files are in the WORKDIR/logs directory.  agentcore.log and infaagent.log are in the WORKDIR directory
-* Infaagent.ini - in WORKDIR/conf/infaagent.ini.  Contains the registration information for the secure agent
-* data - Not sure what this does right now and whether this needs to be mounted as a volume.
+Following ports(among others) in Secure Agents can be mapped to host for externalized control. 
+* `7080` - Process Engine shutdown port.
+* `7443` - Process Engine https port.
+* `5432` - Process Engine Postgres DB port.
+
+See below section for examples on how to use volume and port mapping.
 
 ### Starting
 
-Setting the hostname will provide the associated name in the IICS web site.
+* Setting the hostname will provide the associated name in the IICS website. 
+* Note: since anybody who has access to see the processes can view the values `INFORMATICA_USER` and `INFORMATICA_PASSWORD`, it's 
+recommenced to configure them in a Docker [.env](https://docs.docker.com/compose/env-file/) file.
 
-`docker run -d -h <hostname> --env INFORMATICA_USER=xxxxx --env INFORMATICA_PASSWORD=xxxx --name <container_name> <image_name:image_tag>`
+```shell
+$ touch agentcore.log infaagent.log 
+$ docker run -d \
+    -h <hostname> \
+    --env INFORMATICA_USER=xxxxx \
+    --env INFORMATICA_PASSWORD=xxxx \
+    -v $(pwd)/infaagent.log:/home/agent/infaagent/apps/agentcore/infaagent.log  \
+    -v $(pwd)/agentcore.log:/home/agent/infaagent/apps/agentcore/agentcore.log  \
+    -v $(pwd)/logs:/home/agent/infaagent/apps/agentcore/logs  \
+    -v $(pwd)/data:/home/agent/infaagent/apps/agentcore/data \
+    -p 7080:7080 \
+    -p 7443:7443 \
+    -p 5432:5432 \ 
+    --name <container_name> iics_secure_agent:<tag>
+```
+
+* Note that if `conf` directory needs mounting, a minimum configuration for starting the Secure Agent is required, see [conf](./conf) for example.
+```shell
+$ touch agentcore.log infaagent.log 
+$ docker run -d \
+    -h <hostname> \
+    --env INFORMATICA_USER=xxxxx \
+    --env INFORMATICA_PASSWORD=xxxx \
+    -v $(pwd)/infaagent.log:/home/agent/infaagent/apps/agentcore/infaagent.log  \
+    -v $(pwd)/agentcore.log:/home/agent/infaagent/apps/agentcore/agentcore.log  \
+    -v $(pwd)/logs:/home/agent/infaagent/apps/agentcore/logs  \
+    -v $(pwd)/data:/home/agent/infaagent/apps/agentcore/data \
+    -v $(pwd)/conf:/home/agent/infaagent/apps/agentcore/conf \
+    -p 7080:7080 \
+    -p 7443:7443 \
+    -p 5432:5432 \ 
+    --name <container_name> iics_secure_agent:<tag>
+```  
 
 ### Monitoring
+If volume mapping is not used use `docke exec` to attach to the running container.
 
 ```shell
-docker exec -it ic-agent1 less agentCore.log
+docker exec -it <container_name> less agentCore.log
 ```
 
-## Hardware requirements
-From https://network.informatica.com/servlet/JiveServlet/download/189174-25674/Informatica%20Cloud%20Secure%20Agent%20Minimum%20Requirements.pdf
-Minimum Hardware Specs
-* Use one of the following operating systems:
+## Hardware Requirements
+Minimum Hardware Specs ([reference](https://network.informatica.com/servlet/JiveServlet/download/189174-25674/Informatica%20Cloud%20Secure%20Agent%20Minimum%20Requirements.pdf)):
 * Windows version 7 (32-bit or 64-bit) or Windows XP Pro SP2/SP3 (32-bit).
-* Linux (32-bit or 64-bit) with the following configuration:
+* Linux (32-bit or 64-bit).
 * Red Hat Enterprise Linux version 5.3 or above.
 * SUSE Linux Enterprise version 10 and above.
-* 2-4 CPU recommended
-* 2 GB RAM recommended dedicated for the Agent (Recent customers have used 8GB)
-* 1 GB Hard disk space
-* 1 GB Network connectivity (faster the better as this will most likely the bottleneck)
+* 2-4 CPU recommended.
+* 2 GB RAM recommended dedicated for the Agent (Recent customers have used 8GB).
+* 1 GB Hard disk space.
+* 1 GB Network connectivity (faster the better as this will most likely the bottleneck).
 
 ## Known Issues
-
 * Doesn't seem to be able to run in host network mode because it won't be able to talk to internal ports.  It would probably work if you expose those ports.
 
+## Enterprise Integration Secure Agent
+Secure Agent containers are deployed in following tiers in interop/AWS account. This deployment is automated through Terraform (see below section on Terraform).
+
+| environment| tier        |
+| -----------|-------------|
+| test      | us-east-1(N. Virginia)|
+| prod      | us-east-2(Ohio)|
+
+
 ## Terraform
 * Infrastructure is available as code in terraform. 
 * Before disposing and create a new instance of Informatica Secure Agent, existing live connections to various targets(for e.g. databases) 

conf/agentcore.cfg

+# Number of seconds to refresh the rmi info file of the AgentCore's rmi server.
+rmi-info-refresh-period=300
+
+# Number of seconds that the Agent Core waits for the Agent Manager's notification that it is shuting down so the Agent Core can continue it's process (e.g. upgrade).
+resource-release-waiting-time=10
+
+# Number of seconds to check for new upgrade in the server.
+upgrade-check-period=600
+
+# Number of seconds to update connectivity status and other agent core status
+maintenance-check-period=10
+
+# Number of seconds for timeout of http connection with Informatica cloud
+http-connection-timeout=30
+
+# Number of seconds for timeout of http socket with Informatica cloud
+http-socket-timeout=30
+
+# Number of total seconds for retry when getting application specification from Informatica cloud
+appspec-retry-timeout=300
+
+# Number of seconds between each retry getting application specification from Informatica cloud
+appspec-retry-interval=5
+
+# Enable TLS certificate generation
+enable-application-certificate-generation=true

conf/agentcore.properties

+#AgentCore's properties
+#Tue Feb 04 16:37:13 GMT 2020
+rmi_bind_name=AgentCore
+pid=16
+rmi_port_number=20884
+rmi_bind_name_kms=AgentCoreKMS

conf/agentcoreversion

+56.08

conf/agentmgr.cfg

+start_as_windows_service=true
+#wait time in seconds for agentcore to start
+restart_agentcore_wait_time=10
+#wait time in seconds for Infa Agent Service to shutdown
+shutdown_infa_agent_service_wait_time=5
+#if language and locale are empty, then setting taken from system
+infa_agent_language=
+infa_agent_locale=

infaagent.ini.example → conf/infaagent.ini

 InfaAgent.UseToken=true
+InfaAgent.GroupName=aws
 #
 #Tue Jan 14 01:16:27 PST 2020
 InfaAgent.MasterUrl=https\://dm-us.informaticacloud.com
\ No newline at end of file

conf/proxy.ini

+InfaAgent.ProxyHost=
+InfaAgent.ProxyPort=
+InfaAgent.ProxyUser=
+InfaAgent.ProxyPassword=

docker-compose.yml

@@ -8,7 +8,4 @@ services:
     environment:            # Values can be set in a .env file or passed in at runtime
       INFORMATICA_USER:
       INFORMATICA_PASSWORD:
-      # JSON_LOG: "true"
-    # volumes:
-    #   - ./volumes/infaagent.ini:/home/infaagent/apps/agentcore/conf/infaagent.ini
-    #   - ./volumes/export:/export
+      JSON_LOG: "true"