README.md

Table Of Contents

• User Guide
• IICS Secure Agent Docker Image
  • Description
  • Requirements
  • The Image
    • ODBC
    • Image Scanning
  • Containers
    • Environment Variables
    • Externalized Configurations
    • Starting
    • Monitoring
  • Hardware Requirements
  • Known Issues
  • Enterprise Integration Secure Agent
    • Architecture Diagram
    • CI/CD and Terraform
    • Informatica User
    • Credentials in Terraform
      • Test Environment
      • Production Environment
    • Memory and CPU for Secure Agent
    • EFS Mounting, Accessing Secure Agent Configurations, Logs and Additional Debugging

User Guide

User guide for AWS hosted Secure Agent is available here.

IICS Secure Agent Docker Image

This document covers following topics:

1. How ro run IICS Secure Agent using Docker image.
2. IICS Secure Agent deployed(including Terraform configuration) in AWS(interop).

Description

This project was forked from jbrazda/ic-sagent-docker with the intention of containerizing the IICS Secure Agent to run in Amazon Web Services. As of early January 2019, there is no official Docker image for IICS Secure Agent.

Requirements

• Docker.
• An IICS user account and password for your organization that has appropriate rights to create and manage secure agents. This user will be used to login to the Informatica APIs to register the agent. At a minimum, the user should be assigned a role that has Create, Read, and Update privileges for the "Secure Agent" and "Secure Agent Group" asset types.
• Your Informatica POD and REGION, which can be inferred from https://${POD}.${REGION}.informaticacloud.com/. Defaults are set to POD=usw3 and REGION=dm-us, but can be overriden with Docker environment variables.

The Image

The image is based on Ubuntu 18.04. It downloads and installs necessary dependencies, and then downloads the installer file from the correct Informatica URL based on your POD and REGION.

1. Clone this repository.
2. docker build . -t iics_secure_agent:<tag> - optionally pass --build-arg parameters for POD and REGION if the defaults are not correct. POD and REGION are also set as environment variables, because containers will use these values for communicating with the Informatica API.

ODBC

ODBC support has been added to this image, specifically support for SQLite3 and Snowflake. An odbc.ini file can be mounted into the container as /home/agent/.odbc.ini for setting up connections. Local files can be mounted or copied into the container as necessary.

Image Scanning

The image is scanned by Qualys, a container image vulnerability scanning system, through a Gitlab CI/CD job: image_scan. This job is built upon the qualy_scan job from thecheck-qualys-job.yml template in checkQualys tool repository. An image_scan job will be created and a Gitlab issue will be added to this repository for each CVE vulnerability that occurs.

To enable container image scanning, you need to first check following criteria:

• A Gitlab API Project Access Token for checkQualys tool is valid and stored as CI_TOKEN in Settings -> CI/CD -> Environemnt Variables, if not, you need to generate one in checkQualys repository under Settings -> Access Token, following the instruction here;
• A Gitlab runner in Settings -> CI/CD -> Runners with qualys tag is enabled.

Notes:

• A Gitlab API Project Access Token specify for checkQualys is necessary and can be found in checkQualys repository that allows the checkQualys tool to post issues back to the project that is being scanned.
• A manual qualys_scan job at post stage will be created when including job template from checkQualys, and cannot be removed, but it can be ignored.
• This scan job will report all vulnerabilities in the Gitlab issue once occur, if there are a particular vulnerability that does not impact this deployment, a .cveignore file with certain CVE ID can be added at the root level to ignore/silence it.

Containers

The container executes a bash script called run_agent.sh upon start. It needs INFORMATICA_USER and INFORMATICA_PASSWORD environment variables set during runtime in order to communicate with the Informatica API to check and register the Secure Agent.

Environment Variables

• INFORMATICA_USER (required) - User that can run the agent and access Informatica APIs
• INFORMATICA_PASSWORD (required) - Password for above credential. Used for API access only.
• JSON_LOG (optional) - If this is set, then the log output will be in JSON format.

It is best practice to put these environment variables in a Docker .env file.

Persisting Secure Agent Configuration With Docker Volumes

Once the container starts and registers with Informatica API using the INFORMATICA_USER and INFORMATICA_PASSWORD credentials, a file will be created in the INFA_HOME/apps/agentcore/conf directory called infaagent.ini. This file contains the organization and agent ids and the registration token and should be persisted, so that when a container is restarted or redeployed, the configuration will be saved.

In addition to infagent.ini, there are additional files in the INFA_HOME/apps/agentcore/conf directory that you may want to persist, including proxy.ini used when running behind a proxy server.

To persist these configuration files, use Docker volumes to either mount individual files or the conf directory into the Docker container.

Logs and Log Directories

The Secure Agent parent process has a startup log in INFA_AGENT/apps/agentcore.log and port and Tomcat logs in INFA_AGENT/apps/agentcore/logs.

The following directories contain log files for each of the IICS applications:

• INFA_HOME/apps/Administrator/logs
• INFA_HOME/apps/CIHProcessor/logs
• INFA_HOME/apps/Common_Integration_Components/logs
• INFA_HOME/apps/Data_Integration_Server/logs
• INFA_HOME/apps/FileIntegrationService/logs
• INFA_HOME/apps/MassIngestionRuntime/logs
• INFA_HOME/apps/OpsInsightsDataCollector/logs
• INFA_HOME/apps/process-engine/logs

The above file and directories can also be persisted using Docker volumes.

Ports

There are several ports in use by the secure agent, although it isn't strictly necessary to allow access to these ports in most cases.

• 7005 - Tomcat Shutdown Port
• 7080 and 7443 - Process Engine HTTP and HTTPS ports used to call the Secure Agent as a Platform (AAP) directly. In other words, Application Integration Processes that you have created can be called on a specific Secure Agent by URL, e.g. https://your_agent_IP:7443/process-engine/public/rt/LookupInvoiceAndDetails?invoiceID=123.
• 5432 - Process Engine PostgreSQL DB port. Used for load balanced Secure Agent groups that must all connect to the same master database.

See below section for examples on how to use volume and port mapping.

Starting

• Setting the hostname will provide the associated name in the IICS website.
• Note: since anybody who has access to see the processes can view the values INFORMATICA_USER and INFORMATICA_PASSWORD, it's recommenced to configure them in a Docker .env file.

$ touch agentcore.log infaagent.log
$ docker run -d \
    -h <hostname> \
    --env INFORMATICA_USER=xxxxx \
    --env INFORMATICA_PASSWORD=xxxx \
    -v $(pwd)/infaagent.log:/home/agent/infaagent/apps/agentcore/infaagent.log  \
    -v $(pwd)/agentcore.log:/home/agent/infaagent/apps/agentcore/agentcore.log  \
    -v $(pwd)/logs:/home/agent/infaagent/apps/agentcore/logs  \
    -p 7080:7080 \
    -p 7443:7443 \
    -p 5432:5432 \
    --name <container_name> iics_secure_agent:<tag>

• Note that if conf directory needs mounting, a minimum configuration for starting the Secure Agent is required, see conf for example.

$ touch agentcore.log infaagent.log
$ docker run -d \
    -h <hostname> \
    --env INFORMATICA_USER=xxxxx \
    --env INFORMATICA_PASSWORD=xxxx \
    -v $(pwd)/infaagent.log:/home/agent/infaagent/apps/agentcore/infaagent.log  \
    -v $(pwd)/agentcore.log:/home/agent/infaagent/apps/agentcore/agentcore.log  \
    -v $(pwd)/logs:/home/agent/infaagent/apps/agentcore/logs  \
    -v $(pwd)/data:/home/agent/infaagent/apps/agentcore/data \
    -v $(pwd)/conf:/home/agent/infaagent/apps/agentcore/conf \
    -p 7080:7080 \
    -p 7443:7443 \
    -p 5432:5432 \
    --name <container_name> iics_secure_agent:<tag>

Using Docker Compose

Create a .env file with the following entries.

INFORMATICA_USER=
INFORMATICA_PASSWORD=
HOSTNAME=
CONFIG_DIR=
LOG_DIR=
ODBC_DATABASE_DIR=
APPS_DIR=/home/agent/infaagent/apps

The included docker-compose.yml file uses the above environment variables to map directories into the container to persist log directories and infaagent.ini and odbc.ini files. Modify for your use.

docker-compose up

Monitoring

If volume mapping is not used use docker exec to attach to the running container.

docker exec -it <container_name> less agentcore.log

Hardware Requirements

Minimum Hardware Specs (reference):

• Windows version 7 (32-bit or 64-bit) or Windows XP Pro SP2/SP3 (32-bit).
• Linux (32-bit or 64-bit).
• Red Hat Enterprise Linux version 5.3 or above.
• SUSE Linux Enterprise version 10 and above.
• 2-4 CPU recommended.
• 2 GB RAM recommended dedicated for the Agent (Recent customers have used 8GB).
• 1 GB Hard disk space.
• 1 GB Network connectivity (faster the better as this will most likely the bottleneck).

Known Issues

• Doesn't seem to be able to run in host network mode because it won't be able to talk to internal ports. It would probably work if you expose those ports.

Enterprise Integration Secure Agent

Secure Agent containers are deployed in following tiers in interop/AWS account. This deployment is automated through Terraform (see below section on Terraform).