Skip to content
Snippets Groups Projects

Playbook content for flat file alternative using S3 INPLATFORM-176

Merged Shreyash Shrestha requested to merge (removed):s3 into master

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • jont@wisc.edu
  • added 1 commit

    Compare with previous version

  • Shreyash Shrestha resolved all threads

    resolved all threads

  • Nuwan Rajika Kumarasiri approved this merge request

    approved this merge request

  • jont@wisc.edu approved this merge request

    approved this merge request

  • added 1 commit

    Compare with previous version

  • Shreyash Shrestha unmarked as a Work In Progress

    unmarked as a Work In Progress

  • added 1 commit

    Compare with previous version

  • added 1 commit

    Compare with previous version

  • added 1 commit

    Compare with previous version

  • Nuwan Rajika Kumarasiri approved this merge request

    approved this merge request

  • jont@wisc.edu approved this merge request

    approved this merge request

  • mentioned in commit 312ac351

  • 24 After you have acquired an AWS account, you can create an IAM User with Read/Write permissions to AWS S3 Buckets in your account.
    25 If you are using AWS S3 only as a source, then you may only need Read permission. An AWS Access and Secret key can be generated for this IAM User which is then used in the IICS AWS S3 Connection Properties.
    26
    27 [IICS AWS S3 Connection Documentation](https://docs.informatica.com/integration-cloud/cloud-data-integration/current-version/data-integration-connections/connection-properties/amazon-s3-connection-properties.html)
    28
    29 ### AWS S3 for testing and debugging integrations
    30
    31 AWS S3 provides a great mechanism to test pieces of a large ETL project.
    32 When an ETL project relies on multiple sources and transformations, small pieces of the transformation can be designed and tested by outputting to an S3 bucket for verification.
    33 Once the verification is successful, the next transformation can be designed and tested in the same fashion.
    34 This method can also be used for testing changes that need to be made on an ETL process.
    35 Parts that need to be changed can be duplicated and verified in an S3 bucket.
    36
    37 ## WiscAlerts example
    38
    39 The [WiscAlerts example](./tutorials/wiscalerts/wiscalerts.md) describes a successful use of AWS S3 as a Flat file alternative.
  • 26
    27 [IICS AWS S3 Connection Documentation](https://docs.informatica.com/integration-cloud/cloud-data-integration/current-version/data-integration-connections/connection-properties/amazon-s3-connection-properties.html)
    28
    29 ### AWS S3 for testing and debugging integrations
    30
    31 AWS S3 provides a great mechanism to test pieces of a large ETL project.
    32 When an ETL project relies on multiple sources and transformations, small pieces of the transformation can be designed and tested by outputting to an S3 bucket for verification.
    33 Once the verification is successful, the next transformation can be designed and tested in the same fashion.
    34 This method can also be used for testing changes that need to be made on an ETL process.
    35 Parts that need to be changed can be duplicated and verified in an S3 bucket.
    36
    37 ## WiscAlerts example
    38
    39 The [WiscAlerts example](./tutorials/wiscalerts/wiscalerts.md) describes a successful use of AWS S3 as a Flat file alternative.
    40
    41 EI Staff have implemented the integration using AWS S3 instead of relying on flat files.
    • Suggested change
      41 EI Staff have implemented the integration using AWS S3 instead of relying on flat files.
      41 The Integration Platform team has implemented the integration using AWS S3 instead of relying on flat files.

      Recommend using the name of our team to align with other external documents.

    • Please register or sign in to reply
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading