In the following tutorial we will explain how to setup an S3 bucket and give us access to it securely.

If you already export Kissmetrics, Segment, or your own data to a S3 bucket, we can access data directly from that bucket.

If the bucket you want to share already exists, please skip the following section (Create an S3 Bucket).

Create an S3 Bucket

Go to AWS Management Console
Go to S3 service from Services > Storage > S3

Create Bucket Amazon S3 > Create bucket

Fill in the form and make note of your bucket
You can use any bucket name, example: my-madkudu-shared-bucket

Setting up the correct access to your S3 bucket

For MadKudu to access your S3 bucket, our preferred option is for you to grant access to your S3 bucket to a MadKudu IAM role.

To do this, you'll need MadKudu's AWS account ID and External ID to Create an IAM policy and Create an IAM role (see AWS documentation).

Step 1. Get MadKudu AWS account ID and external ID

Visit app.madkudu.com > Integrations > Amazon S3 > Configuration to find MadKudu's account id and external id or use:

- Account ID: 203796963081
- External ID: your MadKudu API key (see below where to find it in app.madkudu.com)

Step 2. Create an IAM Policy for MadKudu

Go to your AWS Management Console.
Go to IAM Identity and Access Management from: Services > Security, Identity & Compliance > IAM.
Go to Policies and click on Create Policy.
Go to the JSON tab.

Copy the following policy and paste it in the JSON tab, replacing bucket_name with the name of your S3 bucket.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::203796963081:root"
            },
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::bucket_name"
            ]
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::203796963081:root"
            },
            "Action": [
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::bucket_name/*"
            ]
        }
    ]
}

Alternative: Instead of the JSON, use the Visual editor (AWS documentation).
Choosing the Service: S3 and Actions List: ListBucket, Read:GetObject.

Click Review Policy.
Name the policy "MadKudu-S3-Access".
Click Create Policy.

Step 3. Create an IAM Role for MadKudu

Got to Roles and Click on Create role.
Choose the Another AWS account role type.
For Account ID, type the MadKudu AWS account ID to which you want to grant access to your resources.
As you are granting permissions to users from an account that you do not control, and the users will assume this role programmatically, select Require external ID. (see more info)
Enter the External ID you got at Step 1.
Confirm that Require MFA is not selected.
Click Next: Permissions.
Select the "MadKudu-S3-Access" policy that you created in Step 2.
Click Next:Tags.
Add any tag and description to this role to identify it.
Click Next:Review.
Use any role name you want but we suggest to include "madkudu" in it, for example "integration-madkudu-s3-read" .
Click Create role.
Select the MadKudu role you just created..
Find the Role ARN and make note of it.
Change the maximum session duration to 12 hours (This is to ensure that MadKudu will be able to extract data from your bucket without the credentials expiring in the middle).
Go the Amazon S3 page in the MadKudu app: app.madkudu.com > Integrations > Amazon S3 > Configuration and input & save:
1. the Role ARN
2. region of your Bucket
3. bucket directory (with a folder path(s) if MadKudu can only access specific folders): this specifies the portion of the bucket in which you'd like MadKudu to pull data from. Any files under the specified folder and all of its nested subfolders will be examined for files we can upload. If no prefix is supplied, we'll look through the entire bucket for files to sync.

Encryption

We recommend that you encrypt your data in the S3 bucket for increased protection. If your data in S3 is encrypted server-side, you would need to add a policy to let us use the encryption key.

Step 1. Update the KMS key policy

1. Open the AWS KMS console, and then view the key's policy document using the policy view. Modify the key's policy to grant the IAM user permissions for the kms:GenerateDataKey and kms:Decrypt actions at minimum. You can add a statement that's similar to the following:

{
  "Sid": "ExampleStmt",
  "Action": [
    "kms:Decrypt",
    "kms:GenerateDataKey"
  ],
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::111122223333:user/Jane"
  },
  "Resource": "*"
}

Note: This example policy includes only the minimum permissions required for an individual IAM user to download and upload to an encrypted S3 bucket. You can modify or expand the permissions based on your use case.

Step 2. Update the IAM policy

2. Open the IAM console, update the IAM policy (created in Step 2 above) that grants the permissions to read from the bucket to work with the AWS KMS key that's associated with the bucket.

For the Resource value, enter the AWS KMS key's ARN.

    {
      "Sid": "KMSAccess",
      "Action": [
        "kms:Decrypt",
        "kms:GenerateDataKey"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:kms:example-region-1:123456789098:key/111aa2bb-333c-4d44-5555-a111bb2c33dd"
    }

Please contact us at support@madkudu.com if you are facing difficulties, or consult the F.A.Q.

NOTE: If all of this sounds like gibberish, please forward it directly to your favorite developer :)

Amazon S3 - Giving MadKudu access to an S3 bucket

Create an S3 Bucket

Setting up the correct access to your S3 bucket

Step 1. Get MadKudu AWS account ID and external ID

Step 2. Create an IAM Policy for MadKudu

Step 3. Create an IAM Role for MadKudu

Encryption

Step 1. Update the KMS key policy

Step 2. Update the IAM policy

Comments

Create an S3 Bucket

Setting up the correct access to your S3 bucket

Step 1. Get MadKudu AWS account ID and external ID

Step 2. Create an IAM Policy for MadKudu

Step 3. Create an IAM Role for MadKudu

Encryption

Step 1. Update the KMS key policy

Step 2. Update the IAM policy

Related articles