In the following tutorial we will explain how to setup an s3 bucket and give us access to it securely.
If you already export Kissmetrics, Segment (or your own data) to a S3 bucket, we can access data directly from that bucket.
If the bucket you want to share already exist please skip the following section - Create an S3 Bucket
Create an S3 Bucket
- Go to AWS Management Console
- Go to S3 service from Services > Storage > S3
- Create Bucket Amazon S3 > Create bucket
- Fill in the form and make note of your bucket
- You can use any bucket name, example: my-madkudu-shared-bucket
Setting up the correct access to your S3 bucket
For MadKudu to access your S3 bucket, our preferred option is for you to grant access to your S3 bucket to a MadKudu IAM role.
To do this, you'll need MadKudu's AWS account ID and External ID to Create an IAM policy and Create an IAM role (see AWS documentation).
Step 1. Get MadKudu AWS account ID and external ID
Visit app.madkudu.com > Integrations > Amazon S3 > Configuration to find MadKudu's account id and external id
or use:
- Account ID: 203796963081
- External ID: your MadKudu API key (see below where to find it in app.madkudu.com)
Step 2. Create an IAM Policy for MadKudu
- Go to your AWS Management Console
- Go to IAM Identity and Access Management from: Services > Security, Identity & Compliance > IAM
- Go to Policies and click on Create Policy
-
Go to the JSON tab
- Copy the following policy and paste it in the JSON tab, replacing
bucket_name
with the name of your S3 bucket{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::203796963081:root"
},
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::bucket_name"
]
},
{
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::203796963081:root"
},
"Action": [
"s3:GetObject"
],
"Resource": [
"arn:aws:s3:::bucket_name/*"
]
}
]
}
Alternative: Instead of the JSON, use the Visual editor (AWS documentation)
Choosing the Service: S3 and Actions List: ListBucket, Read:GetObject -
Click Review Policy.
-
Name the policy "MadKudu-S3-Access".
- Click Create Policy
Step 3. Create an IAM Role for MadKudu
- Got to Roles and Click on Create role
- Choose the Another AWS account role type.
- For Account ID, type the MadKudu AWS account ID to which you want to grant access to your resources.
- As you are granting permissions to users from an account that you do not control, and the users will assume this role programmatically, select Require external ID. (see more info)
- Enter the External ID you got at Step 1
- Confirm that Require MFA is not selected
- Click Next: Permissions
- Select the "MadKudu-S3-Access" policy that you created in Step 2.
- Click Next:Tags
Add any tag and description to this role to identify it - Click Next:Review
Use any role name you want but we suggest to include "madkudu" in it, for example "integration-madkudu-s3-read" - Click Create role
-
Select the MadKudu role you just created.
- Find the Role ARN and make note of it
- Change the maximum session duration to 12 hours (for MadKudu to be able to extract data from your bucket without the credentials expiring in the middle))
- Go the Amazon S3 page in the MadKudu app: app.madkudu.com > Integrations > Amazon S3 > Configuration and input & save
- The Role ARN
- region of your Bucket
- bucket directory (with a folder path(s) if MadKudu can only access specific folders): this specifies the portion of the bucket in which you'd like MadKudu to pull data from. Any files under the specified folder and all of its nested subfolders will be examined for files we can upload. If no prefix is supplied, we'll look through the entire bucket for files to sync.
Encryption
We recommend that you encrypt your data in the S3 bucket for increased protection. If your data in S3 is encrypted server-side, you would need to add a policy to let us use the encryption key.
Step 1. Update the KMS key policy
1. Open the AWS KMS console, and then view the key's policy document using the policy view. Modify the key's policy to grant the IAM user permissions for the kms:GenerateDataKey and kms:Decrypt actions at minimum. You can add a statement that's similar to the following:
{
"Sid": "ExampleStmt",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::111122223333:user/Jane"
},
"Resource": "*"
}
Note: This example policy includes only the minimum permissions required for an individual IAM user to download and upload to an encrypted S3 bucket. You can modify or expand the permissions based on your use case.
Step 2. Update the IAM policy
2. Open the IAM console, update the IAM policy (created in Step 2 above) that grants the permissions to read from the bucket to work with the AWS KMS key that's associated with the bucket.
For the Resource value, enter the AWS KMS key's ARN.
{
"Sid": "KMSAccess",
"Action": [
"kms:Decrypt",
"kms:GenerateDataKey"
],
"Effect": "Allow",
"Resource": "arn:aws:kms:example-region-1:123456789098:key/111aa2bb-333c-4d44-5555-a111bb2c33dd"
}
Please contact us at support@madkudu.com if you are facing difficulties, or consult the F.A.Q
NOTE: If all of this sounds like gibberish, please forward it directly to your favorite developer :)
Comments
0 comments
Please sign in to leave a comment.