AWS Databricks Setup

Some companies require different cloud devops teams to be involved with this step for security reasons. Please contact [e-mail] for any assistance.

Step 1: Cloud provider and region

• Cloud Provider - Select AWS as your cloud provider

• AWS Region - This is necessary only if you specify AWS as your cloud provider.

• Logs and Data Collection - Choose how you want to provide logs to Gradient. We recommend Sync-Hosted collection which manages collecting logs with just a few configurations on your end. Self-Hosted requires additional set up on your end.

For the recommended Sync-Hosted collection method, AWS IAM roles and permissions are required to be set up to complete the rest of the workspace integration, as seen in the screen shot below.

We recommend using Sync-Hosted collection for your logs and other data. Sync-Hosted collection enables you to integrate with Gradient in 5 minutes.

Step 2: Create AWS IAM Permissions Policy

Copy and paste the json in a new AWS IAM policy permission, as see in the example screen shot below. Give the policy a name, such as sync-external-access

This permission will be used in the next few steps after a new role is created.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DescribeVolumes"
            ],
            "Resource": "*"
        }
    ]
}

AWS IAM Policy: sync-external-access

Step 2: Create an AWS IAM Role (custom trust policy)

Create a new AWS IAM role with the "Custom trust policy" trust entity and paste the JSON in the policy field, as seen in the example screenshot below. Give the role a name, such as sync-external-user-role

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::533267411813:role/sync-computing-collector"
            },
            "Action": "sts:AssumeRole",
            "Condition": {
                "StringEquals": {
                    "sts:ExternalId": "<externalId from Gradient>"
                }
            }
        }
    ]
}

AWS IAM Custom Trust Role: sync-external-user-role

Step 3: Add permission created in step 2

In the next step of the AWS IAM role creation, add the permission created previously to the new AWS IAM role (in the example above it is named sync-external-access). Example screen shot below:

Step 4: Name and create the IAM role

Give the new IAM role a name, such as sync-external-user-role, and create the new role.

Step 5: Copy and paste the AWS IAM Role ARN

Go back to the AWS IAM role just created (in the previous example with the name sync-external-user-role), and copy the ARN link and paste it in the last field in the Gradient dialogue box.

Once entered, click on "Save & Test Access" to proceed in the Gradient UI.

Access check test - A series of access checks will occur to verify the permissions have been properly set up. If all checks are passed with green checks, you may proceed to the next section. If there are any red checks, please verify the installation is correct. Please email [e-mail] if there are any issues or questions.

Final Step

Click on the link below to proceed to the webhook setup.

pageWebhook Setup