Skip to main content

Accessing Amazon Secrets Manager

This section provides instructions on configuring access to Amazon Secrets Manager for Anyscale clusters. Accessing stored secrets in Amazon Secrets Manager is often necessary for various workloads and clusters. The configuration process involves two key steps:

  1. Policy Assignment for IAM Role: Ensure that the IAM Role associated with Anyscale Clusters has the appropriate IAM policies for reading secrets.
  2. (Optional) Resource Policy for Secrets:: Optionally, the Resource Policy associated with the secret may require modification to grant access permissions to the IAM Role used by Anyscale Clusters.

The following documentation will guide users through these steps to facilitate secure and effective integration with Amazon Secrets Manager.

Configure IAM Role​

Required Information​

  • Secret ARN: This ARN will look like arn:aws:secretsmanager:<region>:<account_id>:secret:<secret_name>.
  • The IAM Role that the cluster is running with.
note

Determine the Cluster's IAM Role​

By default, Anyscale clusters run with a cloud-specific IAM role.

If you followed instructions on how to run with a custom IAM role, use that role for the rest of the instructions.

Ensure the Cluster's IAM Role has access to your Secret​

  1. In the AWS console, go to the IAM Roles console
  2. Search for the Cluster role and select it.
  3. Add an inline policy to the role. This policy will grant read access to the Amazon Secret.
    1. In the Permissions tab, click Add permission and then Create inline policy.
    2. Click on the JSON tab
    3. Use the following JSON as a starting point - make sure to modify <secret-arn> with your Secret's ARN.
{
  "Statement": [
    {
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Effect": "Allow",
      "Resource": [
        "<secret-arn>"
      ],
      "Sid": "SecretsManagerGetSecretValue"
    }
  ]
}
  1. (Optional) You may need to update or attach a resource-based policy to the Secret. Please refer to the AWS Secrets Manager Policy documentation
info

More information on managing IAM Policies associated with IAM Roles can be found in the AWS IAM documentation.

More information and additional example Secrets Manager policies can be found in the AWS Secrets Manager documentation.

Using Secrets​

There are several ways that Secrets can be utilized with clusters and applications running with Anyscale. Here is an example:

Cluster Initialization Scripts​

Cluster Init Scripts are part of an Anyscale Cluster Environment. These shell scripts run on all nodes in a cluster after the node is started, but before Ray starts. This gives you a very flexible way to integrate third-party tools, clone private git repos, or perform other application startup requirements. Init scripts need to be created inside the folder /anyscale/init.

Here is an example of using a init script to retrieve a GitHub Personal Access Token stored in Secrets Manager, and perform a git clone of a private git repo.

  1. Create a Personal Access Token in GitHub, granting it permissions to read from your private GitHub repository.
  2. Create a Cluster Environment using either a Custom Docker Image hosted in ECR, or a standard Cluster Environment using a YAML file for the definition.
  3. Follow one of the following examples to create a Cluster Environment.
base_image: anyscale/ray:2.9.0-py310
env_vars:
  PRIVATE_REPO_NAME: <your_private_repo>
  SECRET_NAME: <example_secret/github_token>
  AWS_DEFAULT_REGION: <your_aws_region>
debian_packages:
  - jq
post_build_cmds:
  - sudo mkdir -p /anyscale/init
  - sudo touch /anyscale/init/github_clone.sh
  - echo 'mkdir -p $ANYSCALE_WORKING_DIR && cd $ANYSCALE_WORKING_DIR' | sudo tee -a /anyscale/init/github_clone.sh
  - echo 'SECRET_VALUE=$(aws secretsmanager get-secret-value --secret-id ${SECRET_NAME} | jq -r .SecretString)' | sudo tee -a /anyscale/init/github_clone.sh
  - echo 'GITHUB_TOKEN=$(echo ${SECRET_VALUE} | jq -r .github_token)' | sudo tee -a /anyscale/init/github_clone.sh
  - echo 'git clone https://${GITHUB_TOKEN}@github.com/${PRIVATE_REPO_NAME}' | sudo tee -a /anyscale/init/github_clone.sh