Skip to main content

Anyscale Cloud IAM mapping (Developer Preview)

info

Cloud IAM mapping is in Developer Preview. It's only available on self-hosted AWS Clouds on EC2 and self-hosted GCP Clouds on GCE.

What is Cloud IAM mapping?​

By default, Anyscale deploys with a single data plane role that covers core functions. Cloud IAM mapping introduces capabilities that let cloud admins configure rules that apply different IAM roles to data plane nodes based on customizable parameters. When Anyscale launches a Ray cluster, the control plane matches the request against the configured rules and assigns the corresponding IAM role or Service Account. Cloud IAM mapping supports the VM stack (AWS EC2 and GCP GCE).

Configure IAM mapping settings​

Cloud IAM mapping consists of the following settings:

Set the mapping mode​

  • CUSTOMER_MANAGED: Use statically provisioned, existing IAM roles and configure the IAM mapping rules to apply them.

Define IAM mapping rules​

A list of selector-based rules determine which IAM role Anyscale applies based on workload attributes.

Use selector syntax​

A Kubernetes label selector syntax that defines the conditions for applying an IAM role. Supported labels:

  • workload-type=<job|service|workspace>
  • project=<project_name> (the Anyscale project name)
  • user=<user_email> (the user email or the Anyscale service account email)

Specify IAM role values​

The IAM role to use:

  • AWS: Specify the IAM instance profile name (not the ARN). Example: For ARN arn:aws:iam::12345:instance-profile/cld_abc-cluster_node_role, use cld_abc-cluster_node_role.
  • GCP: Provide the service account email.

Configure default behavior​

Specifies the behavior when no mapping rule matches a workload:

  • CLOUD_DEFAULT: Use the default IAM role of the Anyscale cloud.
  • FAIL: The control plane doesn't start the workload.

Understand permission requirements​

Configure cross-account permissions​

  • The Anyscale role for cross-account access used to launch EC2 or GCE instances must have permission to PassRole to any role specified in the configuration.
  • Each specified role must be able to read and write to the Anyscale bucket in the cloud using IAM policies or bucket policies.
info

The recommended way to ensure the correct permissions and access are maintained is to create a reusable policy that you can attach to enable additional roles to be added without needed explicit changes to other resources.

For example, on AWS, ensuring a standard naming convention for IAM roles allows you to create a wildcard rule for the PassRole operation within the Anyscale cross-account IAM role. Similarly, using a tagging framework and conditions within the S3 bucket policy for the cloud enables a more consistent experience when adding new IAM roles.

S3 bucket policy example
{
    "Effect": "Allow",
    "Principal": {
        "AWS": "*"
    },
    "Action": [
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListBucket"
    ],
    "Resource": [
        "<S3 bucket ARN>",
        "<S3 bucket ARN>/*"
    ],
    "Condition": {
        "StringEquals": {
            "aws:PrincipalAccount": "<AWS Account ID>",
            "aws:PrincipalTag/anyscale-cloud-id": "<Anyscale Cloud ID>"
        },
        "StringLike": {
            "aws:PrincipalArn": "arn:aws:iam::<AWS Account ID>:role/*"
        }
    }
}

View configuration examples​

The following examples demonstrate how to configure IAM mapping for different cloud providers:

Configure AWS EC2 IAM mapping​

To assign different IAM roles based on workload type and user, use the following configuration example:

cloud_deployment_id: cldrsrc_XXXX
cloud_provider: AWS
compute_stack: VM
dataplane_iam_mapping:
  mode: CUSTOMER_MANAGED
  rules:
    - selector: workload-type in (workspace),user=user1@example.com
      value: role_name1
    - selector: workload-type in (job, service),user=user2@example.com
      value: role_name2
  default_rule: CLOUD_DEFAULT

Configure GCP GCE IAM mapping​

To assign different service accounts based on workload type and user, use the following configuration example:

cloud_deployment_id: cldrsrcgcp_i93p4pp6k8fe1hca7kggbjvdyu
cloud_provider: GCP
compute_stack: VM
dataplane_iam_mapping:
  mode: CUSTOMER_MANAGED
  rules:
    - selector: workload-type in (workspace),user=user1@example.com
      value: sa1@xxx.iam.gserviceaccount.com
    - selector: workload-type in (job, service),user=user2@example.com
      value: sa2@xxx.iam.gserviceaccount.com
  default_rule: CLOUD_DEFAULT

Set up IAM mapping step by step​

Install the Anyscale CLI​

To begin configuring IAM mapping, install the Anyscale CLI:

pip install anyscale

Retrieve your current cloud configuration​

To see your existing cloud configuration, run the following command:

anyscale cloud config get -n YOUR_CLOUD_NAME

Expected Output:

cloud_deployment_id: cldrsrc_XXXX
cloud_provider: AWS
dataplane_iam_mapping: {}

Update your cloud configuration​

Export the current configuration to a YAML file and add your IAM mapping settings using the examples above.

Apply the updated configuration​

To deploy your IAM mapping changes, run the following command:

anyscale cloud config update -n YOUR_CLOUD_NAME --spec-file=cloud-config.yaml

Expected Output:

(anyscale +1.7s) Successfully updated cloud configuration for cloud YOUR_CLOUD_NAME

Verify your configuration​

To confirm that your IAM mapping was applied correctly, retrieve the cloud configuration again:

anyscale cloud config get -n YOUR_CLOUD_NAME