Skip to main content

Anyscale Cloud IAM mapping (Developer Preview)

info

Cloud IAM mapping is in Developer Preview. It's only available on self-hosted AWS Clouds on EC2 and self-hosted GCP Clouds on GCE.

Introduction​

By default, Anyscale deploys with a single data plane role that covers core functions. Cloud IAM mapping introduces capabilities that let cloud admins configure rules that apply different IAM roles to data plane nodes based on customizable parameters. When Anyscale launches a Ray cluster, the control plane matches the request against the configured rules and assigns the corresponding IAM role or Service Account. Cloud IAM mapping supports the VM stack (AWS EC2 and GCP GCE).

Schema​

Cloud IAM mapping consists of the following settings:

Mode​

  • CUSTOMER_MANAGED:
    Use statically provisioned, existing IAM roles and configure the IAM mapping rules to apply them.

IAM mapping rules​

A list of selector-based rules determine which IAM role Anyscale applies based on workload attributes.

  • Selector:
    A Kubernetes label selector syntax that defines the conditions for applying an IAM role. Supported labels:

    • workload-type=<job|service|workspace>
    • project=<project_name> (the Anyscale project name)
    • user=<user_email> (the user email or Anyscale service account email)

  • Value:
    The IAM role to use.

    • AWS: Specify the IAM role name (not the ARN).
      Example: For ARN arn:aws:iam::12345:role/cld_abc-cluster_node_role, use cld_abc-cluster_node_role.
    • GCP: Provide the service account email.

Additional notes​

  • The Anyscale role for cross-account access used to launch EC2 or GCE instances must have permission to PassRole to any role specified in the configuration.
  • Each specified role must be able to read and write to the Anyscale bucket in the cloud using IAM policies or bucket policies.
info

The recommended way to ensure the correct permissions and access are maintained is to create a reusable policy that you can attach to enable additional roles to be added without needed explicit changes to other resources.

For example, on AWS, ensuring a standard naming convention for IAM roles allows you to create a wildcard rule for the PassRole operation within the Anyscale cross-account IAM role. Similarly, using a tagging framework and conditions within the S3 bucket policy for the cloud enables a more consistent experience when adding new IAM roles.

S3 bucket policy example
{
    "Effect": "Allow",
    "Principal": {
        "AWS": "*"
    },
    "Action": [
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:GetObject",
        "s3:ListBucket"
    ],
    "Resource": [
        "<S3 bucket ARN>",
        "<S3 bucket ARN>/*"
    ],
    "Condition": {
        "StringEquals": {
            "aws:PrincipalAccount": "<AWS Account ID>",
            "aws:PrincipalTag/anyscale-cloud-id": "<Anyscale Cloud ID>"
        },
        "StringLike": {
            "aws:PrincipalArn": "arn:aws:iam::<AWS Account ID>:role/*"
        }
    }
}

Default rule​

Specifies the behavior when no mapping rule matches a workload:

  • CLOUD_DEFAULT: Use the default IAM role of the Anyscale cloud.
  • FAIL: The control plane doesn't start the workload.

Example configuration​

Follow these steps to configure cloud IAM mapping:

Install the Anyscale CLI​

$ pip install anyscale

Retrieve the current cloud configuration​

$ anyscale cloud config get -n YOUR_CLOUD_NAME

Output

cloud_deployment_id: cldrsrc_XXXX
cloud_provider: AWS
dataplane_iam_mapping: {}

Update the cloud configuration​

Export the current configuration to a YAML file and add your IAM mapping settings.

AWS EC2 cloud example
cloud_deployment_id: cldrsrc_XXXX
cloud_provider: AWS
compute_stack: VM
dataplane_iam_mapping:
  mode: CUSTOMER_MANAGED
  rules:
    - selector: workload-type in (workspace),user=user1@example.com
      value: role_name1  
    - selector: workload-type in (job, service),user=user2@example.com
      value: role_name2
  default_rule: CLOUD_DEFAULT
GCP GCE cloud example
cloud_deployment_id: cldrsrcgcp_i93p4pp6k8fe1hca7kggbjvdyu
cloud_provider: GCP
compute_stack: VM
dataplane_iam_mapping:
  mode: CUSTOMER_MANAGED
  rules:
    - selector: workload-type in (workspace),user=user1@example.com
      value: sa1@xxx.iam.gserviceaccount.com
    - selector: workload-type in (job, service),user=user2@example.com
      value: sa2@xxx.iam.gserviceaccount.com
  default_rule: CLOUD_DEFAULT

Apply the updated configuration​

$ anyscale cloud config update -n YOUR_CLOUD_NAME --spec-file=cloud-config.yaml

Output:

(anyscale +1.7s) Successfully updated cloud configuration for cloud YOUR_CLOUD_NAME

Verify the updated configuration​

Retrieve the cloud configuration again to confirm that you applied the IAM mapping:

$ anyscale cloud config get -n YOUR_CLOUD_NAME