Skip to main content

Clouds

An Anyscale cloud is used in cluster computes. It holds information about where and how to start clusters:

  • the cloud provider and region where the cluster will be deployed in your own AWS or GCP account, and
  • the identity (which maps to credentials) used to deploy clusters in your AWS or GCP account (if applicable).

This page describes how to set up and manage additional clouds.

Bring Your Own Cloud Setup

You can bring your own cloud to Anyscale to leverage your existing cloud account with a cloud service provider (AWS and GCP are currently supported). When you bring your own cloud, Anyscale still manages startup and shutdown of clusters, but the virtual machines are started in your account. The following diagram summarizes the architecture under this model:

Bring Your Own Cloud Architecture

Deployment on AWS

Prerequisites

  • You have registered a user account on Anyscale and have set up the Anyscale CLI locally.
  • You have IAM and EC2 privileges.
  • You have set up AWS credentials locally, i.e., you have run aws configure (for more details see here).

Creating a Cloud in AWS

In order for Anyscale to manage your compute, we'll need to register a set of AWS credentials with an Anyscale cloud.

note
  • This will create an IAM role (arn:aws:iam:::role/anyscale-iam-role-<hex>) in your account, allow Anyscale to assume this role, and grant the role permissions to perform machine-management related EC2 actions and IAM PassRole/GetInstanceProfile action (full policies here).
  • The setup process will use the default AWS credentials you set up locally. You can also specify environment variables such as AWS_PROFILE=user_1 when calling anyscale cloud setup to select a specific profile. Do not use the root user for any deployment operations.
  • The setup process will register a particular AWS region to your Anyscale cloud. Please ensure your AWS Service Quotas for EC2 for this region have sufficient capacity for your workloads.

The cloud setup process can be completed interactively on the Anyscale CLI:

$ anyscale cloud setup
Provider (aws, gcp): aws
Region [us-west-2]: us-east-1
Name: mycloud

You are about to give anyscale full access to EC2 and IAM in your AWS account.

Continue? [y/N]: y
Created IAM role arn:aws:iam::123456789:role/anyscale-iam-role-aabbccdd
AWS credentials setup complete!
You can revoke the access at any time by deleting anyscale IAM user/role in your account.
Head over to the web UI to create new sessions in your AWS account!

The same result can be achieved with a single command:

$ anyscale cloud setup --name mycloud --provider aws --region us-east-1

At the end of this process, your cloud will be immediately available for use if your region has a properly configured VPC.

VPC Requirements for Anyscale

Anyscale requires one subnet in your cloud's region to have the following properties.

  • Auto-associates IPs on instance launch.
  • Has an outbound route to the internet (via an internet-gateway).

The auto-created 'default VPC' has these properties. If the auto-created VPC has been removed, you can re-create it by going to the VPC console, clicking "Actions" and then selecting "Create default VPC".

To manually create a VPC and subnets, this gist provides the necessary Boto3 calls.

Credentialing the Anyscale Ray Cluster Role

Anyscale creates a role for use by Ray clusters (this is a different role than the one described above). By default, this role has the following permissions:

  • Full S3 Access

This role is named:

arn:aws:iam:::role/ray-autoscaler-v1

If there are other resources you want your applications to have access to, e.g., writing CloudWatch logs, reading S3, setting up AWS CLI, you'll need to grant them to the above ray-autoscaler-v1 role. Otherwise, you might see errors like the following:

botocore.exceptions.ClientError: An error occurred (AccessDeniedException) when calling the CreateLogGroup operation:
User: arn:aws:sts::<ACCN_NUMBER>:assumed-role/ray-autoscaler-v1/i-<INSTANCE> is not authorized to perform: logs:CreateLogGroup on resource: arn:aws:logs:ap-southeast-2:<ACCN_NUMBER>:log-group:<GRP_NAME>:log-stream:

This is a sign you need to add those permissions to the Anyscale role.

FAQ

How do I add the AWS CLI to my nodes in my Ray cluster?

We don't include AWS CLI by default in our Docker images. You can add this yourself by adding the following to the post-build commands of your cluster environment:

apt-get install -y curl unzip
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install

Please note you may need to properly assign permissions (as shown above) to the Anyscale role for the respective parts of the CLI to work correctly.

Do you store my credentials anywhere?

No. The credentials never travel across the network to Anyscale. Instead, Anyscale will create an IAM role in your cloud account, grant it permissions to interact with EC2 and IAM in your account and allow Anyscale to assume that role. Anyscale then only stores the IAM role ARN that is created in your account.

How do I revoke access?

You can revoke Anyscale's access to your AWS account by deleting the Anyscale IAM role in your account, which will look like anyscale-iam-role or anyscale-iam-role-<8 hex digits>.

What AWS regions are supported?

Anyscale supports all commercially available regions. We do not currently support the Beijing, Ningxia, or US GovCloud regions.

Deployment on GCP

Prerequisites

  • You have registered a user account on Anyscale and have set up the Anyscale CLI locally.
  • You have permissions to create a folder, either in the organization's root, or in the folder specified in setup, and you have roles/billing.admin role in the organization.

Creating a Cloud in GCP

In order for Anyscale to manage your compute, we'll need to register a set of GCP credentials with an Anyscale cloud and create an Anyscale folder in your GCP account.

The cloud setup process can be completed interactively on the Anyscale CLI:

$ anyscale cloud setup
Provider (aws, gcp): gcp
Region [us-west1]: us-central1
Name: mycloud
Please select the GCP Folder ID where the 'Anyscale' folder will be created.
Your GCP account must have permissions to create sub-folders in the specified folder.
View your organization's folder layout here: https://console.cloud.google.com/cloud-resource-manager
If not specified, the 'Anyscale' folder will be created directly under the organization.
Folder ID (numerals only): 1234567890
Launching GCP Oauth Flow:
https://console.anyscale.com/api/v2/clouds/gcp/create/mycloud?region=us-central1&folder_id=1234567890
(If this window does not auto-launch, use the link above)

The same result can be achieved with a single command:

$ anyscale cloud setup --name mycloud --provider gcp --region us-central1 --gcp-folder-id 123456789

Go to the URL returned by the Anyscale CLI, where you will be prompted to log into Google. Once logged in, you will then be prompted to verify the Google account and billing information. After you confirm the information, the cloud setup process will begin, which will take approximately 3 hours. You can check if the cloud creation process has completed by trying to start a cluster on the cloud: if this succeeds, it means that your cloud is ready.

Creating a VPC peering connection

Anyscale lets you connect your Anyscale managed GCP environment to existing resources in other GCP Projects through VPC peering. VPC peering lets your Anyscale clusters connect to existing applications, databases, or other resources using known CIDR ranges and without traversing the public Internet. Anyscale supports this need with VPC peering. To peer with another VPC Anyscale needs the following information:

  • VPC peering IP range

    • Specifies the range of IP addresses reserved for use by your Anyscale cloud. The CIDR range may not overlap with subnets peered with your Anyscale environment. Overlapping CIDR ranges will cause the peering connection to fail.

    • Anyscale accepts the following IP ranges (see here for valid IP ranges on GCP):

      • 10.0.0.0/8
      • 172.16.0.0/12
      • 192.168.0.0/16
    • The smallest IP range acceptable on Anyscale is /16

    • If a VPC IP range is not specified, the default IP range 172.16.0.0/12 is used.

    • The IP range limits the number of ray nodes that can be concurrently running in your Anyscale cloud.

      | IP range | # of Max Ray Nodes |
      | :------- | :----------------- |
      | /16 | 256 |
      | /15 | 512 |
      | /14 | 1024 |
      | /13 | 2048 |
      | /12 | 4096 |
  • Target VPC ID

    • The ID of the VPC that will be peered to the Anyscale VPC.
  • Target project ID

    • The ID of the project where the target VPC of the peering connection resides.

To specify anyscale VPC peering options, run the following command when setting up the Anyscale cloud (all 3 options must be specified at the same time):

anyscale cloud setup --vpc-peering-ip-range 10.0.0.0/12  --vpc-peering-target-vpc-id your_vpc_id --vpc-peering-target-project-id your_gcp_project_id

Resources Managed by Anyscale

In order for anyscale to function, we manage two types of resources in your GCP account.

Generational

Each Anyscale cloud has generations. A generation includes a set of resources and it can be rotated to the next generation to roll out features and fix intractable problems with the current generation (rotation is the process of creating a new set of resources and deleting the old ones). Generational resources include:

  • Subnet (each generation has a subnet)
  • GKE service account
  • Kubernetes cluster and node pools
  • Resources within the Kubernetes cluster (such as namespace and Kubernetes service)
Persistent

Persistent resources survive generations and are useful to provide persistent identity and networking for you to interact with the clouds. Persistent resources include:

  • GCP projects (currently Anyscale manages 2 GCP projects per cloud in your GCP account).

  • Google Artifacts Registry (this is used to store your cluster environments images).

  • Service Accounts

    • Google Artifacts Registry service account
    • Anyscale cloud service account (for more details of this service account, see here)
  • VPC (Anyscale manages one VPC per cloud)

Deployment on Azure

Azure support is currently under development.

Please contact Anyscale at support@anyscale.com if you'd like to request more information!

Cloud Management

You can also use the Anyscale CLI to manage clouds:

  • You can delete clouds by running anyscale cloud delete <cloud_name_or_cloud_id>. This operation is only supported if the cloud has no clusters associated with it.
  • You can make a cloud the default cloud by running anyscale cloud set-default <cloud_name_or_cloud_id>. The default cloud will be used when creating new clusters if no cloud is specified in the compute configs.