Skip to main content

Configure Google Cloud resources for an Anyscale cloud

Before you run Ray workloads on Anyscale, an Anyscale organization owner must configure Google Cloud resources for an Anyscale cloud. This integration enables Anyscale to manage resources like compute instances and storage directly in a GCP project.

Prerequisites​

  1. Create a Google Cloud project for Anyscale to operate in.
  2. Install the Google Cloud CLI.
  3. Optional: Authenticate the Google Cloud CLI in the project.

1. Install the Anyscale CLI​

  1. Run the following command to install the Anyscale CLI and Python client package:
pip install -U "anyscale[gcp]"
  1. To authenticate your credentials, run the following command, which fetches and updates the token that confirms your identity in the ~/.anyscale/credentials.json file:
anyscale login

If necessary, log in to the Anyscale console to complete authentication.

2. Choose a resource configuration method​

Configuring Google Cloud resources for an Anyscale cloud integrates Anyscale's capabilities into your GCP project to leverage its compute, storage, and networking resources for scalable, distributed computing.

You can use one of two different configuration methods that use the Anyscale CLI. Choose a method based on your organization's requirements:

  • anyscale cloud setup - Use for rapid configuration and a straightforward, low-maintenance solution; deploy in public subnets and access over public IP addresses without setting up additional networking infrastructure.
  • anyscale cloud register - Suited for teams with advanced cloud expertise, seeking enhanced security, custom private networking, and specific compliance needs.

3. Configure cloud resources​

Based on the configuration method selected from the previous section, configure Google Cloud resources for your Anyscale cloud with the following instructions.

For the anyscale cloud setup method, Anyscale automatically creates and configures the necessary resources within your GCP project. You deploy Ray clusters in public subnets and access them using public IP addresses without needing to set up additional networking infrastructure like VPNs.

Note: To manually customize resources, use the (custom) cloud register method instead.

Direct Networking

Step 1: Verify permissions in the GCP project

Go to your role in the GCP project and ensure that your user account has the iam.serviceAccounts.setIamPolicy permission enabled so that Anyscale can grant Google Cloud Deployment Manager the permission to set IAM policies.

Step 2: Enable Google Cloud APIs

Enable the Cloud Resource Manager API so that Anyscale can enable all other required APIs.

Step 3: Configure cloud resources

Run the following command to configure Google Cloud resources for a new cloud:

anyscale cloud setup \
--provider gcp \
--name ANYSCALE_CLOUD_NAME \
--region GCP_COMPUTE_REGION \
--project-id GCP_PROJECT_ID \
--enable-head-node-fault-tolerance
🏁Optional flags

--enable-head-node-fault-tolerance: Enables head node fault tolerance in Anyscale services by configuring an additional Memorystore instance for the Ray Global Control Store. Note that this flag extends the setup time by approximately 10 minutes.

4. Verify cloud resources​

Anyscale provides a CLI command to verify cloud resources for both configuration methods. Anyscale runs verification automatically during cloud creation and you can also run the verification on demand.

Trigger functional verification by specifying --functional-verify workspace or --functional-verify service. Anyscale launches a workspace or a service to verify that the cloud is functional.

$ anyscale cloud verify --name my-gcp-cloud

Authenticating
Loaded Anyscale authentication token from ANYSCALE_CLI_TOKEN.

Output
(anyscale +0.3s) Verifying APIs enabled for project...
(anyscale +0.6s) All required APIs are enabled.
(anyscale +0.6s) Verifying service accounts...
(anyscale +1.2s) Service accounts verification succeeded.
(anyscale +1.2s) Verifying VPC and subnets...
(anyscale +1.5s) VPC and subnet verification succeeded.
(anyscale +1.5s) Verifying firewall rules...
(anyscale +1.8s) Firewall rules verification succeeded.
(anyscale +1.8s) Verifying Cloud Storage bucket...
(anyscale +2.0s) Cloud Storage bucket verification succeeded.
(anyscale +2.0s) Verifying Filestore...
(anyscale +2.2s) Filestore verification succeeded.
(anyscale +2.2s) Start functional verification...
Functional verification for WORKSPACE is about to begin.
It will spin up one n1-standard-4 instance and will incur a small amount of cost.
For workspace verification, it takes about 5 minutes.
The instances will be terminated after verification. Do you want to continue? [y/N]: y
╭──────────────────────────────────────────── workspace verification ────────────────────────────────────────────╮
│ 0:00:02 Workspace created at https://console.anyscale.com/workspaces/expwrk_xyz/ses_xyz                        │
│ 0:01:45 Workspace is active.                                                                                   │
│ 0:00:00 Workspace termination initiated.                                                                       │
╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
0:01:47 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ Workspace verification succeeded!

Manage cloud resources​

Update an existing cloud​

If you configured your cloud using anyscale cloud setup, you can update certain cloud features:

anyscale cloud update --name <cloud-name> --enable-head-node-fault-tolerance

Delete cloud resources​

To delete an Anyscale cloud and optionally its associated Google Cloud resources:

anyscale cloud delete --name <cloud-name>
caution

For clouds created with anyscale cloud setup, this command uses Google Cloud Deployment Manager to delete all resources created by Anyscale, including:

  • VPC and subnets
  • Firewall rules
  • Service accounts
  • Cloud Storage buckets (and all data within them)
  • Filestore instances
  • Memorystore instances

For clouds registered with anyscale cloud register, this command only removes the cloud from Anyscale but doesn't delete any Google Cloud resources. You must manually delete Google Cloud resources or use your Terraform scripts.

Glossary of cloud resources​

ResourceDescription
Virtual Private Cloud (VPC)A virtual network within your GCP project. Anyscale deploys each Cloud in its own VPC, providing logical isolation from others.
SubnetsRanges of IP addresses within your VPC. GCP resources like GCE VM instances are attached to these subnets. Anyscale deploys workloads within these defined VPCs and subnets.
Firewall rulesCollections of firewall rules that secure the cloud environment by controlling incoming and outgoing traffic to GCP resources. Anyscale requires specific firewall rules to enable access to its suite of components and applications.
Service accountsIdentity and access management accounts that allow Anyscale to manage resources in your GCP project. Includes both the Anyscale Access service account (for control plane operations) and the Ray Cluster service account (attached to cluster nodes).
Cloud Storage bucketGoogle Cloud Storage provides object storage to store cluster logs, workspace snapshots, Ray checkpoints, and other artifacts. Anyscale-created clusters always have access to this bucket.
Filestore (Optional)Google Cloud Filestore provides managed file storage for use with GCE instances. When present, Anyscale uses Filestore for shared storage between cluster nodes.
Memorystore (Optional)Google Cloud Memorystore for Redis provides a Redis-compatible in-memory database service used for head node fault tolerance in Anyscale services.
Workload Identity FederationEnables Anyscale's control plane (running in AWS) to authenticate and manage resources in your GCP project without using long-lived service account keys.

Next steps​