Compute configuration on Anyscale
A compute configuration defines the resources Anyscale uses to launch a Ray cluster for a workspace, job, or service.
You use the compute config to define the shape of the cluster. The shape of a cluster reflects the instance types used for the head node and worker nodes, as well as the number of worker nodes. Anyscale cluster support heterogeneous compute, meaning you can define worker nodes using different instance types, including CPUs and GPUs.
You use worker groups to configure a collection of worker nodes. All worker nodes launched from a worker group use the same instance type and configurations. You configure cluster scaling at the worker group level, but can also set a global maximum for CPUs and GPUs for your cluster.
The Anyscale compute config is a superset of the Ray cluster YAML configuration file.
Anyscale scopes compute configs at the Anyscale cloud level. If you need to replicate a compute config to another cloud, you can use the CLI to download the YAML and use it to create a new compute config. See Compute Config API Reference.
Available configurations vary based on where you deploy your Anyscale cloud. See the following pages for cloud-specific details:
- Compute configurations for AWS
- Compute configuration options for Google Cloud
- Compute configuration options for Kubernetes
- Advanced settings for compute configs on Anyscale
Serverless clouds (also called Anyscale-hosted clouds) don't support advanced configurations passed through to the underlying infrastructure. You can still configure Ray and Anyscale features such as scheduling priority.
Compute config settings overview
You specify compute configs for the head node, worker nodes, and cluster-wide.
The following sections provide a high-level overview of configuration options in the Anyscale console. You can also specify these options when creating a compute config using the CLI or SDK, or as part of the cluster configuration for your workspace, service, or job. See Compute Config Models.
When you launch Ray clusters from the Anyscale CLI, you can use the --compute-config flag to specify a compute config already registered to your Anyscale cloud. To specify a compute config directly with YAML, use the compute_config option in the workspace config, job config, or service config.
Anyscale provides tools to create and manage compute configs scoped to an Anyscale cloud deployment. You can also directly specify your compute config when defining a new cluster. See Create or version a compute config.
Node config
The following settings are common to all nodes in your cluster. Use these options to configure your head node or worker groups.
| Setting | Description |
|---|---|
| Instance type | The virtual machine instance type used for the node. Instance types are specific to the cloud. The Anyscale console uses shorthand for instance types to highlight the size CPU and GPU on the instance. |
| Instance config | Node-specific configurations that override cluster-wide configurations. These settings are cloud-specific and passed to the cloud provider to further customize the virtual machine. |
| Ray config | Specify custom resource rules to control how Ray schedules tasks and actors on your nodes. |
Worker nodes scaling config
Use the following settings to control scaling behaviors for worker groups:
| Setting | Description |
|---|---|
| Autoscaling | Enable autoscaling to allow Ray to dynamically scale your cluster between the specified minimum and maximum number of nodes. If you disable autoscaling, your cluster deploys with the specified number of nodes. |
| Scale with | Specify how you want Anyscale to use spot instances when adding nodes to the cluster. Choose from Spot instances, On-demand instances, or Spot first, fall back to on-demand. |
Cluster config advanced settings
Use the following settings to configure cluster-wide behavior:
| Setting | Description |
|---|---|
| Region | The region used to deploy your cluster. Maps to the region of your Anyscale cloud. |
| Allowed zones | The zones within the specified region that Anyscale searches for available compute. |
| Enable cross-zone scaling | Enable this option to allow Anyscale to scale your cluster by adding nodes across multiple available zones. |
| Maximum CPUs | Specify the total limit for the number of CPUs that Anyscale can add to your cluster across all worker groups. |
| Maximum GPUs | Specify the total limit for the number of GPUs that Anyscale can add to your cluster across all worker groups. |
| Instance config | Cluster-wide configurations passed to the cloud provider and used while deploying all nodes. If you specify settings at both the cluster and node level, the node config overrides the cluster config. |
| Advanced features | Additional cluster-wide settings for Ray or Anyscale settings. |
You can set the Instance config field at either the cluster or node level.
If you configure any settings in the Instance config field at the node level, that node ignores any cluster-wide configurations you specified using this field. Anyscale makes no attempt to merge Instance config settings at the node and cluster level.
Supported instance types
When deployed on cloud VMs, Anyscale supports a wide variety of instance types. For a full list, visit Supported Machine Types.
On machine pool and Kubernetes deployments, instance types are defined and managed by cloud administrators.