Skip to main content

Compute configs

Compute configurations define the hardware and hardware provisioning resources that a cluster utilizes. Anyscale provides default templates that users can further customize and version to optimize performance, cost, and developer productivity. Conceptually, a compute config allows you to control the following:

  • Instance types: Virtual machine or hardware instance types for head and worker nodes.
  • CPU and memory specifications: Computational power (CPU cores) and memory per node.
  • GPU requirements: GPU types and quantities for GPU-intensive workloads.
  • Network settings: Network interfaces, including subnet settings, security groups, and public IP options.
  • Storage options: Node-attached storage, detailing disk types and sizes.
  • Scaling parameters: Minimum and maximum worker counts to determine scaling behavior based on workload.
  • Spot instance and fallback options: Spot instances for cost savings and on-demand instances to fallback onto for reliability.
  • Cloud-specific advanced configurations: Provider-specific configurations, such as advanced networking and service integrations for clouds like AWS or GCP.

These resources collectively make up the virtual hardware and its configuration that the cluster uses to run applications and workloads in a distributed computing environment. If you're familiar with a Ray cluster.yaml, an Anyscale compute config represents a subset of that comprehensive configuration file.

Detailed configuration example

The following is a fully detailed compute configuration, which depends on the AWS or GCP environment the Anyscale Cloud runs within.

cloud: CLOUD_NAME # Use `CLOUD_ID` with SDK
allowed_azs:
- AVAILABILITY_ZONE_NAME
head_node_type:
name: HEAD_NODE_NAME
instance_type: INSTANCE_TYPE_HEAD # Defaults to `m5.8xlarge`
worker_node_types:
- name: CPU_WORKER_NAME # Used in logs
instance_type: INSTANCE_TYPE_WORKER_CPU # Defaults to `m5.8xlarge`
min_workers: MIN_WORKERS_CPU
max_workers: MAX_WORKERS_CPU
use_spot: USE_SPOT_INSTANCES # Defaults to false
fallback_to_ondemand: FALLBACK_TO_ONDEMAND # Defaults to false
- name: GPU_WORKER_NAME # Used in logs
instance_type: INSTANCE_TYPE_WORKER_GPU # Defaults to `g4dn.4xlarge`
min_workers: MIN_WORKERS_GPU
max_workers: MAX_WORKERS_GPU
# AWS-specific advanced configurations
aws:
BlockDeviceMappings:
- DeviceName: DEVICE_NAME
Ebs:
VolumeSize: VOLUME_SIZE
DeleteOnTermination: DELETE_ON_TERMINATION
IamInstanceProfile:
Arn: IAM_INSTANCE_PROFILE_ARN
NetworkInterfaces:
- SubnetId: SUBNET_ID
Groups:
- SECURITY_GROUP_ID
AssociatePublicIpAddress: ASSOCIATE_PUBLIC_IP
TagSpecifications:
- ResourceType: RESOURCE_TYPE
Tags:
- Key: TAG_KEY
Value: TAG_VALUE
note

The advanced AWS-specific configurations in the aws block directly interface with the EC2 API during instance creation. This section allows for the customization of instance parameters, including EBS disk size, security groups, and subnet configurations. Anyscale supports the fields in the provided example, and these settings apply uniformly to all instances in the cluster.

Exercise caution with advanced AWS-specific configurations. Inaccurate settings could lead to cluster deployment failures, as these fields aren't validated prior to submission to AWS.

Create a compute config

You can create a compute config with one of three options: the Web UI, CLI, or the Python SDK. The CLI and Python SDK methods require a YAML file that follows the schema in the preceding example.

Navigate to Configurations, then Compute configs, and select Create new config to create a compute config. You can also select, customize, or create compute configs upon the creation of a Workspace, Job, or Service.

Create a new compute config in the configurations page

Managing capacity reservations

Securing certain instance types from cloud providers can sometimes be challenging due to high demand or limited availability. Anyscale Clusters offer capacity reservations, ensuring the availability of required node types for your workloads. To configure capacity reservations for a specific worker node type, you need to modify the Advanced Instance Configuration either through the Web UI or by editing the compute config YAML file.

For the Web UI, navigate to a worker node and select Resources and instance config (advanced). Under the Instance config tab, input the following JSON, substituting in your specific reservation ID:

{
"CapacityReservationSpecification": {
"CapacityReservationTarget": {
"CapacityReservationId": "RESERVATION_ID"
}
}
}

For CLI or SDK usage, the YAML configuration should be as follows:

cloud: CLOUD_NAME # Use `CLOUD_ID` with SDK
allowed_azs:
- AZ_NAME
head_node_type:
name: HEAD_NODE_TYPE
instance_type: INSTANCE_TYPE_HEAD
worker_node_types:
- name: ACCELERATED_WORKER_TYPE
instance_type: INSTANCE_TYPE_ACCELERATED
min_workers: MIN_WORKERS
max_workers: MAX_WORKERS
aws_advanced_configurations_json:
CapacityReservationSpecification:
CapacityReservationTarget:
CapacityReservationId: RESERVATION_ID