Skip to main content

Compute configs

Check your docs version

This version of the Anyscale docs is deprecated. Go to the latest version for up to date information.

Compute configurations define the hardware and hardware provisioning resources that a cluster utilizes. Anyscale provides default templates that users can further customize and version to optimize performance, cost, and developer productivity. Conceptually, a compute config allows you to control the following:

  • Instance types: Virtual machine or hardware instance types for head and worker nodes.
  • CPU and memory specifications: Computational power (CPU cores) and memory per node.
  • GPU requirements: GPU types and quantities for GPU-intensive workloads.
  • Network settings: Network interfaces, including subnet settings, security groups, and public IP options.
  • Storage options: Node-attached storage, detailing disk types and sizes.
  • Scaling parameters: Minimum and maximum worker counts to determine scaling behavior based on workload.
  • Spot instance and fallback options: Spot instances for cost savings and on-demand instances to fallback onto for reliability.
  • Cloud-specific advanced configurations: Provider-specific configurations, such as advanced networking and service integrations for clouds like AWS or GCP.

These resources collectively make up the virtual hardware and its configuration that the cluster uses to run applications and workloads in a distributed computing environment. If you're familiar with a Ray cluster.yaml, an Anyscale compute config represents a subset of that comprehensive configuration file.

Detailed configuration example​

The following is a fully detailed compute configuration, which depends on the AWS or GCP environment the Anyscale Cloud runs within.

cloud: CLOUD_NAME # Use `CLOUD_ID` with SDK
allowed_azs:
  - AVAILABILITY_ZONE_NAME
head_node_type:
  name: HEAD_NODE_NAME
  instance_type: INSTANCE_TYPE_HEAD # Defaults to `m5.8xlarge`
worker_node_types:
  - name: CPU_WORKER_NAME # Used in logs
    instance_type: INSTANCE_TYPE_WORKER_CPU # Defaults to `m5.8xlarge`
    min_workers: MIN_WORKERS_CPU
    max_workers: MAX_WORKERS_CPU
    use_spot: USE_SPOT_INSTANCES # Defaults to false
    fallback_to_ondemand: FALLBACK_TO_ONDEMAND # Defaults to false
  - name: GPU_WORKER_NAME # Used in logs
    instance_type: INSTANCE_TYPE_WORKER_GPU # Defaults to `g4dn.4xlarge`
    min_workers: MIN_WORKERS_GPU
    max_workers: MAX_WORKERS_GPU
# AWS-specific advanced configurations (optional)
aws:
  BlockDeviceMappings:
    - DeviceName: DEVICE_NAME
      Ebs:
        VolumeSize: VOLUME_SIZE
        DeleteOnTermination: DELETE_ON_TERMINATION
  IamInstanceProfile:
    Arn: IAM_INSTANCE_PROFILE_ARN
  NetworkInterfaces:
    - SubnetId: SUBNET_ID
      Groups:
        - SECURITY_GROUP_ID
      AssociatePublicIpAddress: ASSOCIATE_PUBLIC_IP
  TagSpecifications:
    - ResourceType: RESOURCE_TYPE
      Tags:
        - Key: TAG_KEY
          Value: TAG_VALUE
note

The advanced AWS-specific configurations in the aws block directly interface with the EC2 API during instance creation. This section allows for the customization of instance parameters, including EBS disk size, security groups, and subnet configurations. Anyscale supports the fields in the provided example, and these settings apply uniformly to all instances in the cluster.

Exercise caution with advanced AWS-specific configurations. Inaccurate settings could lead to cluster deployment failures, as these fields aren't validated prior to submission to AWS.

Create a compute config​

You can create a compute config with one of three options: the Web UI, CLI, or the Python SDK. The CLI and Python SDK methods require a YAML file that follows the schema in the preceding example.

Navigate to Configurations, then Compute configs, and select Create new config to create a compute config. You can also select, customize, or create compute configs upon the creation of a Workspace, Job, or Service.

Create a new compute config in the configurations page

Managing capacity reservations​

Securing certain instance types from cloud providers can sometimes be challenging due to high demand or limited availability. Anyscale Clusters offer capacity reservations, ensuring the availability of required node types for your workloads. To configure capacity reservations for a specific worker node type, you need to modify the Advanced Instance Configuration either through the Web UI or by editing the compute config YAML file.

For the Web UI, navigate to a worker node and select Resources and instance config (advanced). Under the Instance config tab, input the following JSON, substituting in your specific reservation ID:

{
  "CapacityReservationSpecification": {
    "CapacityReservationTarget": {
      "CapacityReservationId": "RESERVATION_ID"
    }
  }
}

For CLI or SDK usage, the YAML configuration should be as follows:

cloud: CLOUD_NAME # Use `CLOUD_ID` with SDK
allowed_azs:
  - AZ_NAME
head_node_type:
  name: HEAD_NODE_TYPE
  instance_type: INSTANCE_TYPE_HEAD
worker_node_types:
  - name: ACCELERATED_WORKER_TYPE
    instance_type: INSTANCE_TYPE_ACCELERATED
    min_workers: MIN_WORKERS
    max_workers: MAX_WORKERS
    aws_advanced_configurations_json:
      CapacityReservationSpecification:
        CapacityReservationTarget:
          CapacityReservationId: RESERVATION_ID

Anyscale Autoscaler​

The Anyscale Autoscaler is a tool that automatically adjusts the number of running nodes based on the current workload and resource usage. It helps to ensure that the system is running efficiently and cost-effectively. The Autoscaler can scale up or down dynamically, depending on the real-time workload, based on each node type's minimum and maximum worker values.