Skip to main content

Adjustable Down-scaling Compute Configs User Guide

Autoscaler is a component that seamlessly optimizes running nodes based on real-time workload and resource utilization. It ensures that the system's performance remains both efficient and cost-effective.

When a node is not running any workload, and the current number of alive workers is above its min count, Autoscaler will terminate the node after a period of time. This period is 5 minutes by default. Adjustable down-scaling provides user the ability to adjust this time period based on their workload. For instance, if the workload is short and bursty, then the user can set a shorter time period to have faster idle termination.

How to use​

  • The feature is enabled for Ray 2.7+ and nightly images built after Aug 22, 2023.
  • In general, we recommend to use a threshold greater than 1 minute.

Via the Console​

Configuration for Adjustable Down-scaling in the Anyscale Console is done completely via the Advanced Configurations input box. The JSON key value you want to specify is:

{
  "TagSpecifications": [
    {
      "Tags": [
        {
          "Key": "as-feature-idle-terminate-second",
          "Value": "60"
        }
      ],
      "ResourceType": "instance"
    }
  ]
}

Via SDK​

compute_config.yaml

cloud: anyscale_v2_default_cloud_vpn_us_east_2 # You may specify `cloud_id` instead
allowed_azs:
  - us-east-2a
  - us-east-2b
  - us-east-2c
head_node_type:
  name: head_node_type
  instance_type: m5.2xlarge
worker_node_types:
  - name: cpu_worker
    instance_type: c5.8xlarge
    min_workers: 2
    max_workers: 10
    use_spot: true
  - name: gpu_worker
    instance_type: g4dn.8xlarge
    min_workers: 0
    max_workers: 10
aws:
  TagSpecifications:
    - ResourceType: instance
      Tags:
        - Key: as-feature-idle-terminate-second
          Value: 60

Python SDK Code​

import yaml

from anyscale.sdk.anyscale_client.models import CreateClusterCompute
from anyscale import AnyscaleSDK

sdk = AnyscaleSDK()

with open('compute_config.yaml') as f:
compute_configs = yaml.safe_load(f)

# If your config file contains `cloud`, use this to get the `cloud_id`
if "cloud" in compute_configs:
compute_configs["cloud_id"] = sdk.search_clouds(
{"name": {"equals": compute_configs["cloud"]}}
).results[0].id
del compute_configs["cloud"]

config=sdk.create_cluster_compute(CreateClusterCompute(
name="my-cluster-compute",
config=compute_configs
))