Adjustable downscaling

Deprecated

Adjustable downscaling is deprecated. Instead, set the idle_termination_seconds cluster level flag in your compute configuration instead.

Autoscaler is a component that optimizes running nodes based on real-time workload and resource utilization. It ensures that the system's performance remains both efficient and cost-effective.

When a node is not running a workload and the current number of workers is above its minimum count, autoscaler terminates the node after a period of 4 minutes by default. Adjustable downscaling allows you to adjust this time period based on their workload needs. For instance, if the workload is short and intermittant, then the user can set a shorter time period to have faster idle termination.

info

The feature is enabled for Ray 2.7+ and nightly images built after Aug 22, 2023.

Usage

Best practice

In general, it's recommended to use a threshold greater than 1 minute.

Configuration for adjustable downscaling in the Anyscale console is done completely through the Advanced configurations input box. The JSON key value you want to specify is:

AWS (EC2)
GCP (GCE)

{
  "TagSpecifications": [
    {
      "Tags": [
        {
          "Key": "as-feature-idle-terminate-second",
          "Value": "60"
        }
      ],
      "ResourceType": "instance"
    }
  ]
}

{
  "instance_properties": {
    "labels": {
      "as-feature-idle-terminate-second": "60"
    }
  }
}

If you prefer to use the Anyscale CLI or SDK, click here to view an example YAML.

AWS (EC2)
GCP (GCE)

cloud: anyscale_v2_aws_useast2 # You may specify `cloud_id` instead
allowed_azs:
  - us-east-2a
  - us-east-2b
  - us-east-2c
head_node_type:
  name: head_node_type
  instance_type: m5.2xlarge
worker_node_types:
  - name: cpu_worker
    instance_type: c5.8xlarge
    min_workers: 2
    max_workers: 10
    use_spot: true
  - name: gpu_worker
    instance_type: g4dn.8xlarge
    min_workers: 0
    max_workers: 10
aws_advanced_configurations_json:
  TagSpecifications:
    - ResourceType: instance
      Tags:
        - Key: as-feature-idle-terminate-second
          Value: "60"

cloud: anyscale_v2_gcp_uswest1 # You may specify `cloud_id` instead
allowed_azs:
  - us-west1-a
  - us-west1-b
  - us-west1-c
head_node_type:
  name: head_node_type
  instance_type: n2-standard-32
worker_node_types:
  - name: gpu_worker_1
    instance_type: n1-standard-16-nvidia-t4-16gb-1
    min_workers: 0
    max_workers: 10
    use_spot: true
    fallback_to_ondemand: true
  - name: gpu_worker_2
    instance_type: n1-standard-16-nvidia-t4-16gb-4
    min_workers: 0
    max_workers: 10
    use_spot: true
    fallback_to_ondemand: true
gcp_advanced_configurations_json:
  instance_properties:
    labels:
      as-feature-idle-terminate-second: "60"

Adjustable downscaling

Usage

FAQ

Question: What is the default downscaling period?

Question: What is the minimum threshold recommended?

Usage​

FAQ​

Question: What is the default downscaling period?​

Question: What is the minimum threshold recommended?​

Usage

FAQ

Question: What is the default downscaling period?

Question: What is the minimum threshold recommended?