Job queues

A job queue enables sophisticated scheduling and execution algorithms for Anyscale Jobs. This feature improves resource utilization and reduces provisioning times by enabling multiple jobs to share a single cluster.

Anyscale supports flexible scheduling algorithms, including FIFO (first-in, first-out), LIFO (last-in, first-out), and priority-based scheduling.

Job processing

Anyscale job queues optimize resource utilization and throughput by using sophisticated scheduling to run multiple jobs on the same cluster.

1. Submission: The typical Anyscale Job submission workflow adds the job to the specified queue.
2. Scheduling: Based on the scheduling policy, Anyscale determines ordering of the jobs in the queue and picks jobs at the top of the queue for scheduling. Anyscale schedules no more than the specified max-concurrency jobs for running on a cluster at the same time.
3. Execution: Jobs run until completion, including retries up to the specified number of max_retries.

Anyscale provisions a cluster when you submit the first job in a queue, and continues running until there are no more jobs in the queue and it idles.

Create a job queue

Creating a job queue is similar to creating a standalone Anyscale Job. In your job.yaml file, specify additional job queue configurations:

CLI

entrypoint: python hello_world.py
working_dir: "https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip"

name: JOB_NAME

# Use compute_config and image_uri to create the cluster.
# compute_config and image_uri must be the same for all jobs in a given queue.
compute_config: COMPUTE_CONFIG:1
image_uri: IMAGE:1

job_queue_config:
  priority: 100 # Valid when `execution_mode: PRIORITY`; 0 is highest priority, 2^64 is lowest. Jobs of equal priority execute in arrival order.

  job_queue_spec:
    name: JOB_QUEUE_NAME
    execution_mode: PRIORITY # Scheduling algorithm; can also be FIFO (first-in, first-out) or LIFO (last-in, first-out).
    max_concurrency: 5 # Max number of jobs that can run concurrently; limit 100.
    idle_timeout_s: 3600 # Set to 0 to disable idle termination.

Python SDK

import anyscale
from anyscale.job.models import JobConfig, JobQueueConfig, JobQueueSpec, JobQueueExecutionMode

config = JobConfig(
  name=JOB_NAME,
  entrypoint="python main.py",
  working_dir=".",
  max_retries=5,
  image_uri="anyscale/image/IMAGE:1",
  compute_config="COMPUTE_CONFIG:1",
  job_queue_config=JobQueueConfig(
    # Valid when `execution_mode: PRIORITY`; 0 is highest priority, 2^64 is lowest.
    # Jobs of equal priority execute in arrival order.
    priority=100,
    job_queue_spec=JobQueueSpec(
      name=JOB_QUEUE_NAME,
      # Scheduling algorithm; can also be FIFO (first-in, first-out) or LIFO (last-in, first-out).
      execution_mode=JobQueueExecutionMode.PRIORITY,
      # Max number of jobs that can run concurrently; limit 100.
      max_concurrency=5,
      # Set to 0 to disable idle termination
      idle_timeout_s=3600,
    ),
  ),
)

anyscale.job.submit(config)

Replace the following:

• JOB_NAME: (Optional) Name for the job.
• COMPUTE_CONFIG:1: Name of an existing registered compute config with a version number. Omitting specific version would entail using the latest version.
• IMAGE:1: URI of an existing image with a version number. Omitting specific version would entail using the latest version.
• JOB_QUEUE_NAME: Name of the job queue that Anyscale uses to add other jobs to this queue.

See the API reference for JobQueueConfig and JobQueueSpec.

important

If this is the first job for a job queue, Anyscale creates a new cluster based on job_queue_spec, compute_config, and image_uri. For subsequent jobs, use the same config to associate with the existing queue.

The submission will fail if you submit a job with the same job queue name but a different job_queue_spec, compute_config, or image_uri.

note

If you don't specify compute_config or image_uri, Anyscale uses the cloud default or the current workspace ones.

Add jobs to an existing queue

You can reuse the above job_queue_spec, compute_config, and image_uri to submit jobs to an existing queue.

Alternatively, you only need to specify target_job_queue_name in the job.yaml:

CLI

entrypoint: python hello_world.py
working_dir: "https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip"

job_queue_config:
  priority: 100
  target_job_queue_name: JOB_QUEUE_NAME

Python SDK

import anyscale
from anyscale.job.models import JobConfig, JobQueueConfig, JobQueueSpec, JobQueueExecutionMode

config = JobConfig(
  name=JOB_NAME,
  entrypoint="python main.py",
  working_dir=".",
  max_retries=5,
  image_uri="anyscale/image/IMAGE:1",
  compute_config="COMPUTE_CONFIG:1",
  job_queue_config=JobQueueConfig(
    priority=100,
    target_job_queue_name=JOB_QUEUE_NAME,
  ),
)

anyscale.job.submit(config)

Replace JOB_QUEUE_NAME with the name of the queue you're targeting.

Then, submit the job to add it to the queue:

anyscale job submit -f job.yaml

Terminate all jobs in a queue

To terminate all running jobs in the queue, use the Terminate running jobs button on the upper right corner of the Job queue page. Note that Anyscale doesn't terminate pending jobs.

Terminate a specific job in a queue

If the job is still Pending, you can terminate it from the Job page or by using the CLI:

anyscale job terminate --id 'prodjob_...'

If the job is Running, you need to terminate it in the Anyscale terminal:

• Go to Job page, click the Ray dashboard tab, click the Jobs tab. Find and copy the Submission ID for the job you want to terminate.

• Open the Terminal tab and run ray job stop 'raysubmit_...'.