Job queues

A job queue enables sophisticated scheduling and execution algorithms for Anyscale Jobs. This feature improves resource utilization and reduces provisioning times by enabling multiple jobs to share a single cluster.

Anyscale supports flexible scheduling algorithms, including FIFO (first-in, first-out), LIFO (last-in, first-out), and priority-based scheduling.

Job processing

Anyscale job queues optimize resource utilization and throughput by using sophisticated scheduling to run multiple jobs on the same cluster.

1. Submission: The typical Anyscale Job submission workflow adds the job to the specified queue.
2. Scheduling: Based on the scheduling policy, Anyscale determines ordering of the jobs in the queue and picks jobs at the top of the queue for scheduling. Anyscale schedules no more than the specified max-concurrency jobs for running on a cluster at the same time.
3. Execution: Jobs run until completion, including retries up to the specified number of max_retries.

Anyscale provisions a cluster when you submit the first job in a queue, and continues running until there are no more jobs in the queue and it idles.

Create a job queue

Creating a job queue is similar to creating a standalone Anyscale Job. In your job.yaml file, specify additional job queue configurations:

CLI

entrypoint: python hello_world.py
working_dir: "https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip"

name: JOB_NAME
compute_config: COMPUTE_CONFIG:1
image_uri: IMAGE:1

job_queue_config:
  priority: 100 # Valid when `execution_mode: PRIORITY`; 0 is highest priority, 2^64 is lowest. Jobs of equal priority execute in arrival order.

  job_queue_spec:
    name: UNIQUE_JOB_QUEUE_NAME
    execution_mode: PRIORITY # Scheduling algorithm; can also be FIFO (first-in, first-out) or LIFO (last-in, first-out).
    max_concurrency: 5 # Max number of jobs that can run concurrently; limit 100.
    idle_timeout_s: 3600 # Set to 0 to disable idle termination.

Python SDK

import anyscale
from anyscale.job.models import JobConfig, JobQueueConfig, JobQueueSpec, JobQueueExecutionMode

config = JobConfig(
  name=JOB_NAME,
  entrypoint="python main.py",
  working_dir=".",
  max_retries=5,
  image_uri="anyscale/image/IMAGE:1",
  compute_config="COMPUTE_CONFIG:1",
  job_queue_config=JobQueueConfig(
    # Valid when `execution_mode: PRIORITY`; 0 is highest priority, 2^64 is lowest.
    # Jobs of equal priority execute in arrival order.
    priority=100,
    job_queue_spec=JobQueueSpec(
      name=UNIQUE_JOB_QUEUE_NAME,
      # Scheduling algorithm; can also be FIFO (first-in, first-out) or LIFO (last-in, first-out).
      execution_mode=JobQueueExecutionMode.PRIORITY,
      # Max number of jobs that can run concurrently; limit 100.
      max_concurrency=5,
      # Set to 0 to disable idle termination
      idle_timeout_s=3600,
    ),
  ),
)

anyscale.job.submit(config)

Replace the following:

• JOB_NAME: (Optional) Name for the job.
• COMPUTE_CONFIG:1: Name of an existing registered compute config with a version number. Omitting specific version would entail using the latest version.
• IMAGE:1: URI of an existing image with a version number. Omitting specific version would entail using the latest version.
• UNIQUE_JOB_QUEUE_NAME: Name of the job queue that later on will be used to add other jobs to this queue. Must be unique within a project.

See the full Anyscale Jobs API reference for complete JobQueueConfig and JobQueueSpec fields.

When submitting jobs, you must provide the corresponding job queue specification, job_queue_spec, for the first job to create a managed job queue. For subsequent jobs, you can use the same job_queue_spec with the same job queue name to associate with the existing queue, provided the configuration doesn't change.

note

If you submit a job with the same job queue name but a different job_queue_spec configuration, the submission fails because names must be unique for active queues and Anyscale ties the name to the initial job_queue_spec.

Add jobs to an existing queue

To submit jobs to an existing queue, specify the queue identifier in the job.yaml:

CLI

entrypoint: python hello_world.py
working_dir: "https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip"

job_queue_config:
  priority: 100
  target_job_queue_name: JOB_QUEUE_NAME

Python SDK

import anyscale
from anyscale.job.models import JobConfig, JobQueueConfig, JobQueueSpec, JobQueueExecutionMode

config = JobConfig(
  name=JOB_NAME,
  entrypoint="python main.py",
  working_dir=".",
  max_retries=5,
  image_uri="anyscale/image/IMAGE:1",
  compute_config="COMPUTE_CONFIG:1",
  job_queue_config=JobQueueConfig(
    priority=100,
    target_job_queue_name=JOB_QUEUE_NAME,
  ),
)

anyscale.job.submit(config)

Replace JOB_QUEUE_NAME with the name of the queue you're targeting.

Then, submit the job to add it to the queue:

anyscale job submit -f job.yaml