Job queues
A job queue enables sophisticated scheduling and execution algorithms for Anyscale Jobs. This feature improves resource utilization and reduces provisioning times by enabling multiple jobs to share a single cluster.
Anyscale supports flexible scheduling algorithms, including FIFO (first-in, first-out), LIFO (last-in, first-out), and priority-based scheduling.
Job processing
Anyscale job queues optimize resource utilization and throughput by using sophisticated scheduling to run multiple jobs on the same cluster.
- Submission: The typical Anyscale Job submission workflow adds the job to the specified queue.
- Scheduling: Based on the scheduling policy, Anyscale determines ordering of the jobs in the queue and picks jobs at the top of the queue for scheduling. Anyscale schedules no more than the specified
max-concurrency
jobs for running on a cluster at the same time. - Execution: Jobs run until completion, including retries up to the specified number of
max_retries
.
Anyscale provisions a cluster when you submit the first job in a queue, and continues running until there are no more jobs in the queue and it idles.
Create a job queue
Creating a job queue is similar to creating a standalone Anyscale Job. In your job.yaml
file, specify additional job queue configurations:
- CLI
- Python SDK
entrypoint: python hello_world.py
working_dir: "https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip"
name: JOB_NAME
compute_config: COMPUTE_CONFIG:1
image_uri: IMAGE:1
job_queue_config:
priority: 100 # Valid when `execution_mode: PRIORITY`; 0 is highest priority, 2^64 is lowest. Jobs of equal priority execute in arrival order.
job_queue_spec:
name: UNIQUE_JOB_QUEUE_NAME
execution_mode: PRIORITY # Scheduling algorithm; can also be FIFO (first-in, first-out) or LIFO (last-in, first-out).
max_concurrency: 5 # Max number of jobs that can run concurrently; limit 100.
idle_timeout_s: 3600 # Set to 0 to disable idle termination.
import anyscale
from anyscale.job.models import JobConfig, JobQueueConfig, JobQueueSpec, JobQueueExecutionMode
config = JobConfig(
name=JOB_NAME,
entrypoint="python main.py",
working_dir=".",
max_retries=5,
image_uri="anyscale/image/IMAGE:1",
compute_config="COMPUTE_CONFIG:1",
job_queue_config=JobQueueConfig(
# Valid when `execution_mode: PRIORITY`; 0 is highest priority, 2^64 is lowest.
# Jobs of equal priority execute in arrival order.
priority=100,
job_queue_spec=JobQueueSpec(
name=UNIQUE_JOB_QUEUE_NAME,
# Scheduling algorithm; can also be FIFO (first-in, first-out) or LIFO (last-in, first-out).
execution_mode=JobQueueExecutionMode.PRIORITY,
# Max number of jobs that can run concurrently; limit 100.
max_concurrency=5,
# Set to 0 to disable idle termination
idle_timeout_s=3600,
),
),
)
anyscale.job.submit(config)
Replace the following:
JOB_NAME
: (Optional) Name for the job.COMPUTE_CONFIG:1
: Name of an existing registered compute config with a version number. Omitting specific version would entail using the latest version.IMAGE:1
: URI of an existing image with a version number. Omitting specific version would entail using the latest version.UNIQUE_JOB_QUEUE_NAME
: Name of the job queue that later on will be used to add other jobs to this queue. Must be unique within a project.
See the full Anyscale Jobs API reference for complete JobQueueConfig
and JobQueueSpec
fields.
When submitting jobs, you must provide the corresponding job queue specification, job_queue_spec
, for the first job to create a managed job queue. For subsequent jobs, you can use the same job_queue_spec
with the same job queue name
to associate with the existing queue, provided the configuration doesn't change.
If you submit a job with the same job queue name
but a different job_queue_spec
configuration, the submission fails because names must be unique for active queues and Anyscale ties the name to the initial job_queue_spec
.
Add jobs to an existing queue
To submit jobs to an existing queue, specify the queue identifier in the job.yaml
:
- CLI
- Python SDK
entrypoint: python hello_world.py
working_dir: "https://github.com/anyscale/docs_examples/archive/refs/heads/main.zip"
job_queue_config:
priority: 100
target_job_queue_name: JOB_QUEUE_NAME
import anyscale
from anyscale.job.models import JobConfig, JobQueueConfig, JobQueueSpec, JobQueueExecutionMode
config = JobConfig(
name=JOB_NAME,
entrypoint="python main.py",
working_dir=".",
max_retries=5,
image_uri="anyscale/image/IMAGE:1",
compute_config="COMPUTE_CONFIG:1",
job_queue_config=JobQueueConfig(
priority=100,
target_job_queue_name=JOB_QUEUE_NAME,
),
)
anyscale.job.submit(config)
Replace JOB_QUEUE_NAME
with the name of the queue you're targeting.
Then, submit the job to add it to the queue:
anyscale job submit -f job.yaml