Skip to main content

Job API Reference (0.26.92)

View Markdown

Job API Reference (0.26.92)

note

This is archived documentation for version 0.26.92. For the current documentation, see current documentation.

Customer-hosted cloud features​

note

Some features are only available on customer-hosted clouds. Reach out to support@anyscale.com for info.

Job CLI​

anyscale job submit​

Usage

anyscale job submit [OPTIONS] [ENTRYPOINT]...

Submit a job.

The job config can be specified in one of the following ways:

  • Job config file can be specified as a single positional argument. E.g. anyscale job submit config.yaml.

  • Job config can also be specified with command-line arguments. In this case, the entrypoint should be specified as the positional arguments starting with --. Other arguments can be specified with command-line flags. E.g.

    • anyscale job submit -- python main.py: submit a job with the entrypoint python main.py.

    • anyscale job submit --name my-job -- python main.py: submit a job with the name my-job and the entrypoint python main.py.

  • [Experimental] If you want to specify a config file and override some arguments with the commmand-line flags, use the --config-file flag. E.g.

    • anyscale job submit --config-file config.yaml: submit a job with the config in config.yaml.

    • anyscale job submit --config-file config.yaml -- python main.py: submit a job with the config in config.yaml and override the entrypoint with python main.py.

Either containerfile or image-uri should be used, specifying both will result in an error.

By default, this command submits the job asynchronously and exits. To wait for the job to complete, use the --wait flag.

Options

  • -n/--name: Name of the job.
  • -w/--wait: Block this CLI command and print logs until the job finishes.
  • --config-file/-f: Path to a YAML config file to use for this job. Command-line flags will overwrite values read from the file.
  • --compute-config: Named compute configuration to use for the job. This defaults to the compute configuration of the workspace.
  • --image-uri: Container image to use for this job. When running in a workspace, this defaults to the image of the workspace.
  • --registry-login-secret: Name or identifier of the secret containing credentials to authenticate to the docker registry hosting the image. This can only be used when 'image_uri' is specified and the image is not hosted on Anyscale.
  • --containerfile: Path to a containerfile to build the image to use for the job.
  • --env: Environment variables to set for the job. The format is 'key=value'. This argument can be specified multiple times. When the same key is also specified in the config file, the value from the command-line flag will overwrite the value from the config file.
  • --working-dir: Path to a local directory or a remote URI to a .zip file (S3, GS, HTTP) that will be the working directory for the job. The files in the directory will be automatically uploaded to cloud storage. When running in a workspace, this defaults to the current working directory.
  • -e/--exclude: File pattern to exclude when uploading local directories. This argument can be specified multiple times and the patterns will be appended to the 'excludes' list in the config file (if any).
  • -r/--requirements: Path to a requirements.txt file containing dependencies for the job. Anyscale installs these dependencies on top of the image. If you run a job from a workspace, the default is to use the workspace dependencies, but specifying this option overrides them.
  • --py-module: Python modules to be available for import in the Ray workers. Each entry must be a path to a local directory.
  • --tag: Tag in key=value (or key:value) format. Repeat to add multiple.
  • --cloud: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • --ray-version: The Ray version (X.Y.Z) to the image specified by --image-uri. This is only used when --image-uri is provided. If you don't specify a Ray version, Anyscale defaults to the latest Ray version available at the time of the Anyscale CLI/SDK release.
  • --max-retries: Maximum number of retries to attempt before failing the entire job.
  • --timeout-s/--timeout/-t: The timeout in seconds for each job run. Set to None for no limit to be set.
  • --connection: [Beta] Third-party connection to associate with the job (e.g., Databricks). Format: type=TYPE,name=NAME. Example: --connection type=databricks,name=my-conn. Can be repeated for multiple connections. This feature is in beta preview. Contact Anyscale support to request enablement.

Examples​

# Submit a job without a yaml file, this will inherit the environment from the workspace.
$ anyscale job submit --name my-job --wait -- python main.py
Output
(anyscale +1.0s) Submitting job with config JobConfig(name='my-job', image_uri=None, compute_config=None, env_vars=None, py_modules=None, cloud=None, project=None, ray_version=None, job_queue_config=None).
(anyscale +1.7s) Uploading local dir '.' to cloud storage.
(anyscale +2.6s) Including workspace-managed pip dependencies.
(anyscale +3.2s) Job 'my-job' submitted, ID: 'prodjob_6ntzknwk1i9b1uw1zk1gp9dbhe'.
(anyscale +3.2s) View the job in the UI: https://console.anyscale.com/jobs/prodjob_6ntzknwk1i9b1uw1zk1gp9dbhe
(anyscale +3.2s) Waiting for the job to run. Interrupting this command will not cancel the job.
(anyscale +3.5s) Waiting for job 'prodjob_6ntzknwk1i9b1uw1zk1gp9dbhe' to reach target state SUCCEEDED, currently in state: STARTING
(anyscale +1m19.7s) Job 'prodjob_6ntzknwk1i9b1uw1zk1gp9dbhe' transitioned from STARTING to SUCCEEDED
(anyscale +1m19.7s) Job 'prodjob_6ntzknwk1i9b1uw1zk1gp9dbhe' reached target state, exiting

# Submit a job specified by a yaml file, this will run a job with the specified settings.
# See the API reference for a list of all available fields.
$ cat job.yaml
name: my-job
entrypoint: python main.py
image_uri: anyscale/ray:2.44.1-slim-py312-cu128 # (Optional) Exclusive with `containerfile`.
# compute_config: # (Optional) An inline dictionary can also be provided.
#   head_node:
#     instance_type: m5.8xlarge
working_dir: . # (Optional) Use current working directory "." as the working_dir. Can be any local path or remote .zip file in cloud storage.
requirements: # (Optional) List of requirements files to install. Can also be a path to a requirements.txt.
  - emoji
max_retries: 0 # (Optional) Maximum number of times the job will be retried before being marked failed. Defaults to `1`.
$ anyscale job submit -f job.yaml
Output
(anyscale +1.0s) Submitting job with config JobConfig(name='my-job', image_uri='anyscale/ray:2.44.1-slim-py312-cu128', compute_config=None, env_vars=None, py_modules=None, py_executable=None, cloud=None, project=None, ray_version=None, job_queue_config=None).
(anyscale +4.2s) Uploading local dir '.' to cloud storage.
(anyscale +5.4s) Job 'my-job' submitted, ID: 'prodjob_ugk21r7pt6sbsxfuxn1an82x3e'.
(anyscale +5.4s) View the job in the UI: https://console.anyscale.com/jobs/prodjob_ugk21r7pt6sbsxfuxn1an82x3e
(anyscale +5.4s) Use `--wait` to wait for the job to run and stream logs.

anyscale job status​

Usage

anyscale job status [OPTIONS]

Query the status of a job.

To specify the job by name, use the --name flag. To specify the job by id, use the --id flag. Either name or id should be used, specifying both will result in an error.

If job is specified by name and there are multiple jobs with the specified name, the most recently created job status will be returned.

Options

  • --id/--job-id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --cloud: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • --json/-j: Output the status in a structured JSON format.
  • --verbose/-v: Include verbose details in the status.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job status -n my-job
id: prodjob_6ntzknwk1i9b1uw1zk1gp9dbhe
name: my-job
state: STARTING
runs:
- name: raysubmit_ynxBVGT1SmzndiXL
  state: SUCCEEDED

anyscale job terminate​

Usage

anyscale job terminate [OPTIONS]

Terminate a job.

To specify the job by name, use the --name flag. To specify the job by id, use the --id flag. Either name or id should be used, specifying both will result in an error.

If job is specified by name and there are multiple jobs with the specified name, the most recently created job status will be terminated.

Options

  • --id/--job-id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --cloud: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job terminate -n my-job
(anyscale +5.4s) Marked job 'my-job' for termination
(anyscale +5.4s) Query the status of the job with `anyscale job status --name my-job`.

anyscale job archive​

Usage

anyscale job archive [OPTIONS]

Archive a job.

To specify the job by name, use the --name flag. To specify the job by id, use the --id flag. Either name or id should be used, specifying both will result in an error.

If job is specified by name and there are multiple jobs with the specified name, the most recently created job status will be archived.

Options

  • --id/--job-id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --cloud: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job archive -n my-job
(anyscale +8.5s) Job prodjob_vzq2pvkzyz3c1jw55kl76h4dk1 is successfully archived.

anyscale job delete​

Usage

anyscale job delete [OPTIONS]

Delete a job and all associated job runs.

The job must be in a terminal state (SUCCEEDED, FAILED) before deletion. This action permanently removes the job and cannot be undone.

To specify the job by name, use the --name flag. To specify by ID, use --id. Either name or ID must be provided, but not both.

Options

  • --id/--job-id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --cloud: The Anyscale Cloud of this workload. If not provided, the organization default will be used.
  • --project: Named project to use for the job. If not provided, the default project will be used.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job delete -n my-job
(anyscale +3.2s) Job 'my-job' (ID: prodjob_abc123) has been deleted.

anyscale job logs​

Usage

anyscale job logs [OPTIONS]

Print the logs of a job.

By default from the latest job attempt.

Options

  • --id/--job-id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --run: Name of the job run.
  • --cloud: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • --head: Used with --max-lines to get max-lines lines from the head of the log.
  • --tail: Used with --max-lines to get max-lines lines from the tail of the log.
  • --max-lines: Used with --head or --tail to limit the number of lines output.
  • --follow/-f: Whether to follow the log.
  • --all-attempts: Listing logs from all attempts no longer supported, instead list jobs by run name.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job logs -n my-job
2024-08-23 20:31:10,913 INFO job_manager.py:531 -- Runtime env is setting up.
hello world

anyscale job wait​

Usage

anyscale job wait [OPTIONS]

Wait for a job to enter a specific state (default: SUCCEEDED).

To specify the job by name, use the --name flag. To specify the job by id, use the --id flag.

If the job reaches the target state, the command will exit successfully.

If the job reaches a terminal state other than the target state, the command will exit with an error.

If the command reaches the timeout, the command will exit with an error but job execution will continue.

Options

  • --id/--job-id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --cloud: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • --state/-s: The state to wait for this job to enter
  • --timeout-s/--timeout/-t: The timeout in seconds after which this command will exit.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job wait -n my-job
(anyscale +5.7s) Waiting for job 'my-job' to reach target state SUCCEEDED, currently in state: STARTING
(anyscale +1m34.2s) Job 'my-job' transitioned from STARTING to SUCCEEDED
(anyscale +1m34.2s) Job 'my-job' reached target state, exiting

anyscale job list​

Usage

anyscale job list [OPTIONS]

Display information about existing jobs.

Options

  • --v2: [RECOMMENDED] Enable extended filtering options. Needs migration to match return values.
  • --name/-n: Filter by job name.
  • --id/--job-id: Filter by job id.
  • --project-id: Filter by project ID. Use --project instead.
  • --project: Filter by project name. Only with --v2 flag.
  • --cloud: Filter by cloud name. Only with --v2 flag.
  • --include-all-users: Include jobs not created by current user.
  • --include-archived: List archived jobs as well as unarchived jobs.If not provided, defaults to listing only unarchived jobs.
  • --tag: This option can be repeated to filter by multiple tags. Tags with the same key are ORed, whereas tags with different keys are ANDed. Example: --tag team:mlops --tag team:infra --tag env:prod. Filters with team: (mlops OR infra) AND env:prod.
  • --max-items: Max total items (only with --no-interactive).
  • --state/-s: Filter jobs by state. Accepts one or more states. With --v2, use: STARTING, RUNNING, SUCCEEDED, FAILED. Without --v2, use: PENDING, AWAITING_CLUSTER_START, UPDATING, RUNNING, SUCCESS, ERRORED, TERMINATED, CLEANING_UP, BROKEN, OUT_OF_RETRIES, RESTARTING.
  • --page-size: Number of items per page (1-50). Defaults to 10. Only with --v2.
  • --sort: Sort by field. Prefix with '-' for descending order. Defaults to -created_at. Allowed: STATUS, CREATED_AT, LAST_UPDATED_AT, LAST_STATUS_UPDATED_AT, ENDED_AT, NAME, ID. Only with --v2.
  • --created-at-from: Filter for jobs created at or after this time (ISO 8601, e.g. 2024-01-01T00:00:00Z). Only with --v2.
  • --created-at-to: Filter for jobs created at or before this time (ISO 8601, e.g. 2024-12-31T23:59:59Z). Only with --v2.
  • --updated-at-from: Filter for jobs last updated at or after this time (ISO 8601, e.g. 2024-01-01T00:00:00Z). Only with --v2.
  • --updated-at-to: Filter for jobs last updated at or before this time (ISO 8601, e.g. 2024-12-31T23:59:59Z). Only with --v2.
  • --status-updated-at-from: Filter for jobs whose status was last updated at or after this time (ISO 8601, e.g. 2024-01-01T00:00:00Z). Only with --v2.
  • --status-updated-at-to: Filter for jobs whose status was last updated at or before this time (ISO 8601, e.g. 2024-12-31T23:59:59Z). Only with --v2.
  • --json/-j: Output results as JSON. Only with --v2.
  • --interactive/--no-interactive: Enable interactive pagination. Only with --v2.

Examples​

$ anyscale job list -n my-job
Output
View your Jobs in the UI at https://console.anyscale.com/jobs
JOBS:
NAME    ID                                    COST  PROJECT NAME    CLUSTER NAME                                    CURRENT STATE           CREATOR           ENTRYPOINT
my-job  prodjob_s9x4uzc5jnkt5z53g4tujb3y2e       0  default         cluster_for_prodjob_s9x4uzc5jnkt5z53g4tujb3y2e  SUCCESS                 doc@anyscale.com  python main.py

anyscale job tags add​

Usage

anyscale job tags add [OPTIONS]

Add or update tags on a job.

Options

  • --id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --tag: Tag in key=value (or key:value) format. Repeat to add multiple.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job tags add --name my-job --tag owner=alice --tag env=staging

anyscale job tags remove​

Usage

anyscale job tags remove [OPTIONS]

Remove tags by key from a job.

Options

  • --id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --key: Tag key to remove. Repeatable.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job tags remove --name my-job --key owner --key env

anyscale job tags list​

Usage

anyscale job tags list [OPTIONS]

List tags for a job.

Options

  • --id: Unique ID of the job.
  • --name/-n: Name of the job.
  • --json: JSON output.
  • --include-archived: Include archived jobs when searching by name. Ignored when using --id.

Examples​

$ anyscale job tags list --name my-job

Job SDK​

anyscale.job.submit​

Submit a job.

Returns the id of the submitted job.

Arguments

  • config (JobConfig): The config options defining the job.

Returns: str

Examples​

import anyscale
from anyscale.job.models import JobConfig, ConnectionConfig, ConnectionType

# Basic job submission
anyscale.job.submit(
    JobConfig(
        name="my-job",
        entrypoint="python main.py",
        working_dir=".",
    ),
)

# Job with Databricks connection for credential injection
anyscale.job.submit(
    JobConfig(
        name="my-job-with-databricks",
        entrypoint="python main.py",
        working_dir=".",
        connections=[
            ConnectionConfig(
                type=ConnectionType.DATABRICKS,
                name="my-databricks-connection",
            )
        ],
    ),
)

anyscale.job.status​

Get the status of a job.

Arguments

  • name (str | None) = None: Name of the job.
  • id (str | None) = None: Unique ID of the job
  • cloud (str | None) = None: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using id.

Returns: JobStatus

Examples​

import anyscale
from anyscale.job.models import JobStatus

status: JobStatus = anyscale.job.status(name="my-job")

anyscale.job.terminate​

Terminate a job.

This command is asynchronous, so it always returns immediately.

Returns the id of the terminated job.

Arguments

  • name (str | None) = None: Name of the job.
  • id (str | None) = None: Unique ID of the job
  • cloud (str | None) = None: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using id.

Returns: str

Examples​

import anyscale

anyscale.job.terminate(name="my-job")

anyscale.job.archive​

Archive a job.

This command is asynchronous, so it always returns immediately.

Returns the id of the archived job.

Arguments

  • name (str | None) = None: Name of the job.
  • id (str | None) = None: Unique ID of the job
  • cloud (str | None) = None: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the job . If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using id.

Returns: str

Examples​

import anyscale

anyscale.job.archive(name="my-job")

anyscale.job.delete​

Delete a job and all associated job runs.

The job must be in a terminal state (SUCCEEDED, FAILED). This action permanently removes the job and cannot be undone.

Returns the id of the deleted job.

Arguments

  • name (str | None) = None: Name of the job.
  • id (str | None) = None: Unique ID of the job
  • cloud (str | None) = None: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using id.

Returns: str

Examples​

import anyscale

anyscale.job.delete(name="my-job")

anyscale.job.get_logs​

Query the jobs for a job run.

Arguments

  • id (str | None) = None: Unique ID of the job
  • name (str | None) = None: Name of the job
  • cloud (str | None) = None: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • run (str | None) = None: The name of the run to query. Names can be found in the JobStatus. If not provided, the last job run will be used.
  • mode (str | JobLogMode) = TAIL: The mode of log fetching to be used. Supported modes can be found in JobLogMode. If not provided, JobLogMode.TAIL will be used.
  • max_lines (int | None) = None: The number of log lines to be fetched. If not provided, the complete log will be fetched.
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using id.

Returns: str

Examples​

import anyscale

anyscale.job.get_logs(name="my-job", run="job-run-name")

anyscale.job.wait​

"Wait for a job to enter a specific state.

Arguments

  • name (str | None) = None: Name of the job.
  • id (str | None) = None: Unique ID of the job
  • cloud (str | None) = None: The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the job. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • state (JobState | str) = SUCCEEDED: Target state of the job
  • timeout_s (float) = 1800: Number of seconds to wait before timing out, this timeout will not affect job execution
  • follow (bool) = False: Whether to follow the logs of the job. If True, the logs will be streamed to the console.
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using id.

Examples​

import anyscale

anyscale.job.wait(name="my-job", timeout_s=180)

anyscale.job.list​

List jobs with filtering and pagination.

Returns a ResultIterator that lazily fetches pages of jobs.

Arguments

  • name (str | None) = None: Filter by job name.
  • job_id (str | None) = None: Fetch a specific job by ID.
  • project (str | None) = None: Filter by project name.
  • cloud (str | None) = None: Filter by cloud name.
  • include_all_users (bool) = False: Include jobs from all users.
  • include_archived (bool) = False: Include archived jobs.
  • state_filter (List[JobState | str] | None) = None: Filter by job states (list of JobState or str).
  • tags_filter (Dict[str, List[str]] | None) = None: Filter by tags (dict of key to list of values).
  • page_size (int | None) = None: Number of items per page.
  • max_items (int | None) = None: Maximum total items to return.
  • sort_field (str | None) = None: Field to sort by (CREATED_AT, NAME, STATUS, etc.).
  • sort_order (str | None) = None: Sort order (ASC or DESC).
  • created_at_from (str | None) = None: Filter for jobs created at or after this time (e.g. 2026-01-01T00:00:00Z).
  • created_at_to (str | None) = None: Filter for jobs created at or before this time (e.g. 2026-12-31T23:59:59Z).
  • updated_at_from (str | None) = None: Filter for jobs updated at or after this time (e.g. 2026-01-01T00:00:00Z).
  • updated_at_to (str | None) = None: Filter for jobs updated at or before this time (e.g. 2026-12-31T23:59:59Z).
  • status_updated_at_from (str | None) = None: Filter for jobs whose status was last updated at or after this time (e.g. 2026-01-01T00:00:00Z).
  • status_updated_at_to (str | None) = None: Filter for jobs whose status was last updated at or before this time (e.g. 2026-12-31T23:59:59Z).

Returns: ResultIterator[JobStatus]

Examples​

import anyscale
from anyscale.job.models import JobStatus

# List all jobs
for job in anyscale.job.list(max_items=10):
    print(f"{job.name}: {job.state}")

# Filter by project
jobs = list(anyscale.job.list(project="my-project"))

anyscale.job.add_tags​

Upsert (add/update) tag key/value pairs for a job.

Arguments

  • job_id (str | None) = None: ID of the job. Provide either job_id or name.
  • name (str | None) = None: Name of the job. Provide either job_id or name.
  • cloud (str | None) = None: Cloud name (used when resolving by name).
  • project (str | None) = None: Project name (used when resolving by name).
  • tags (Dict[str, str]): Key/value tags to upsert as a map {key: value}.
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using job_id.

Examples​

import anyscale

anyscale.job.add_tags(id="job_123", tags={"team": "mlops", "env": "prod"})

anyscale.job.remove_tags​

Remove tags by key from a job.

Arguments

  • job_id (str | None) = None: ID of the job. Provide either job_id or name.
  • name (str | None) = None: Name of the job. Provide either job_id or name.
  • cloud (str | None) = None: Cloud name (used when resolving by name).
  • project (str | None) = None: Project name (used when resolving by name).
  • keys (List[str]): List of tag keys to remove.
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using job_id.

Examples​

import anyscale

anyscale.job.remove_tags(id="job_123", keys=["team", "env"])

anyscale.job.list_tags​

List tags for a job as a key/value mapping.

Arguments

  • job_id (str | None) = None: ID of the job. Provide either job_id or name.
  • name (str | None) = None: Name of the job. Provide either job_id or name.
  • cloud (str | None) = None: Cloud name (used when resolving by name).
  • project (str | None) = None: Project name (used when resolving by name).
  • include_archived (bool) = False: Include archived jobs when searching by name. Ignored when using job_id.

Returns: Dict[str, str]

Examples​

import anyscale

tags: dict[str, str] = anyscale.job.list_tags(name="my-job")

Job Models​

JobQueueSpec​

Options defining a job queue.

When the first job with a given job queue spec is submitted, the job queue will be created. Subsequent jobs with the same job queue spec will reuse the queue instead of creating another one.

Jobs can also target an existing queue using the name parameter.

Fields​

  • idle_timeout_s (int): Timeout that the job queue cluster will be kept running while no jobs are running.
  • name (str | None): Name of the job queue that can be used to target it when submitting future jobs. The name of a job queue must be unique within a project.
  • execution_mode (JobQueueExecutionMode): Execution mode of the jobs submitted into the queue (one of: FIFO,LIFO,PRIORITY
  • compute_config (str | None): The name of an existing compute config that will be used to create the job queue cluster. If not specified, the compute config of the associated job will be used.
  • max_concurrency (int): Max number of jobs that can run concurrently.Defaults to 1, meaning only one job can run at a given time.
  • auto_termination_threshold_job_count (int | None): Maximum number of jobs the cluster can run before it becomes eligible for termination

Python Methods​

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

job_queue_spec:
    # Unique name that can be used to target this queue by other jobs.
    name: my-job-queue
    execution_mode: FIFO
    # Name of a compute config that will be used to create a cluster to execute jobs in this queue.
    # Must match the compute config of the job if specified.
    compute_config: my-compute-config:1
    max_concurrency: 5
    idle_timeout_s: 3600

JobQueueConfig​

Configuration options for a job related to using a job queue for scheduling and execution.

Fields​

  • priority (int | None): Job's relative priority (only relevant for Job Queues of type PRIORITY). Valid values range from 0 (highest) to +inf (lowest). Default value is None
  • target_job_queue_name (str | None): The name of an existing job queue to schedule this job in. If this is provided, job_queue_spec cannot be.
  • job_queue_spec (JobQueueSpec | None): Configuration options defining a job queue to be created for the job if needed. If this is provided, target_job_queue_name cannot be.

Python Methods​

def __init__(self, **fields) -> JobQueueConfig
    """Construct a model with the provided field values set."""

def options(self, **fields) -> JobQueueConfig
    """Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

# An example configuration that creates a job queue if one does not exist with the provided options.
job_queue_config:
    # Priority of the job (only relevant if the execution_mode is "PRIORITY").
    priority: 100
    # Specification of the target Job Queue (will be created if does not exist)
    job_queue_spec:
        name: my-job-queue
        compute_config: my-compute-config:1
        idle_timeout_s: 3600

# An example config that targets an existing job queue by name.
job_queue_config:
    # Priority of the job (only relevant if the execution_mode is "PRIORITY").
    priority: 100
    # Name for the job queue this job should be added to (specified in `JobQueueSpec.name`
    # on queue's creation)
    target_job_queue_name: my-new-queue

JobQueueExecutionMode​

Execution mode for job queues.

Values​

  • FIFO: Executes jobs in chronological order ('first in, first out')
  • LIFO: Executes jobs in reversed chronological order ('last in, first out')
  • PRIORITY: Executes jobs in the order induced by ordering their priorities in ascending order, with 0 being the highest priority

JobConfig​

Configuration options for a job.

Fields​

  • name (str | None): Name of the job. Multiple jobs can be submitted with the same name.
  • image_uri (str | None): URI of an existing image. Exclusive with containerfile.
  • containerfile (str | None): The file path to a containerfile that will be built into an image before running the workload. Exclusive with image_uri.
  • compute_config (ComputeConfig | MultiResourceComputeConfig | Dict | str | None): The name of an existing registered compute config or an inlined ComputeConfig or MultiResourceComputeConfig object.
  • working_dir (str | None): Directory that will be used as the working directory for the application. If a local directory is provided, it will be uploaded to cloud storage automatically. When running inside a workspace, this defaults to the current working directory ('.').
  • excludes (List[str] | None): A list of file path globs that will be excluded when uploading local files for working_dir.
  • requirements (str | List[str] | None): A list of pip requirements or a path to a requirements.txt file for the workload. Anyscale installs these dependencies on top of the image. If you run a workload from a workspace, the default is to use the workspace dependencies, but specifying this option overrides them.
  • env_vars (Dict[str, str] | None): A dictionary of environment variables that will be set for the workload.
  • py_modules (List[str] | None): A list of local directories or remote URIs that will be uploaded and added to the Python path.
  • py_executable (str | None): Specifies the executable used for running the Ray workers. It can include arguments as well.
  • cloud (str | None): The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None): The project for the workload. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • registry_login_secret (str | None): A name or identifier of the secret containing credentials to authenticate to the docker registry hosting the image. This can only be used when 'image_uri' is specified and the image is not hosted on Anyscale.
  • ray_version (str | None): The Ray version (X.Y.Z) specified for this image specified by either an image URI or a containerfile. If you don't specify a Ray version, Anyscale defaults to the latest Ray version available at the time of the Anyscale CLI/SDK release.
  • connections (List[ConnectionConfig | Dict] | None): Connections to third-party integrations (e.g., Databricks) to associate with the workload. This feature is in beta preview. Contact Anyscale support to request enablement.
  • entrypoint (str): Command that will be run to execute the job, e.g., python main.py.
  • max_retries (int | None): Maximum number of times the job will be retried before being marked failed. Defaults to 1.
  • job_queue_config (JobQueueConfig | None): Job's configuration related to scheduling & execution using job queues
  • timeout_s (int | None): The timeout in seconds for each job run. Set to None for no limit to be set.
  • tags (Dict[str, str] | None): Tags to associate with the job.

Python Methods​

def __init__(self, **fields) -> JobConfig
    """Construct a model with the provided field values set."""

def options(self, **fields) -> JobConfig
    """Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

name: my-job
entrypoint: python main.py
image_uri: anyscale/image/my-image:1 # (Optional) Exclusive with `containerfile`.
containerfile: /path/to/Dockerfile # (Optional) Exclusive with `image_uri`.
compute_config: my-compute-config:1 # (Optional) An inline dictionary can also be provided.
working_dir: /path/to/working_dir # (Optional) Defaults to `.`.
excludes: # (Optional) List of files to exclude from being packaged up for the job.
    - .git
    - .env
    - .DS_Store
    - __pycache__
requirements: # (Optional) List of requirements files to install. Can also be a path to a requirements.txt.
    - emoji==1.2.0
    - numpy==1.19.5
env_vars: # (Optional) Dictionary of environment variables to set in the job.
    MY_ENV_VAR: my_value
    ANOTHER_ENV_VAR: another_value
py_modules: # (Optional) A list of local directories or remote URIs that will be added to the Python path.
    - /path/to/my_module
    - s3://my_bucket/my_module
cloud: anyscale-prod # (Optional) The name of the Anyscale Cloud.
project: my-project # (Optional) The name of the Anyscale Project.
max_retries: 3 # (Optional) Maximum number of times the job will be retried before being marked failed. Defaults to `1`.
tags:
    team: mlops
    purpose: training
connections: # (Optional) List of third-party connections for credential injection.
    - type: databricks
      name: my-databricks-connection

ConnectionConfig​

Configuration for a third-party connection.

Connections allow workloads (jobs, workspaces, etc.) to access external services like Databricks Unity Catalog. Each connection is identified by its type and name.

This feature is in beta preview. Contact Anyscale support to request enablement.

Fields​

  • type (ConnectionType): The type of connection (e.g., DATABRICKS).
  • name (str): The name of the connection as registered in the organization settings.

Python Methods​

def __init__(self, **fields) -> ConnectionConfig
    """Construct a model with the provided field values set."""

def options(self, **fields) -> ConnectionConfig
    """Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

connections:
  - type: databricks
    name: my-databricks-connection

ConnectionType​

Type of third-party connection.

Values​

  • DATABRICKS: Databricks connection for Unity Catalog access

JobState​

Current state of a job.

Values​

  • STARTING: The job is being started and is not yet running.
  • RUNNING: The job is running. A job will have state RUNNING if a job run fails and there are remaining retries.
  • FAILED: The job did not finish running or the entrypoint returned an exit code other than 0 after retrying up to max_retries times.
  • SUCCEEDED: The job finished running and its entrypoint returned exit code 0.
  • UNKNOWN: The CLI/SDK received an unexpected state from the API server. In most cases, this means you need to update the CLI.

JobStatus​

Current status of a job.

Fields​

  • id (str): Unique ID of the job (generated when the job is first submitted).
  • name (str): Name of the job. Multiple jobs can be submitted with the same name.
  • state (str | JobState): Current state of the job.
  • config (JobConfig): Configuration of the job.
  • runs (List[JobRunStatus]): List of job run states.
  • creator_id (str): ID of the user who created the job.
  • created_at (datetime | None): Timestamp when the job was created.
  • updated_at (datetime | None): Timestamp when the job was last updated.
  • status_updated_at (datetime | None): Timestamp when the job's status was last updated.

Python Methods​

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

import anyscale
from anyscale.job.models import JobStatus
status: JobStatus = anyscale.job.status(name="my-job")

JobRunStatus​

Current status of an individual job run.

Fields​

  • name (str): Name of the job run.
  • state (str | JobRunState): Current state of the job run.

Python Methods​

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

import anyscale
from anyscale.job.models import JobRunStatus
run_statuses: List[JobRunStatus] = anyscale.job.status(name="my-job").runs

JobRunState​

Current state of an individual job run.

Values​

  • STARTING: The job run is being started and is not yet running.
  • RUNNING: The job run is running.
  • FAILED: The job run did not finish running or the entrypoint returned an exit code other than 0.
  • SUCCEEDED: The job run finished running and its entrypoint returned exit code 0.
  • UNKNOWN: The CLI/SDK received an unexpected state from the API server. In most cases, this means you need to update the CLI.

JobLogMode​

Mode to use for getting job logs.

Values​

  • HEAD: Fetch logs from the start of the job's log.
  • TAIL: Fetch logs from the end of the job's log.