Skip to main content

Service API Reference (0.26.64)

View Markdown

Service API Reference (0.26.64)

note

This is archived documentation for version 0.26.64. For the current documentation, see current documentation.

Customer-hosted cloud features​

note

Some features are only available on customer-hosted clouds. Reach out to support@anyscale.com for info.

Service CLI​

anyscale service list​

Usage

anyscale service list [OPTIONS]

List services.

Options

  • --service-id/--id: ID of the service to display.
  • --name/-n: Name of the service to display.
  • --cloud: The Anyscale Cloud of this workload; defaults to your org/workspace cloud.
  • --project: Named project to use; defaults to your org/workspace project.
  • --created-by-me: List services created by me only.
  • --state/-s: Filter by service state (repeatable). Allowed: STARTING, RUNNING, UPDATING, ROLLING_OUT, ROLLING_BACK, UNHEALTHY, TERMINATING, TERMINATED, UNKNOWN, SYSTEM_FAILURE
  • --include-archived: Include archived services.
  • --max-items: Max total items (only with --no-interactive).
  • --page-size: Items per page (max 50).
  • --interactive/--no-interactive: Use interactive paging.
  • --sort: Sort by FIELD (prefix with '-' for desc). Allowed: STATUS, NAME, CREATED_AT
  • -v/--verbose: Include full config in JSON output.
  • -j/--json: Emit structured JSON to stdout.

Examples​

$ anyscale service list --state running --sort -created_at

anyscale service deploy​

Usage

anyscale service deploy [OPTIONS] [IMPORT_PATH] [ARGUMENTS]...

Deploy or update a service.

Options

  • -f/--config-file: Path to a YAML config file to deploy. When deploying from a file, import path and arguments cannot be provided. Command-line flags will overwrite values read from the file.
  • -n/--name: Unique name for the service. When running in a workspace, this defaults to the workspace name.
  • --image-uri: Container image to use for the service. This cannot be changed when using --in-place and is exclusive with --containerfile. When running in a workspace, this defaults to the image of the workspace.
  • --registry-login-secret: Name or identifier of the secret containing credentials to authenticate to the docker registry hosting the image. This can only be used when 'image_uri' is specified and the image is not hosted on Anyscale.
  • --containerfile: Path to a containerfile to build the image to use for the service. This cannot be changed when using --in-place and is exclusive with --image-uri.
  • --ray-version: The Ray version (X.Y.Z) to the image specified by --image-uri. This is only used when --image-uri is provided. If you don't specify a Ray version, Anyscale defaults to the latest Ray version available at the time of the Anyscale CLI/SDK release.
  • --compute-config: Named compute configuration to use for the service. This cannot be changed when using --in-place. When running in a workspace, this defaults to the compute configuration of the workspace.
  • -w/--working-dir: Path to a local directory or a remote URI to a .zip file (S3, GS, HTTP) that will be the working directory for the service. The files in the directory will be automatically uploaded to cloud storage. When running in a workspace, this defaults to the current working directory.
  • -e/--exclude: File pattern to exclude when uploading local directories. This argument can be specified multiple times and the patterns will be appended to the 'excludes' list in the config file (if any).
  • -r/--requirements: Path to a requirements.txt file containing dependencies for the service. Anyscale installs these dependencies on top of the image. If you deploy a service from a workspace, the default is to use the workspace dependencies, but specifying this option overrides them.
  • -i/--in-place: Perform an in-place upgrade without starting a new cluster. This can be used for faster iteration during development but is not currently recommended for production deploys. This cannot be used to change cluster-level options such as image and compute config (they will be ignored).
  • --canary-percent: The percentage of traffic to send to the canary version of the service (0-100). This can be used to manually shift traffic toward (or away from) the canary version. If not provided, traffic will be shifted incrementally toward the canary version until it reaches 100. Not supported when using --in-place. This is ignored when restarting a service or creating a new service.
  • --max-surge-percent: Amount of excess capacity allowed to be used while updating the service (0-100). Defaults to 100. Not supported when using --in-place.
  • --env: Environment variables to set for the service. The format is 'key=value'. This argument can be specified multiple times. When the same key is also specified in the config file, the value from the command-line flag will overwrite the value from the config file.
  • --py-module: Python modules to be available for import in the Ray workers. Each entry must be a path to a local directory.
  • --cloud: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

anyscale service status​

Usage

anyscale service status [OPTIONS]

Get the status of a service.

Options

  • -n/--name: Name of the service.
  • -f/--config-file: Path to a YAML config file to read the name from.
  • --cloud: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • -j/--json: Output the status in a structured JSON format.
  • -v/--verbose: Include verbose details in the status.

anyscale service wait​

Usage

anyscale service wait [OPTIONS]

Wait for a service to enter a target state.

Options

  • -n/--name: Name of the service.
  • -f/--config-file: Path to a YAML config file to read the name from.
  • --cloud: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • --project: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • -s/--state: The ServiceState to wait for the service to reach. Defaults to RUNNING.
  • -t/--timeout-s: Timeout to wait for the service to reach the target state. Defaults to 600s (10min).

anyscale service rollback​

Usage

anyscale service rollback [OPTIONS]

Roll back a service.

Options

  • --service-id/--id: ID of service.
  • -n/--name: Name of service.
  • --project-id: Filter by project id.
  • -f/--config-file/--service-config-file: Path to a YAML config file to read the name from. --service-config-file is deprecated, use -f or --config-file.
  • --max-surge-percent: Max amount of excess capacity allocated during the rollback (0-100).

anyscale service terminate​

Usage

anyscale service terminate [OPTIONS]

Terminate a service.

Options

  • --service-id/--id: ID of service.
  • -n/--name: Name of service.
  • --project-id: Filter by project id.
  • -f/--config-file/--service-config-file: Path to a YAML config file to read the name from. --service-config-file is deprecated, use -f or --config-file.

anyscale service archive​

Usage

anyscale service archive [OPTIONS]

Archive a service.

Options

  • --service-id/--id: ID of service.
  • -n/--name: Name of the service.
  • --cloud: The Anyscale Cloud of this workload. If not provided, the organization default will be used.
  • --project: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

Examples​

$ anyscale service archive --name my_service

anyscale service delete​

Usage

anyscale service delete [OPTIONS]

Delete a service.

Options

  • --service-id/--id: ID of service.
  • -n/--name: Name of the service.
  • --cloud: The Anyscale Cloud of this workload. If not provided, the organization default will be used.
  • --project: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

Examples​

$ anyscale service delete --name my_service

Service SDK​

anyscale.service.list​

List services or fetch a single service by ID.

Arguments

  • service_id (str | None) = None: If provided, returns just the service with this ID wrapped in a one-page iterator.
  • name (str | None) = None: Substring to match against the service name.
  • state_filter (List[ServiceState] | List[str] | None) = None: List of states to include. May be ServiceState enums or case-insensitive strings.
  • creator_id (str | None) = None: Filter services by user ID.
  • cloud (str | None) = None: Name of the Anyscale Cloud to search in.
  • project (str | None) = None: Name of the Anyscale Project to search in.
  • include_archived (bool) = False: Include archived services (default: False).
  • max_items (int | None) = None: Maximum total number of items to yield (default: iterate all).
  • page_size (int | None) = None: Number of items to fetch per API request (default: API default).
  • sort_field (str | ServiceSortField | None) = None: Field to sort by (NAME, STATUS, CREATED_AT).
  • sort_order (str | ServiceSortOrder | None) = None: Sort direction (ASC or DESC).

Returns: ResultIterator[ServiceStatus]

Examples​

import anyscale
from anyscale.service.models import ServiceState

# Example: Get the first 50 running services
for svc in anyscale.service.list(max_items=50, state_filter=[ServiceState.RUNNING]):
    print(svc.name)

anyscale.service.deploy​

Deploy a service.

If no service with the provided name is running, one will be created, else the existing service will be updated.

This command is asynchronous, so it always returns immediately.

Returns the id of the deployed service.

Arguments

  • config (ServiceConfig): The config options defining the service.
  • in_place (bool) = False: Perform an in-place upgrade without starting a new cluster. This can be used for faster iteration during development but is not currently recommended for production deploys. This cannot be used to change cluster-level options such as image and compute config (they will be ignored).
  • canary_percent (int | None) = None: The percentage of traffic to send to the canary version of the service (0-100). This can be used to manually shift traffic toward (or away from) the canary version. If not provided, traffic will be shifted incrementally toward the canary version until it reaches 100. Not supported when using --in-place. This is ignored when restarting a service or creating a new service.
  • max_surge_percent (int | None) = None: Amount of excess capacity allowed to be used while updating the service (0-100). Defaults to 100. Not supported when using --in-place.

Returns: str

Examples​

import anyscale
from anyscale.service.models import ServiceConfig

anyscale.service.deploy(
    ServiceConfig(
        name="my-service",
        applications=[
            {"import_path": "main:app"},
        ],
        working_dir=".",
    ),
    canary_percent=50,
)

anyscale.service.status​

Get the status of a service.

Arguments

  • name (str): Name of the service.
  • cloud (str | None) = None: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

Returns: ServiceStatus

Examples​

import anyscale
from anyscale.service.models import ServiceStatus

status: ServiceStatus = anyscale.service.status(name="my-service")

anyscale.service.wait​

Wait for a service to reach a target state.

Arguments

  • name (str): Name of the service.
  • cloud (str | None) = None: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • state (str | ServiceState) = RUNNING: The state to wait for the service to reach.
  • timeout_s (float) = 600: Timeout to wait for the service to reach the target state.

Examples​

import anyscale
from anyscale.service.models import ServiceState

anyscale.service.wait(name="my-service", state=ServiceState.RUNNING)

anyscale.service.rollback​

Rollback to the primary version of the service.

This command can only be used when there is an active rollout in progress. The rollout will be cancelled and the service will revert to the primary version.

This command is asynchronous, so it always returns immediately.

Returns the id of the rolled back service.

Arguments

  • name (str | None): Name of the service. When running in a workspace, this defaults to the workspace name.
  • cloud (str | None) = None: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • max_surge_percent (int | None) = None: Amount of excess capacity allowed to be used while rolling back to the primary version of the service (0-100). Defaults to 100.

Returns: str

Examples​

import anyscale

anyscale.service.rollback(name="my-service")

anyscale.service.terminate​

Terminate a service.

This command is asynchronous, so it always returns immediately.

Returns the id of the terminated service.

Arguments

  • id (str | None) = None: ID of the service.
  • name (str | None) = None: Name of the service. When running in a workspace, this defaults to the workspace name.
  • cloud (str | None) = None: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

Returns: str

Examples​

import anyscale

anyscale.service.terminate(name="my-service")

anyscale.service.archive​

Archive a service.

This command is asynchronous, so it always returns immediately.

Returns the ID of the archived service.

Arguments

  • id (str | None) = None: ID of the service.
  • name (str | None) = None: Name of the service. When running in a workspace, this defaults to the workspace name.
  • cloud (str | None) = None: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

Returns: str

Examples​

import anyscale

anyscale.service.archive(name="my-service")

anyscale.service.delete​

Delete a service.

This command is asynchronous, so it always returns immediately.

Returns the ID of the deleted service.

Arguments

  • id (str | None) = None: ID of the service.
  • name (str | None) = None: Name of the service. When running in a workspace, this defaults to the workspace name.
  • cloud (str | None) = None: The Anyscale Cloud of this workload. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None) = None: Named project to use for the service. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).

Returns: str

Examples​

import anyscale

anyscale.service.delete(name="my-service")

Service Models​

ServiceConfig​

Configuration options for a service.

Fields​

  • name (str | None): Unique name of the service. When running inside a workspace, this defaults to the name of the workspace, else it is required.
  • image_uri (str | None): URI of an existing image. Exclusive with containerfile.
  • containerfile (str | None): The file path to a containerfile that will be built into an image before running the workload. Exclusive with image_uri.
  • compute_config (ComputeConfig | MultiResourceComputeConfig | Dict | str | None): The name of an existing registered compute config or an inlined ComputeConfig or MultiResourceComputeConfig object.
  • working_dir (str | None): Directory that will be used as the working directory for the application. If a local directory is provided, it will be uploaded to cloud storage automatically. When running inside a workspace, this defaults to the current working directory ('.').
  • excludes (List[str] | None): A list of file path globs that will be excluded when uploading local files for working_dir.
  • requirements (str | List[str] | None): A list of pip requirements or a path to a requirements.txt file for the workload. Anyscale installs these dependencies on top of the image. If you run a workload from a workspace, the default is to use the workspace dependencies, but specifying this option overrides them.
  • env_vars (Dict[str, str] | None): A dictionary of environment variables that will be set for the workload.
  • py_modules (List[str] | None): A list of local directories or remote URIs that will be uploaded and added to the Python path.
  • py_executable (str | None): Specifies the executable used for running the Ray workers. It can include arguments as well.
  • cloud (str | None): The Anyscale Cloud to run this workload on. If not provided, the organization default will be used (or, if running in a workspace, the cloud of the workspace).
  • project (str | None): The project for the workload. If not provided, the default project for the cloud will be used (or, if running in a workspace, the project of the workspace).
  • registry_login_secret (str | None): A name or identifier of the secret containing credentials to authenticate to the docker registry hosting the image. This can only be used when 'image_uri' is specified and the image is not hosted on Anyscale.
  • ray_version (str | None): The Ray version (X.Y.Z) specified for this image specified by either an image URI or a containerfile. If you don't specify a Ray version, Anyscale defaults to the latest Ray version available at the time of the Anyscale CLI/SDK release.
  • applications (List[Dict[str, Any]]): List of Ray Serve applications to run. At least one application must be specified. For details, see the Ray Serve config file format documentation: https://docs.ray.io/en/latest/serve/production-guide/config.html
  • query_auth_token_enabled (bool): Whether or not queries to this service is gated behind an authentication token. If True, an auth token is generated the first time the service is deployed. You can find the token in the UI or by fetching the status of the service.
  • http_options (Dict[str, Any] | None): HTTP options that will be passed to Ray Serve. See https://docs.ray.io/en/latest/serve/production-guide/config.html for supported options.
  • grpc_options (Dict[str, Any] | None): gRPC options that will be passed to Ray Serve. See https://docs.ray.io/en/latest/serve/production-guide/config.html for supported options.
  • logging_config (Dict[str, Any] | None): Logging options that will be passed to Ray Serve. See https://docs.ray.io/en/latest/serve/production-guide/config.html for supported options.
  • ray_gcs_external_storage_config (RayGCSExternalStorageConfig | Dict | None): Configuration options for external storage for the Ray Global Control Store (GCS).
    • Only available on customer-hosted clouds.
  • tracing_config (TracingConfig | Dict | None): Configuration options for tracing.
    • Only available on customer-hosted clouds.

Python Methods​

def __init__(self, **fields) -> ServiceConfig
    """Construct a model with the provided field values set."""

def options(self, **fields) -> ServiceConfig
    """Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

name: my-service
applications:
  - import_path: main:app
image_uri: anyscale/image/my-image:1 # (Optional) Exclusive with `containerfile`.
containerfile: /path/to/Dockerfile # (Optional) Exclusive with `image_uri`.
compute_config: my-compute-config:1 # (Optional) An inline dictionary can also be provided.
working_dir: /path/to/working_dir # (Optional) If a local directory is provided, it will be uploaded automatically.
excludes: # (Optional) List of files to exclude from being packaged up for the service.
  - .git
  - .env
  - .DS_Store
  - __pycache__
requirements: # (Optional) List of requirements files to install. Can also be a path to a requirements.txt.
  - emoji==1.2.0
  - numpy==1.19.5
env_vars: # (Optional) Dictionary of environment variables to set in the service.
  MY_ENV_VAR: my_value
  ANOTHER_ENV_VAR: another_value
py_modules: # (Optional) A list of local directories or remote URIs that will be added to the Python path.
  - /path/to/my_module
  - s3://my_bucket/my_module
cloud: anyscale-prod # (Optional) The name of the Anyscale Cloud.
project: my-project # (Optional) The name of the Anyscale Project.
query_auth_token_enabled: True # (Optional) Whether or not to gate this service behind an auth token.
http_options:
  request_timeout_s: 60 # (Optional) Timeout for HTTP requests in seconds. Default is no timeout.
  keep_alive_timeout_s: 60 # (Optional) Timeout for HTTP keep-alive connections in seconds. Default is 5 seconds.
grpc_options:
  port: 50051 # (Optional) Port to listen on for gRPC requests. Default is 9000.
  grpc_servicer_functions: # (Optional) List of functions to expose as gRPC servicers.
    - my_module:my_function
logging_config: # (Optional) Configuration options for logging.
  encoding: JSON # JSON or TEXT.
  log_level: INFO
  enable_access_log: true
tracing_config: # (Optional) Configuration options for tracing.
  enabled: true
  exporter_import_path: my_module:custom_tracing_exporter
  sampling_ratio: 1.0

RayGCSExternalStorageConfig​

Configuration options for external storage for the Ray Global Control Store (GCS).

When external storage is enabled for a service, it will be able to continue serving traffic if the head node goes down. Without external storage, the entire cluster will be restarted.

Fields​

  • enabled (bool): Enable or disable external storage. When False, the other fields are ignored.
  • address (str | None): Address to connect to the redis instance at. Defaults to the cloud-wide configuration.
  • certificate_path (str | None): Path to the TLS certificate file to use for authentication when using secure connections. Defaults to the cloud-wide configuration.

Python Methods​

def __init__(self, **fields) -> RayGCSExternalStorageConfig
    """Construct a model with the provided field values set."""

def options(self, **fields) -> RayGCSExternalStorageConfig
    """Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

name: my-service
applications:
  - import_path: main:app
ray_gcs_external_storage_config:
  enabled: true
  address: http://redis-address:8888
  certificate_path: /etc/ssl/certs/ca-certificates.crt

TracingConfig​

Configuration options for tracing.

Tracing provides a way to collect telemetry on requests that are handled by your Serve application. If tracing is enabled, you can instrument your Serve application and export traces to a specific backend by configuring the exporter_import_path option. This feature is currently experimental.

Fields​

  • enabled (bool): Flag to enable tracing. When True and no exporter_import_path is defined, then the default tracing exporter will be used to export traces to serve logs. When False, the other fields are ignored.
  • exporter_import_path (str | None): Path to exporter function. Should be of the form "module.submodule_1...submodule_n.export_tracing". This is equivalent to "from module.submodule_1...submodule_n import export_tracing. Exporter function takes no arguments and returns List[SpanProcessor].
  • sampling_ratio (float): Ratio of traces to record and export. Reducing the ratio can help with minimizing serving latency and storage overhead.

Python Methods​

def __init__(self, **fields) -> TracingConfig
    """Construct a model with the provided field values set."""

def options(self, **fields) -> TracingConfig
    """Return a copy of the model with the provided field values overwritten."""

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

name: my-service
applications:
  - import_path: main:app
tracing_config:
  enabled: true
  exporter_import_path: "my_module:custom_tracing_exporter"
  sampling_ratio: 1.0

ServiceStatus​

Current status of a service.

Fields​

  • name (str): Unique name of the service.
  • id (str): Unique ID of the service (generated when the service is first deployed).
  • state (ServiceState | str): Current state of the service.
  • query_url (str): URL used to query the service.
  • creator (str | None): Email of the user or entity that created the service.
  • query_auth_token (str | None): Query auth token required to query the service (if any). Must be passed in as a header like: Authorization: Bearer \<token\>.
  • primary_version (ServiceVersionStatus | Dict | None): Primary version of the service. During a rollout, this is the service version that was running prior to the deploy.
  • canary_version (ServiceVersionStatus | Dict | None): Canary version of the service (the one that is being deployed). When there is no rollout ongoing, this will be None.
  • project (str | None): Name of the project that this service version belongs to.

Python Methods​

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

import anyscale
from anyscale.service.models import ServiceStatus

status: ServiceStatus = anyscale.service.status("my-service")

ServiceState​

Current state of a service.

Values​

  • STARTING: The service is starting up.
  • RUNNING: The service is running and healthy.
  • UPDATING: One of the service versions or the networking stack is updating.
  • ROLLING_OUT: The service is rolling out to a new canary version.
  • ROLLING_BACK: The service is rolling back to the primary version.
  • UNHEALTHY: The service has become unhealthy due to an unexpected issue such as a node failure. In most cases these issues should automatically recover.
  • TERMINATING: The service is terminating.
  • TERMINATED: The service is terminated.
  • UNKNOWN: The CLI/SDK received an unexpected state from the API server. In most cases, this means you need to update the CLI.
  • SYSTEM_FAILURE: An unexpected error occurred in the Anyscale stack. Please reach out to customer support immediately.

ServiceVersionStatus​

Current status of a single version of a running service.

Fields​

  • id (str): Unique ID of the service version (not the service).
  • state (ServiceVersionState | str): Current state of the service version.
  • weight (int): Current canary weight of the service version. This is 100 if there is no ongoing rollout.
  • created_at (datetime): Creation time of the service version.
  • config (ServiceConfig | Dict): Configuration of this service version.

Python Methods​

def to_dict(self) -> Dict[str, Any]
    """Return a dictionary representation of the model."""

Examples​

import anyscale
from anyscale.service.models import ServiceStatus, ServiceVersionStatus

status: ServiceStatus = anyscale.service.status("my-service")
primary_version_status: ServiceVersionStatus = status.primary_version

ServiceVersionState​

Current state of a service version.

Values​

  • STARTING: The service version is starting up.
  • UPDATING: The application(s) on the service version are updating due to a config deploy or update.
  • RUNNING: The service version is running and healthy.
  • UNHEALTHY: The service version has become unhealthy due to an unexpected issue such as a node failure. In most cases these issues should automatically recover.
  • TERMINATING: The service version is terminating.
  • TERMINATED: The service version is terminated.
  • UNKNOWN: The CLI/SDK received an unexpected state from the API server. In most cases, this means you need to update the CLI.
  • SYSTEM_FAILURE: An unexpected error occurred in the Anyscale stack. Please reach out to customer support immediately.

ServiceSortField​

Fields available for sorting services.

Values​

  • STATUS: Sort by service status (active first by default).
  • NAME: Sort by service name.
  • CREATED_AT: Sort by creation timestamp.

ServiceSortOrder​

Enum for sort order directions.

Values​

  • ASC: Sort in ascending order.
  • DESC: Sort in descending order.

Service CLI Legacy​

anyscale service rollout Deprecated​

Deprecated

anyscale service rollout has been deprecated. and will be removed on 2025-10-01. Please use anyscale service deploy instead.

Usage

anyscale service rollout [OPTIONS]

[DEPRECATED - use 'deploy' instead] Roll out a service.

Options

  • -f/--config-file/--service-config-file: The path of the service configuration file.
  • --name/-n: Name of service.
  • --version: Version of service.
  • --max-surge-percent: Max amount of excess capacity allocated during the rollout (0-100).
  • --canary-percent: The percentage of traffic to send to the canary version of the service.
  • --rollout-strategy: Strategy for rollout.
  • -i/--in-place: Alias for --rollout-strategy=IN_PLACE.
  • --no-auto-complete-rollout: Do not complete the rollout (terminate the existing version cluster) after the canary version reaches 100%

Service SDK Legacy​

The AnyscaleSDK class must be constructed in order to make calls to the SDK. This class allows you to create an authenticated client in which to use the SDK.

ParamTypeDescription
auth_tokenOptional StringAuthentication token used to verify you have permissions to access Anyscale. If not provided, permissions default to the credentials set for your current user. Credentials can be set by following the instructions on this page: https://console.anyscale.com/credentials

Example

from anyscale import AnyscaleSDK

sdk = AnyscaleSDK()

get_service Legacy​

warning

This command is deprecated. Upgrade to anyscale.service.status.

Get a Service

Parameters

NameTypeDescriptionNotes
service_idstrDefaults to null

Returns ServicemodelResponse

rollback_service Legacy​

warning

This command is deprecated. Upgrade to anyscale.service.rollback.

Rollback a Service

Parameters

NameTypeDescriptionNotes
service_idstrDefaults to null
rollback_service_modelRollbackServiceModel

Returns ServicemodelResponse

rollout_service Legacy​

warning

This command is deprecated. Upgrade to anyscale.service.deploy.

Rollout a service

Parameters

NameTypeDescriptionNotes
apply_service_modelApplyServiceModel

Returns ServicemodelResponse

terminate_service Legacy​

warning

This command is deprecated. Upgrade to anyscale.service.terminate.

Terminate a Service

Parameters

NameTypeDescriptionNotes
service_idstrDefaults to null

Returns ServicemodelResponse

list_services Legacy​

Limited support

This command is not actively maintained. Use with caution.

Parameters

NameTypeDescriptionNotes
project_idoptional strproject_id to filter byDefaults to null
nameoptional strname to filter byDefaults to null
state_filterList[ServiceEventCurrentState]A list of Service states to filter byDefaults to []
archive_statusArchiveStatusThe archive status to filter by. Defaults to unarchived.Defaults to null
creator_idoptional strcreator_id to filter byDefaults to null
cloud_idoptional strcloud_id to filter byDefaults to null
sort_fieldServiceSortFieldIf absent, the default sorting order is 1. status (active first).2. Last updated at (desc). 3. Name (asc).Defaults to null
sort_orderSortOrderIf sort_field is absent, this field is ignored.If absent, this field defaults to ascending.Defaults to null
paging_tokenoptional strDefaults to null
countoptional intDefaults to null

Returns ServicemodelListResponse

Service Models Legacy​

ApplyProductionServiceV2Model Legacy​

DEPRECATED. Please do not use this model directly. Use ApplyServiceModel.

NameTypeDescriptionNotes
namestrName of the Service[default to null]
descriptionstrDescription of the Service[optional] [default to null]
project_idstrId of the project this Service will start clusters in[optional] [default to null]
versionstrA version string that represents the version for this service. Will be populated with the hash of the config if not specified.[optional] [default to null]
canary_percentintA manual target percent for this service. If this field is not set, the service will automatically roll out. If set, this should be a number between 0 and 100. The newly created version will have weight `canary_percent` and the existing version will have `100 - canary_percent`.[optional] [default to null]
ray_serve_configobjectThe Ray Serve config to use for this service. This config defines your Ray Serve application, and will be passed directly to Ray Serve. You can learn more about Ray Serve config files here: https://docs.ray.io/en/latest/serve/production-guide/config.html[default to null]
build_idstrThe id of the cluster env build. This id will determine the docker image your Service is run using.[default to null]
compute_config_idstrThe id of the compute configuration that you want to use. This id will specify the resources required for your ServiceThe compute template includes a `cloud_id` that must be fixed for each service.[default to null]
configServiceConfigTarget Service's configuration[optional] [default to null]
rollout_strategyRolloutStrategyStrategy for rollout. The ROLLOUT strategy will deploy your Ray Serve configuration onto a newly started cluster, and then shift traffic over to the new cluster. You can manually control the speed of the rollout using the canary_percent configuration. The IN_PLACE strategy will use Ray Serve in place upgrade to update your existing cluster in place. When using this rollout strategy, you may only change the ray_serve_config field. You cannot partially shift traffic or rollback an in place upgrade. In place upgrades are faster and riskier than rollouts, and we recommend only using them for relatively safe changes (for example, increasing the number of replicas on a Ray Serve deployment). Default strategy is ROLLOUT.[optional] [default to null]
ray_gcs_external_storage_configRayGCSExternalStorageConfigConfig for the Ray GCS to connect to external storage. If populated, head node fault tolerance will be enabled for this service.[optional] [default to null]
tracing_configTracingConfigConfig for initializing tracing within Anyscale runtime.[optional] [default to null]
auto_complete_rolloutboolFlag to indicate whether or not to complete the rollout after the canary version reaches 100%.[optional] [default to true]
max_surge_percentintMax amount of excess capacity allocated during the rollout (0-100).[optional] [default to null]

ApplyServiceModel Legacy​

This is the model used to apply a Service.

NameTypeDescriptionNotes
namestrName of the Service[default to null]
descriptionstrDescription of the Service[optional] [default to null]
project_idstrId of the project this Service will start clusters in[optional] [default to null]
versionstrA version string that represents the version for this service. Will be populated with the hash of the config if not specified.[optional] [default to null]
canary_percentintA manual target percent for this service. If this field is not set, the service will automatically roll out. If set, this should be a number between 0 and 100. The newly created version will have weight `canary_percent` and the existing version will have `100 - canary_percent`.[optional] [default to null]
ray_serve_configobjectThe Ray Serve config to use for this service. This config defines your Ray Serve application, and will be passed directly to Ray Serve. You can learn more about Ray Serve config files here: https://docs.ray.io/en/latest/serve/production-guide/config.html[default to null]
build_idstrThe id of the cluster env build. This id will determine the docker image your Service is run using.[default to null]
compute_config_idstrThe id of the compute configuration that you want to use. This id will specify the resources required for your ServiceThe compute template includes a `cloud_id` that must be fixed for each service.[default to null]
configServiceConfigTarget Service's configuration[optional] [default to null]
rollout_strategyRolloutStrategyStrategy for rollout. The ROLLOUT strategy will deploy your Ray Serve configuration onto a newly started cluster, and then shift traffic over to the new cluster. You can manually control the speed of the rollout using the canary_percent configuration. The IN_PLACE strategy will use Ray Serve in place upgrade to update your existing cluster in place. When using this rollout strategy, you may only change the ray_serve_config field. You cannot partially shift traffic or rollback an in place upgrade. In place upgrades are faster and riskier than rollouts, and we recommend only using them for relatively safe changes (for example, increasing the number of replicas on a Ray Serve deployment). Default strategy is ROLLOUT.[optional] [default to null]
ray_gcs_external_storage_configRayGCSExternalStorageConfigConfig for the Ray GCS to connect to external storage. If populated, head node fault tolerance will be enabled for this service.[optional] [default to null]
tracing_configTracingConfigConfig for initializing tracing within Anyscale runtime.[optional] [default to null]
auto_complete_rolloutboolFlag to indicate whether or not to complete the rollout after the canary version reaches 100%.[optional] [default to true]
max_surge_percentintMax amount of excess capacity allocated during the rollout (0-100).[optional] [default to null]

ListServiceModel Legacy​

The list model for Services. Please note that this model can be used for both Service v1 and v2. You can use the type field to differentiate between the two.

NameTypeDescriptionNotes
idstr[default to null]
namestr[default to null]
creator_idstrId of the user who created the Service[default to null]
created_atdatetimeTime the Service was created[default to null]
project_idstrId of the project this Service will start clusters in. This configuration cannot be changed after the service is created.[default to null]
current_stateServiceEventCurrentStateThe current state of this service[default to null]
endtimedatetime[optional] [default to null]
typeServiceTypeType of the Service[optional] [default to null]
service_observability_urlsServiceObservabilityUrlsA JSON object with useful urls pointing to Grafana dashboards.[default to null]

ListservicemodelListResponse Legacy​

A list response form the API. Contains a field "results" which has the contents of the response.

NameTypeDescriptionNotes
resultsList[ListServiceModel][default to null]
metadataListResponseMetadata[optional] [default to null]

ProductionServiceV2Model Legacy​

DEPRECATED. Please use ServiceModel with new sdk calls.

NameTypeDescriptionNotes
idstrId of the Service[default to null]
namestrName of the Service[default to null]
descriptionstrDescription of the Service[optional] [default to null]
project_idstrId of the project this Service will start clusters in. This configuration cannot be changed after the service is created.[default to null]
cloud_idstrId of the cloud this Service belongs to, and will launch clusters in. This configuration cannot be changed.[default to null]
creator_idstrId of the user who created the Service[default to null]
created_atdatetimeTime the Service was created[default to null]
hostnamestrThe hostname of the service[default to null]
current_stateServiceEventCurrentStateThe current state of this service[default to null]
goal_stateServiceGoalStatesThe goal state of this service[default to null]
auth_tokenstrToken to use for service auth. To use the token, add it as a header with the key 'Authorization' and the value 'Bearer <token>'[optional] [default to null]
auto_rollout_enabledboolWhether or not the service is using auto rollout[default to null]
versionsList[ProductionServiceV2VersionModel]DEPRECATED. Please use `primary_version` and `canary_version` fields. Active versions of this service, sorted by creation time in ascending order.[default to null]
primary_versionProductionServiceV2VersionModelPrimary version of this service. If the service is terminated, this field refers to the most recently active version.[default to null]
canary_versionProductionServiceV2VersionModelCanary version of this service. Present only if the service is in the `ROLLING_OUT` state.[optional] [default to null]
service_observability_urlsServiceObservabilityUrlsA JSON object with useful urls pointing to Grafana dashboards.[default to null]
base_urlstrThe base url of this service[default to null]
ended_atdatetimeTime the Service was terminated[optional] [default to null]

ProductionServiceV2VersionModel Legacy​

NameTypeDescriptionNotes
idstrId of the Service Version[default to null]
created_atdatetimeTime the version was created[default to null]
weightintThe target percentage of traffic sent to this version. This is a number between 0 and 100.[default to null]
current_weightintThe current percentage of traffic sent to this version. This is a number between 0 and 100.[optional] [default to null]
versionstrThe version string identifier for this version[default to null]
ray_serve_configobject[default to null]
ray_gcs_external_storage_configRayGCSExternalStorageConfigConfig for the Ray GCS to connect to external storage. If populated, head node fault tolerance is enabled for this service.[optional] [default to null]
tracing_configTracingConfigConfig for initializing tracing within Anyscale runtime.[optional] [default to null]
build_idstrThe id of the cluster env build. This id will determine the docker image your Service is run using.[default to null]
compute_config_idstrThe id of the compute configuration that you want to use. This id will specify the resources required for your Service.The compute template includes a `cloud_id` that cannot be updated.[default to null]
production_job_idsList[str]The list of production job ids associated with this service version.[default to null]
current_stateServiceVersionStateThe current state of the service version.[default to null]

Productionservicev2ModelResponse Legacy​

A response from the API. Contains a field "result" which has the contents of the response.

NameTypeDescriptionNotes
resultProductionServiceV2Model[default to null]

RollbackServiceModel Legacy​

The rollback model for Services.

NameTypeDescriptionNotes
max_surge_percentintThe max_surge_percent to use when rolling back.[optional] [default to null]

RolloutStrategy Legacy​

An enumeration.

Possible Values: ['ROLLOUT', 'IN_PLACE']

ServiceConfig Legacy​

NameTypeDescriptionNotes
max_uptime_timeout_secintAuto-termination timeout (in seconds) for target Service to be unconditionally terminated after specified period. Setting this to 0 disables auto-termination (default).[optional] [default to 0]
accessAccessConfigAccess configuration[optional] [default to null]
protocolsProtocolsProtocol setups for the service[optional] [default to null]
env_varsobjectEnvironment variables to set on the service cluster[optional] [default to null]

ServiceEventCurrentState Legacy​

The possible 'current state' values for a service These states may be shown in the UI as the primary state of the service

Possible Values: ['RUNNING', 'UNHEALTHY', 'SYSTEM_FAILURE', 'USER_ERROR_FAILURE', 'STARTING', 'TERMINATING', 'TERMINATED', 'UPDATING', 'ROLLING_OUT', 'ROLLING_BACK']

ServiceGoalStates Legacy​

An enumeration.

Possible Values: ['RUNNING', 'TERMINATED']

ServiceModel Legacy​

This is the Service model returned by the get path.

NameTypeDescriptionNotes
idstrId of the Service[default to null]
namestrName of the Service[default to null]
descriptionstrDescription of the Service[optional] [default to null]
project_idstrId of the project this Service will start clusters in. This configuration cannot be changed after the service is created.[default to null]
cloud_idstrId of the cloud this Service belongs to, and will launch clusters in. This configuration cannot be changed.[default to null]
creator_idstrId of the user who created the Service[default to null]
created_atdatetimeTime the Service was created[default to null]
hostnamestrThe hostname of the service[default to null]
current_stateServiceEventCurrentStateThe current state of this service[default to null]
goal_stateServiceGoalStatesThe goal state of this service[default to null]
auth_tokenstrToken to use for service auth. To use the token, add it as a header with the key 'Authorization' and the value 'Bearer <token>'[optional] [default to null]
auto_rollout_enabledboolWhether or not the service is using auto rollout[default to null]
versionsList[ProductionServiceV2VersionModel]DEPRECATED. Please use `primary_version` and `canary_version` fields. Active versions of this service, sorted by creation time in ascending order.[default to null]
primary_versionProductionServiceV2VersionModelPrimary version of this service. If the service is terminated, this field refers to the most recently active version.[default to null]
canary_versionProductionServiceV2VersionModelCanary version of this service. Present only if the service is in the `ROLLING_OUT` state.[optional] [default to null]
service_observability_urlsServiceObservabilityUrlsA JSON object with useful urls pointing to Grafana dashboards.[default to null]
base_urlstrThe base url of this service[default to null]
ended_atdatetimeTime the Service was terminated[optional] [default to null]

ServiceObservabilityUrls Legacy​

NameTypeDescriptionNotes
service_dashboard_urlstrURL that points to a dashboard with relevant graphs about the entire service.[optional] [default to null]
service_dashboard_embedding_urlstrURL that points to a dashboard with relevant graphs about the entire service for embedding.[optional] [default to null]
serve_deployment_dashboard_urlstrURL that points to a dashboard with relevant graphs about a single deployent or replica of a service.[optional] [default to null]
serve_deployment_dashboard_embedding_urlstrURL that points to a dashboard with relevant graphs about a single deployent or replica of a service for embedding.[optional] [default to null]

ServiceSortField Legacy​

An enumeration.

Possible Values: ['STATUS', 'NAME', 'CREATED_AT']

ServiceType Legacy​

An enumeration.

Possible Values: ['V1', 'V2']

ServiceVersionState Legacy​

The possible 'current state' values for a service version. These states are used in the UI and SDK/CLI.

Possible Values: ['UNKNOWN', 'STARTING', 'UPDATING', 'RUNNING', 'UNHEALTHY', 'SYSTEM_FAILURE', 'TERMINATING', 'TERMINATED']

ServicemodelListResponse Legacy​

A list response form the API. Contains a field "results" which has the contents of the response.

NameTypeDescriptionNotes
resultsList[ServiceModel][default to null]
metadataListResponseMetadata[optional] [default to null]

ServicemodelResponse Legacy​

A response from the API. Contains a field "result" which has the contents of the response.

NameTypeDescriptionNotes
resultServiceModel[default to null]