Skip to main content

Configure Anyscale Service

Check your docs version

This version of the Anyscale docs is deprecated. Go to the latest version for up to date information.

note

Use of Anyscale Services requires Ray 2.3+.

Anyscale Service YAML Schema​

Services need to be defined in a YAML file like this:

name: my-first-service
project_id: prj_7S7Os7XBvO6vdiVC1J0lgj
compute_config: my-cluster-compute # You may specify `compute_config_id` or `cloud` instead
# Alternatively, a one-off compute config
# compute_config:
#   cloud_id: cld_4F7k8814aZzGG8TNUGPKnc
#   region: us-west-2
#   head_node_type:
#     name: head
#     instance_type: m5.large
#   worker_node_types: []
# Or, a cloud
# cloud: my_cloud
cluster_env: my-cluster-env:5 # You may specify `build_id` instead
ray_serve_config:
  http_options:
    request_timeout_s: 5
  applications:
    - name: app1
      route_prefix: "/"
      import_path: app:entrypoint
      runtime_env:
        working_dir: s3://my_bucket/my_service_files.zip
        # If your working_dir is a local directory, you may want to define an upload_path
        # upload_path: path_to_cloud_storage
        # You may also specify other runtime environment properties like `pip` and `env_vars`
        # pip: ./requirements.txt # relative path to the local directory where `anyscale service rollout` is run
        # pip:
        #   - pandas
        #   - torch
        # env_vars:
        #   SECRET: xyz
      deployments: ...
canary_percent: 100 # This field should only specified if you do a manual rollout
rollout_strategy: ROLLOUT # You may specify either ROLLOUT or IN_PLACE
ray_gcs_external_storage_config:
  address: rediss://gcs.clustercfg.memorydb.us-west-2.amazonaws.com:6379
  enable: True
  redis_certificate_path: "/etc/ssl/certs/ca-certificates.crt"
config:
  access:
    use_bearer_token: True

To deploy an Anyscale Service, you must provide the following:

  • (Required) name Name of the Service. Service names must be unique within a Project.

  • (Optional) project_id The id of the Project you want the Service to deploy in. The project_id can be found in the URL by navigating to the project in Anyscale Console UI. If project_id is not specified, the service will use a default project_id associated with the organization.

  • (Optional) compute_config A Compute Config for the cluster the service will run on.

    • On the SDK, this can be specified as compute_config_id or compute_config (a one-off compute config). This is required, and only one of these fields can be specified.
    • On the CLI, you may specify compute_config (the name of the cluster compute config or a one-off) or cloud (the name of an Anyscale cloud) instead for convenience. Both attributes are optional. If you don't specify either attribute, the service uses a default compute config that is associated with the default cloud.

  • (Optional) cluster_env A cluster environment for the cluster the service runs on.

    • On the SDK, this can only be specified as build_id which can be found on the Console UI for Cluster Environments.
    • On the CLI, you may specify cluster_env (the name and version for the cluster environment, colon-separated; if you don't specify a version, the latest will be used). This attribute is optional in the CLI.

  • (Required) ray_serve_config A ray serve config for your serve deployment. The ray_serve_config contains http_options and configurations per application.

    • (Optional) http_options configurations:

      • (Optional) request_timeout_s End-to-end timeout before terminating the request. This configuration value is global to your Ray cluster, and it cannot be updated during runtime. By default, there is no request timeout.

    • (Required) Application configurations:

      • (Optional) name Name for each application, the default name is default. The name per application must be unique.
      • (Optional) route_prefix An application can be called via HTTP at the specified route prefix. It defaults to /. The route prefix for each application must be unique
      • (Required) import_path Path to your top-level Serve deployment.
      • (Optional) runtime_env A runtime environment defines your application code and dependencies.
      • (Optional) deployments A list of deployments. This is optional and allows you to override the @serve.deployment settings specified in the application code. Each entry in this list must include the deployment name, which must match one in the code. If this section is omitted, Serve launches all deployments in the graph with the settings specified in the code.

  • (Optional) canary_percent A manual target percent for this service. If this field is not set, the service will automatically roll out. If set, this should be a number between 0 and 100. The newly created version will have weight canary_percent and the existing version will have 100 - canary_percent. This field will only be respected when rolling out a canary version. Refer to manual rollouts for more information.

  • (Optional) rollout_strategy Strategy for rollout. The ROLLOUT strategy will deploy your Ray Serve configuration onto a newly started cluster, and then shift traffic over to the new cluster. You can manually control the speed of the rollout using the canary_percent configuration. The IN_PLACE strategy will use Ray Serve in place upgrade to update your existing cluster in place. When using this rollout strategy, you may only change the ray_serve_config field. You cannot partially shift traffic or rollback an in place upgrade. In place upgrades are faster and riskier than rollouts, and we recommend only using them for relatively safe changes (for example, increasing the number of replicas on a Ray Serve deployment). Default strategy is ROLLOUT.

  • (Optional) ray_gcs_external_storage_config A config for the Ray GCS to connect to external storage. If populated, head node fault tolerance will be enabled for this service. ray_gcs_external_storage_config will automatically be configured if the cloud was created with the flag --enable-head-node-fault-tolerance during cloud setup or if a Redis cluster was created during cloud register. User-provided config defined in the service.yaml takes precedence over auto-generated one.

    • (Optional) address Address of the Redis endpoint. If TLS is enabled for the Redis cluster, prefix the endpoint with rediss://. This has no effect if enable is set to False.
    • (Optional) enable Flag to enable/disable GCS FT for a service. The default is set to True.
    • (Optional) redis_certificate_path Path to certificate if TLS is enabled for Redis. This path is defaulted to "/etc/ssl/certs/ca-certificates.crt" and points to the public cert within each instance in the cluster. This value should only be updated if the path to the certificate needs to be overridden.

  • (Optional) config Target Service's configuration.

    • (Optional) Access configurations:
      • (Optional) use_bearer_token Flag to enable bearer token for a service. The bearer token can be toggled for a running Service by rolling out a new version. However, it can take up to 5 minutes for the changes to be propagated, so exercise caution when toggling the bearer token for Services in production. The default is set to True.