CLI and SDK

CLI References

Please refer to CLI quickstart on setting up the Anyscale CLI.

anyscale service rollout

Manage the Service rollout.

Please reference Anyscale Service YAML Schema for the supported values in the YAML file.

Options:

• -f, --service-config-file TEXT The path of the Service configuration file.
• -n, --name TEXT Name of service. Overrides the name specified in the configuration file.
• --canary-percent INTEGER The percentage of traffic to send to the canary version of the service. Overrides the canary percent specified in the configuration file.
• --rollout-strategy [ROLLOUT|IN_PLACE] Strategy for rollout. Overrides the rollout strategy specified in the configuration file.
• -h, --help Show this message and exit.

SDK Example

It's a common pattern to auto-deploy your Service as part of your CI/CD pipeline. To get you started, here is sample Python SDK code that rollouts a new version of your Service using your local working directory.

import requests
import time
from anyscale.sdk.anyscale_client.models import *
from anyscale import AnyscaleSDK

def wait_for_service_state(sdk, service_id: str, expected_state: ServiceEventCurrentState, timeout_s = 1200):
    """
    This is a helper method to wait for the Service to enter an expected state.

    It throws an error if the service enters an unexpected error state.
    """
    start = time.time()
    while time.time() - start < timeout_s:
        service = sdk.get_service(service_id).result
        curr_state = service.current_state
        if curr_state == expected_state:
            return
        if expected_state != ServiceEventCurrentState.UNHEALTHY and curr_state == ServiceEventCurrentState.UNHEALTHY:
            raise RuntimeError(
                f"Service entered an unexpected state {curr_state}\n"
                f"Please check the error for {service.name} in the web UI at https://console.anyscale.com/services/{service.id}."
            )
        time.sleep(1)
    curr_state = sdk.get_service(service_id).result.current_state
    raise TimeoutError(
        f"Service did not enter {expected_state} after {timeout_s} seconds."
    )

def main():
    service_name = "<SERVICE_NAME>"
    # IDs can be found on Anyscale Console under Configurations.
    compute_config_id="<COMPUTE_CONFIG_ID>"
    build_id="<CLUSTER_ENV_BUILD_ID>"
    upload_path="<YOUR_REMOTE_BUCKET_PATH>" # example: "s3://anyscale-test-bucket/"

    sdk = AnyscaleSDK()

    services = sdk.list_services(name=service_name).results

    service = services[0]

    apply_model = ApplyProductionServiceV2Model(
            name=service.name,
            project_id=service.project_id,
            compute_config_id=compute_config_id,
            build_id=build_id,
            ray_serve_config={
                "applications": [
                    {
                        "name": "default",
                        "import_path": "serve_hello:entrypoint",
                        "runtime_env": {
                            "working_dir": ".",
                            "upload_path": upload_path,
                        },
                    }
                ]
            },
        )

    print(f"Rolling out Service {service.name}. It may take a few minutes.")

    service = sdk.rollout_service(apply_model).result

    wait_for_service_state(sdk, service.id, ServiceEventCurrentState.RUNNING)

    print(f"Service has entered the running state. View details at https://console.anyscale.com/services/{service.id}")

    # Please note that Service query would succeed only if you are connected to the VPN.
    token = service.auth_token
    base_url = f"https://{service.hostname}"

    # Requests config
    path = "/"
    full_url = f"{base_url}{path}"
    headers = {"Authorization": f"Bearer {token}"}

    resp = requests.get(full_url, headers=headers)

    print(f"Service query returns: {resp.text}")

if __name__ == "__main__":
    main()

info

The script specifies a local working directory ., which will automatically upload your directory to remote storage. Therefore, please ensure you exported credentials with write permissions to your bucket.

SDK References

Please refer to SDK quickstart on setting up the Anyscale SDK.

Below are the SDK methods for Services.

get_service

Get a Service

Parameters

Name	Type	Description	Notes
service_id	str		Defaults to null

Returns ServicemodelResponse

list_services

Parameters

Name	Type	Description	Notes
project_id	optional str	project_id to filter by	Defaults to null
name	optional str	name to filter by	Defaults to null
state_filter	List[ServiceEventCurrentState]	A list of Service states to filter by	Defaults to []
creator_id	optional str	filter by creator id	Defaults to null
sort_field	ServiceSortField	If absent, the default sorting order is 1. status (active first).2. Last updated at (descending). 3. Name (ascending).	Defaults to null
sort_order	SortOrder	If sort_field is absent, this field is ignored. If absent, this field defaults to ascending.	Defaults to null
paging_token	optional str		Defaults to null
count	optional int		Defaults to null

Returns ServicemodelListResponse

rollback_service

Rollback a Service

Parameters

Name	Type	Description	Notes
service_id	str		Defaults to null

Returns ServicemodelResponse

rollout_service

Rollout a service

Parameters

Name	Type	Description	Notes
apply_service_model	ApplyServiceModel

Returns ServicemodelResponse

terminate_service

Terminate a Service

Parameters

Name	Type	Description	Notes
service_id	str		Defaults to null

Returns ServicemodelResponse